Try this, but don't use a docked figure. Just use GETFRAME the normal way, but switch to software rendering of OPENGL. You should still be able to use transparency.
opengl('software')
#Source: http://www.mathworks.com/matlabcentral/answers/9000-cannot-use-getframe-when-rendering-in-opengl
=========================================================================================
In the line before the getframe command, I inserted this code:
lighting phong
set(gcf,'Renderer','zbuffer')
This seemed to solve the problem completely. I'm not sure why it did, but I tried another option the documentation offered which was:
lighting gouraud
set(gcf,'Renderer','OpenGL')
and this gave the original issue. I have an OpenGL graphics card, so perhaps its the drivers? Matlab doesn't like them?
#Source: http://kr.mathworks.com/matlabcentral/newsreader/view_thread/239590