Properly representing image structure is a major problem faced in developing a computer vision system. This paper proposes a concise descriptor called the shape operator for describing image structure. We first consider an image function as a surface, then establish the shape operator by utilizing surface curvatures. The shape operator can give rich, detailed descriptions in terms of topographical shapes and has invariance properties which are required in computer vision systems. We show usefulness of the shape operator with a texture classification application. Experimental results indicate that the shape operator gives a representation that is sufficiently detailed for characterizing texture images. In particular, it can satisfactorily mitigate the adverse effects of variations in lighting.