This paper describes an interface to operate the computer using hand trajectory. The skeleton information in humans of the Kinect sensor was used for detection of arm posture and hand positions. From the obtained information, the motion of the operator was recognized to reflect a mouse operation. In the mouse operations, the user could decide the position of the cursor with the right hand, and click with the left hand. In addition, we built the system which could choose two displays by changes in the depth position of the hand and the operability of the developed system was examined by the trial experiments.