solvePnP?

In NOMU, players steer a ghost by turning their head, which meant I needed to know exactly where their head was pointing. That is what OpenCV's solvePnP solves.

The problem is called Perspective-n-Point: given a handful of known 3D points on an object, and where those same points land as 2D pixels in the camera image, work backwards to figure out the object's pose, its rotation and position relative to the camera.

For a face, it goes like this: MediaPipe hands me 2D landmarks (corners of the eyes, tip of the nose, edges of the mouth), and I pair them with a rough 3D model of where those features sit on a generic human head. Feed both into solvePnP and it returns the head's rotation, which I convert into yaw, pitch, and roll. Tilt your head left/right, roll changes, and the ghost turns left/right.

The satisfying part was how little of the math I actually needed to understand to use it. OpenCV does the heavy geometric lifting; my job was mostly feeding it clean points and not mixing up my coordinate systems, which I of course did anyway, at least twice.