Stop using lookAt!

2020-10-28 Permalink

The most intuitive way of controlling a 3D camera orientation must be by specifying its yaw, pitch and roll. The first two are usually bound to the horizontal and vertical axes of the input device (a mouse, a controller stick, etc...). It can be coupled with back/forth and left/right (strafing) movement to provide what’s the most common navigation method in virtual 3D environments.

Yet, there seems to be a common theme among novices to use the lookAt function for constructing the view matrix based on the forward direction, while independently calculating left/right vectors for movement. That’s not surprising at all, considering that so many tutorials teach exactly that:[1][2][3][4]

forward = { -sin(yaw)*sin(pitch), cos(yaw)*sin(pitch), -cos(pitch) };
right = { cos(yaw), sin(yaw), 0 };
up = { 0, 0, 1 };
view = lookAt(eye, eye + forward, up);

Direct construction

However, what those tutorial do not mention, is that -forward,[5] right and eye vectors already form three columns of the inverse view matrix. Calling lookAt afterwards is a rather contrived way of reconstructing the values that were already calculated (!):

B = back = { sin(yaw)*sin(pitch), -cos(yaw)*sin(pitch), cos(pitch) };
R = right = { cos(yaw), sin(yaw), 0 };
U = up = cross(B, R);
view = inverse({
	Rx, Ux, Bx, eye.x,
	Ry, Uy, By, eye.y,
	Rz, Uz, Bz, eye.z,
	0,  0,  0,  1
});

Notice that back, right and up are already orthonormal by construction, and accordingly the inverse can be easily computed by transposition:

view = {
	Rx, Ry, Rz, -dot(eye, R),
	Ux, Uy, Uz, -dot(eye, U),
	Bx, By, Bz, -dot(eye, B),
	0,  0,  0,  1
};

What’s even worse—once taught the lookAt method, novices tend to keep the forward vector as their camera state, and try to rotate it in response to user input, instead of recalculating it based off yaw and pitch.[6] I once witnessed^{[link lost]} somebody using rotation matrices to calculate forward and right, just to feed them back into lookAt, without realizing that they could use their rotation matrix directly as their view matrix.

The easy way

As it turns out, the above view matrix can be easily composed of primitive transformations as follows:

view = rotateX(-pitch) * rotateZ(-yaw) * translate(-eye);

Once the view matrix is calculated, the movement vectors can be extracted from its rows directly:

right = { view[0][0], view[0][1], view[0][2] };
back = { view[2][0], view[2][1], view[2][2] };

This method is easy to extend to accommodate other parameters. For example camera roll can be added to the equation:

view = rotateZ(-roll) * rotateX(-pitch) * rotateZ(-yaw) * translate(-eye);

Or the camera can orbit around a center at a given radius (e.g. for 3rd person view):

view = translate(-radius) * rotateX(-pitch) * rotateZ(-yaw) * translate(-center);

Conclusion

lookAt shall be used only sparingly, when the camera and the subject move truly independently of each other. For example a helicopter mounted camera tracking an on-road vehicle comes to mind. However even then, a physically simulated camera with its own inertia may result in more pleasant and natural looking animation.

So please, stop using lookAt!

Footnotes

1. Learn OpenGL - Camera by Joey de Vries.
2. Tutorial 6 : Keyboard and Mouse on www.opengl-tutorial.org.
3. Keyboard Example: Moving the Camera on Lighthouse3d.com Tutorials.
4. Moving in a 3D world - Camera by Jérôme Jouvie.
5. Following graphics convention for dummies the camera is looking towards the negative Z axis, therefore the 3rd column has to be the backward direction.
6. Rotating a lookAt camera by an angle