Reconstructing a height map from a normal map

2014-08-20 Permalink

At times it happens that we have the normal map but not the height/displacement map that was used to create it. For some rendering techniques, like parallax mapping, it is the height map which is needed. In addition, having the height map is beneficial for editing the normal map.

Recovering the height map boils down to integrating the normal map. This is what is suggested in multiple places around the web (e.g. here, here and here), but it is not explained how to do this in practice.

The problem is that the normal map at hand is typically encoded with low precision, 8-bits per channel usually, and might be compressed with a lossy algorithm afterwards. In the end, the line integrals along closed curves aren’t zero, and so the end result significantly depends on the curves along which we integrate. In fact, no matter how we integrate, we end up with significant noise in the output.

Through deconvolution

Denote by f(x, y) the height map and n(x, y) = (n_x, n_y, 1) the normal map that was calculated by some finite difference scheme, that is by convolving the original height map with the kernels d_x and d_y. For the sake of boundary condition, we assume that both f and n are periodic of periods w and h. We seek a function f which minimizes:

Taking the Fourier transform F, D_x, D_y, N_x, N_y of f, d_x, d_y, n_x, n_y respectively, and applying Parseval's theorem:

The last expression can be minimized by each F(x,y) independently. The minimum is achieved by:

For when |D_x|² + |D_y|² = 0, E is independent of F(x,y), and so we can choose any value of the later. In such cases we seek the minimum norm solution, which is achieved by setting F = 0 at these points.

Applying the inverse Fourier transform to F leads to the minimum error, minimum norm solution f.

Implementation

A command line utility implementing the above is available at debump.c.

Sample images