Relighting a painting from one photo

Thick paint looks different depending on where the light is. A photo flattens that. On the painting page, moving your mouse over a painting, or tilting your phone, moves a light across it, and the ridges of paint catch it.

Done properly, you'd photograph the painting four times, lit from each side, and work out the surface from the differences. I haven't done that yet, so this version guesses the surface from a single photo.

Guessing the surface

The guess is that brighter means higher. It's wrong in places, but on thick paint the highlights usually are the ridges, so it holds up. The shader reads the brightness either side of each pixel to get the slope, and turns that slope into the direction the surface faces.

It does this at two scales and adds them: a small one for the canvas weave and a larger one for the knife marks. The first version used single pixels and the background came out as a staircase. That was the image's compression blocks being lit like relief. Averaging a few samples at each point fixed it.

Keeping the colours true

If you just multiply the colour by the light, the whole painting gets darker or lighter as the light moves, which looks like a bad monitor. So the shading only uses the difference from a flat surface. Flat canvas keeps its real colour, and only the raised paint changes.

float sh = 1. + 1.1 * (dot(n, ld) - ld.z);
gl_FragColor = vec4(a * sh + sp, 1.);

A small specular term, only where the slope is steep, gives the wet look on the thickest strokes.

Costing nothing when still

The light eases towards your pointer and the loop stops once it gets there. The texture is loaded from the same resized copy the page already uses, so there's no second big download. If WebGL isn't there, you just see the photo.