Projecting content onto a non-planar surface has many technical elements that can make it tricky. However, once the technological and physical challenges of projecting an image onto a surface are handled (rigging and blending projectors , supplying power, tuning the playback system etc.) it really comes down to content.
Unfortunately most "projection mapping" projects turn into what are essentially slide shows on buildings.
In order to make a significantly interesting projection project, the concept of perspective must be addressed.
Let's start out with something simple, such as a multi-colored cube.
The resulting UV map for a cube typically looks like a cross because that is what you would get if you cut the seams of a box and unwrapped the object.
This map alone is enough to create some animation, but it would be quite challenging to create any movement. If, for example, text were supposed to scroll from the yellow square to the magenta square (which would make sense from the camera's viewpoint) it would have to make an illogical jump on the cube map.
The key is to think and work in 3D.
What if we wanted to created a 3D scene where the box is actually a see through container with things floating inside it? Or what if we wanted the box to be a solid, reflective object with something floating outside it?
For that we would need to create a perspective projection.
For our example cube the process goes like this:
- Create a 3D scene and render it from the eventual viewer's perspective
- Add a new set of UV coordinates to the the cube from the perspective of the camera that rendered the scene
- Apply the rendered image as a texture to the model and bake the texture to a new material on the same mesh with the original UV coordinate (i.e. cube map coordinates)
The resulting cube map will look strange on its own, but when reapplied to the object it wall look perspective-correct.
For example here is the object with a solid green color and the 3D scene (blue spheres) visible. And here is the image with the blue spheres removed but the perspective texture map applied. The only difference is in the first image some default lighting is used by the 3D modeling program.
Applying the texture and the original scene can also be helpful in understanding how projection works as one can see which parts of the 3D scene were mapped to the cube.
This process can be difficult to figure out and tedious to do repeatedly. The project I made explored automated ways to texture bake perspective-correct projections in modeling software.