Willy Nolan

A video texture is somewhere between a video and an image. The idea is to create the appearance of an infinitely long video from a fixed-length input video, saving data the process.

The concept is similar to a seamlessly looping video except, in the case of video textures, playback jumps between multiple similar frames providing potentially infinite different iterations of the source video.

The process of creating a video texture starts with a video volume which is essentially a stack of input video frames. The root mean square deviation between every frame against every other frame is computed and stored in a transition matrix. The closer the similarity metric is to zero, the more similar the two frames are.

Next a transition difference matrix is determined by looking at the two frames ahead and behind. This is because even though two frames may be very visually similar, the content before or after may not be. Values in the transition difference matrix are calculated as: $$T_{ij} = \sum_{k=-2}^{2} B * S[i + k, j + k])$$ Where $T$ is the transition matrix $B$ is a binomial filter, $S$ is the similarity matrix and $i$ and $j$ are indexes into $S$.

In practice these matrices could be used to infinitely loop the input video without ever repeating the same transition.

To make the effect even more convincing, techniques such as alignment, blending and brightness correction could be applied.

The program I made finds the longest, smoothest loop and extracts those frames to be made into a video loop.

The featured images show other possible transitions that could be chosen and their associated start and end frames.