This page exhibits images from a side project I’ve been working on using computer vision and machine learning techniques to extract data from the frames of films and translate them into artist output. These are not the “click and go” gen AI that we see so much of these days, but rather carefully considered direct extractions and re-interpretations of image data to create interesting representations of the films from which they are processed.

Each section below illustrates the results along with a brief description of the processing steps included in their generation. This is all done with code, not Photoshop.

Simple Average Colour Sequential Imaging

You may have seen these before, but I include them here anyway in case you haven’t because they’re one of the nicest ways to simply visualise a movie start to end.

These are super easy to generate, simply sampling 3000 frames, taking their average colour and plotting them chronologically.

Here are a few of my favourites.

Linear

Radial

Radial cluster-sorted dominant colour and motion intensity

These images take the dominant colours from the “most important” elements in each frame, and the intensity of motion of the frames, and process them such that image distortion and grunge indicate high intensity within the movie frames.

Process:

  • Sampled 3000 frames over the duration of the move

  • Calculated the “most visually important” colour from each sample frame using the “Saturation-Weighted Colour Dominance” method

  • Calculated the intensity of movement in each sampled frame using the Optical Flow Intensity method

  • Applied K-Means Clustering & Perceptual Color Grouping to extract the highest value dominant colours

  • Applied Brightness-Based Hierarchical Sorting to create the colour ramps for CD-like diffraction effect

  • Ran Radial Gradient Synthesis to render the radial CD-like effect to the image

  • Applied an Intensity-Driven Visual Effects Pipeline as follows:

    • Applied Chromatic Aberration algorithm

    • Applied Fluid Distortion algorithm to distort the image based on movie frame intensity

    • Applied Geometric Shearing to distort the image based on movie frame intensity

    • Applied Procedural Grunge Texturing to add grunge to areas with high movie frame intensity

    • Applied a Bloom algorithm

  • Applied Glitch Aesthetics via Random Square Displacement

    • Used Edge-Weighted Stochastic Sampling to get optimal positions

    • Randomly offset and adjusted brightness of squares