Features/Faces across the whole video

Faces across the whole video

Face detection samples frames from the whole video instead of its first second, and shows each person from the sharpest frame it found.

facesvideopeopleAIDocumentation
A video open in Gallery with the info panel showing Omar, Leah, Chloe and Nina, each face cut from a different sharp frame of the video

One frame was never enough

A photo has one moment. A video has hundreds. Until now face detection looked at just one of them, the poster frame from the first second or two, picked without regard to who was in it. A grandparent who walked in at second ten was never found, and the people who were found got face crops cut from a small, upscaled poster, often too soft to tell them apart.

Sampled from start to finish

Turn on Detect faces across the whole video under Administration › Settings › Machine Learning › Facial Recognition. Gallery then samples a frame every two seconds and at the start of every new shot, within a budget that stretches the interval for long videos so they are still covered to the end.

The Facial Recognition settings with Detect faces across the whole video turned on, two seconds between frames, scene cuts sampled and a budget of 1800 frames per video

One face per person, from the best frame

The detections are grouped so each person appears once per video, not once per frame. For every person Gallery keeps the sharpest frame it saw, so their face in the People panel, on their person page and in every crop comes from a full-resolution frame instead of the poster.

Your names stay

The setting is off by default, and new videos are scanned this way once it is on. To rescan the videos you already have, run Face Detection with Missing on the Jobs page. A re-scan matches the new faces to the ones you named, so nobody loses their name.

Read the full documentation on GitHub

Ready to run Gallery on your own server? The install takes about five minutes.