New research developed at MIT's
CSAIL and Quanta Research enables
the visualization of temporal variations
that are too subtle to be seen
by the naked eye alone.
For example, it can reveal the subtle change
of redness in the face due to blood circulation,
so that you can see someone's pulse flow through their face.
Using the signal, we can automatically
extract vital signs such as the heart rate,
without touching the patient.
Our tests show that the results accurately match
that of state of the art solutions,
such as the monitors used in hospitals.
The method can amplify not only color changes,
but also small motions that are impossible to see
in the original video. For example,
we reveal here the artery pulsation in the wrist.
We can also amplify the subtle breathing motion
of this baby, which can be useful for baby
monitoring and SIDS prevention.
Let's have a look at how the method works.
We take a regular video as input.
We first decompose it into multiple spatial scales
using a so-called Laplacian Pyramid.
This allows us to treat differently large-scale
features and small details.
The heart of the method is then
the temporal processing of the resulting pyramid levels.
We extract temporal variations in a band of frequencies.
For example, for a pulse visualization application,
we would extract frequencies around 1Hz,
which is the typical adult pulse at rest.
We then amplify these variations and recombine the video.
The computation is cheap,
and can be performed in real time, as demonstrated here.
The user has a number of sliders to choose the frequency band
and the amplification factor.
In this input high-speed video, the top two strings
of the guitar were struck.
We show two different amplifications of the same input
corresponding to different temporal frequency bands.
The middle video shows amplification around 82.4Hz,
the frequency of an E, and the top string's motion is revealed.
At 110Hz., shown at the bottom,
the second string gets amplified.
This input video is a simple blend between
two still images, taken 15 second apart.
The output video reveals changes in the scene,
even within this short time period.
Finally, in this high-speed video of a DSLR
taking a picture, we manage to amplify the very
subtle vibrations caused by its shutter and flipping mirror.