WHAT IS EVENT BASED VISION?

EVENT-BASED VISION FUNCTIONS LIKE THE EYE AND THE BRAIN TO OVERCOME

INHERENT LIMITATIONS OF CONVENTIONAL MACHINE VISION

The human eye shares little with a conventional video camera.

Since their inception 150 years ago, all conventional video tools have represented motion by capturing a number of still frames each second. Displayed rapidly, such images create an illusion of continuous movement. From the flip book to the movie camera, the illusion became more convincing but its basic structure never changed

For a computer, this representation of motion is of little use. The camera is blind between each frame, losing information on moving objects. Even when the camera is recording, each of its “snapshot” images contains no information about the motion of elements in the scene. Worse still, within each image, the same irrelevant background objects are repeatedly recorded, generating excessive unhelpful data. 

Consider a video of a golfer taking a swing

Oversampled: Sky, grass and trees

Undersampled: Motion of golfer, club and ball

Consider a video of a golfer taking a swing. A conventional sensor applies an arbitrary frame rate to the whole scene, let’s say 30 frames per second. The important information is the swing of the club and the movement of the ball but ironically, the sensor will miss segments of this information while repeatedly taking an extensive inventory of the sky, trees and grass behind him.

NATURE’S ELEGANT SOLUTION

Evolution developed an elegant solution so that natural vision never encounters these problems. It doesn’t take frames. Cells in your eye report back to the brain when they detect a change in the scene – an event. If nothing changes, the cell doesn’t report anything. The more an object moves, the more your eye and brain sample it.

This process allows human vision to collect all the information it needs, without wasting time and energy reprocessing images of the unchanging parts of the scene.

By only recording what changes, the eye and brain can gather useful information from things changing at up to 1000 times a second, without needing to engage enormous amounts of brain power. Neither the predator nor the prey has time to waste processing irrelevant information.

This is event-based vision – independent receptors collecting all the essential information, and nothing else. 

EVENT-BASED VISION SYSTEMS PERCEIVE THE VITALITY OF THE SCENE

AND OVERLOOK THE IRRELEVANT

PROPHESEE creates both neuromorphic sensors and bio-inspired algorithms that function like the eye and brain. This holistic approach is a fundamental shift in computer vision – the departure from frame-based sensors, to event-based vision systems, also known as event cameras.

Each pixel only reports when it senses movement. Whereas in a frame-based sensor all pixels record at the same time, in an event-based sensor each pixel is perfectly independent. 

Prophesee event based vision sensor

NOTHING IS LOST BETWEEN THE FRAMES

When each pixel is free to record only when it is triggered, the information created does not arrive frame by frame. Rather, movement is captured as a continuous stream of information. Nothing is lost between frames. 

event based vision vs frame-based vision

The Prophesee sensor records a rotating robotic arm as a continuous stream of movement.

A DISRUPTIVE TECHNOLOGY

Event-based vision systems perceive the vitality of the scene and overlook the irrelevant. They produce up to 1000 times fewer data than a conventional sensor whilst achieving a higher equivalent temporal resolution of >10 000 fps

By bypassing inherent limitations in conventional computer vision, event-based vision is disrupting the current technology in fields such as automotive vehicles, artificial intelligence & deep learning, industrial automation, IoT, security, surveillance, and health-care among others.

 

Download the White Paper to learn more about the fundamental shift happening in machine vision.