A geometric view on early and middle level visual coding

Abstract

As opposed to dealing with the geometry of objects in the 3D world, this paper considers the geometry of the visual input itself, i.e. the geometry of the spatio-temporal hypersurface defined by image intensity as a function of two spatial coordinates and time. The results show how the Riemann curvature tensor of this hypersurface represents speed and direction of motion, and thereby allows to predict global motion percepts and properties of MT neurons. It is argued that important aspects of early and middle level visual coding may be understood as resulting from basic geometric processing of the spatio-temporal visual input. Finally, applications show that the approach can improve the computation of motion.