Neural decision boundaries for maximal information transmission

Abstract

We consider here how to separate multidimensional signals into two categories, such that the binary decision transmits the maximum possible information about those signals. Our motivation comes from the nervous system, where neurons process multidimensional signals into a binary sequence of responses (spikes). In a small noise limit, we derive a general equation for the decision boundary that locally relates its curvature to the probability distribution of inputs. We show that for Gaussian inputs the optimal boundaries are planar, but for non-Gaussian inputs the curvature is nonzero. As an example, we consider exponentially distributed inputs, which are known to approximate a variety of signals from natural environment.

Figure 1. Schematic illustration of a hypothetical decision boundary relative to the input probability distribution, shown as a color plot.

In this case, the decision boundary (red solid line) is shown as extending to infinity, but closed contours are also possible. Variations in contours' shape (as illustrated with a dashed curve) not only shift the position of the decision boundary relatively to the input probability distribution, but also change the overall length of the contour and its infinitesimal arc length element.

Figure 2. Comparison of noise entropies for straight line solutions (1) and circles with spiking on the outside (2) or inside (2

Figure 3. Optimal solutions for 2D exponential inputs:

(A) closed “stretched circle” solutions are shown for λ = −0.3,−0.9,−0.99, numbers correspond to the increasing size of the curved segment throughout this legend. (B) Extended solutions symmetric around y = x line are shown for λ  = 0,−0.25,−0.5,−0.75. (C) Extended solutions symmetric around x = 0 line are shown for λ = 0,−0.5,−0.75 [this type turned out to be suboptimal, albeit by a small margin, compared to either A or B, cf. Figure 4].

Figure 4. Noise entropy H along various decision boundaries with exponential inputs:

“stretched circles” with spiking outside A or inside A′, extended solutions B and C; straight lines parallel to coordinate axes (1) and at ±π/4 angle (2). Solutions (A-C) and (1) satisfy the optimality Eq. (9), but not (2), which is optimal only at a single point at p = 1/2 where it becomes part of family of extended solutions B. Inset shows that switching occurs between solutions A and B.