Research Spotlight: The Impact of Synchronizing Phosphenes on Object Binding in Artificial Vision

John S. Pezaris, PhD, an investigator in the Department of Neurosurgery at Massachusetts General Hospital and an assistant professor of Neurosurgery at Harvard Medical School, is the senior author of a recently published study in IOVS, The Influence of Phosphene Synchrony in Driving Object Binding in a Simulation of Artificial Vision.

What Question Were You Investigating?

Artificial vision generated through devices that restore sight to the blind have not had the patient acceptance expected by scientists working in the field. Our laboratory has performed a series of experiments to understand these disappointing clinical results.

With this study, we wanted to understand the effects of a commonly used design approach for visual prostheses called multiplexing that simplifies the circuitry by rapidly switching a single drive circuit among many electrodes, but in doing so, changes the way visual information is presented to the prosthesis recipient.

This question is important because if multiplexing negatively affects the experience of artificial vision, it may be a contributing factor to low patient acceptance, and prosthesis designers would need to concentrate on non-multiplexed designs instead.

What Methods Did You Use?

Our team used a computerized simulation of artificial vision that allows normal, sighted subjects to experience what it is like to see with a visual prosthesis.  Artificial vision is very different from normal vision; in particular it has much lower resolution.

To simulate artificial vision, our setup presents images through patterns of phosphenes, the pixels of artificial vision, on a computer screen in front of the subject to give a virtual reality impression of having a visual prosthesis.

The simulation is highly responsive, allowing the subject to look around the scene we present, in real-time.

For this experiment, we had sighted subjects use our simulation to read simple sentences presented in small to large text sizes, and we measured their reading speed and accuracy. From their reading performance, we assigned a visual acuity (similar to the Snellen fraction one gets from a reading a regular vision chart, like 20/20 for normal vision).

We then looked for changes in visual acuity when we simulated multiplexed visual prostheses by shifting each phosphene's appearance slightly in time—in our paper we talk about adding temporal noise—or not shifting them at all, for non-multiplexed devices.

What Did You Find?

Visual acuity from reading performance was significantly affected by the phosphene temporal noise of multiplexing, with increasing levels temporal noise leading to lower reading scores and poorer visual acuity.  The effect was equivalent to loss of half a line on a vision chart (e.g., from 20/70 to 20/100).

A drop in performance was also observed when the total processing delay of the simulation was increased without adding temporal noise. This incidental finding was also very interesting, and we will investigate it in the future.

What are the Implications?

We know that artificial vision is deeply different from normal vision, and our observations here suggest that visual prosthesis designs that employ multiplexing to switch one (or a small number) of stimulation circuits among many phosphenes will produce a lower quality visual experience than designs that use a dedicated circuit for each phosphene.

Although the design is more complicated with a non-multiplexed approach, the complexity may be necessary to provide high-fidelity artificial vision.

What are the Next Steps?

Having answered the question of multiplexing, but discovered an effect of device latency that may be even larger, we now will turn to understanding how processing delays within a visual prosthesis affect perception in artificial vision.

Paper Cited:

Meital-Kfir, N., & Pezaris, J. S. (2023). The Influence of Phosphene Synchrony in Driving Object Binding in a Simulation of Artificial Vision. Investigative ophthalmology & visual science, 64(15), 5. https://doi.org/10.1167/iovs.64.15.5