Visual prosthesis simulator offers a glimpse into the future

In collaboration with their colleagues at the Donders Institute, researchers at the Netherlands Institute for Neuroscience have developed a simulator that enables artificial visual observations for research into the visual prosthesis. This open source tool is available to researchers and offers those who are interested insight into the future application.

Blindness affects approximately forty million people worldwide and is expected to become increasingly common in the coming years. Patients with a damaged visual system can be broadly divided into two groups: those in whom the damage is located in front of or in the photoreceptors of the retina; and those in whom the damage is further along in the visual system. Various retinal prostheses have been developed for the first group of patients in recent years and clinical tests are underway. The problems for the second group are more difficult to tackle.

A potential solution for these patients is to stimulate the cortex. By implanting electrodes in the brain’s visual cortex and stimulating the surrounding tissue with weak electrical currents, tiny points of light known as ‘phosphenes’ can be generated. This prosthesis converts camera input into electrical stimulation of the cortex. In doing so, it bypasses part of the affected visual system and thus allow some form of vision. You could compare it with a matrix sign along the highway, where individual lights form a combined image.

How we can ensure that such an implant can actually be used to navigate the street or read texts remains an important question. Maureen van der Grinten and Antonio Lozano, from Pieter Roelfsema’s group, along with colleagues from the Donder’s Institute, are members of a large European consortium. This consortium is working on a prosthesis that focuses on the visual cerebral cortex. Maureen van der Grinten emphasizes: “At the moment there is a discrepancy between the amount of electrodes we can implant in people and the functionalities we would like to test. The hardware is simply not far enough yet. To bridge this gap, the process is often imitated through a simulation.”

Simulated Phosphene Vision

“Instead of waiting until blind people have received implants, we’re trying to simulate the situation based on the knowledge we have. We can use that as a basis to see how many points of light people need to find a door for example. We call this ‘simulated phosphene vision’. So far this has only been tested with simple shapes: 200 light points that are neatly-orientated, rectangular pixels of equal size on a screen. People can test this with VR glasses, which is very useful, but does not correspond to the actual vision of blind people with a prosthesis.”

“To make our simulation more realistic, we collected a whole load of literature, created and validated models and looked at the extent to which the results correspond to the effects that people reported. It turns out that the dots vary greatly in shape and size depending on the parameters used in the stimulation. You can imagine that if you increase the current, the stimulation in the brain will spread further, hit more neurons and therefore provide a larger bright spot. The location of the electrode also determines the size of the dots. By influencing the various parameters, we looked at how this actually changes what people see.”

Publicly Accessible

“The simulator is currently being used for research in Nijmegen, where they are investigating the impact of eye movements. With this article we hope to offer other researchers the opportunity to use our simulation as well. We would like to emphasize that the simulator is publicly accessible to everyone, with the flexibility to make adjustments where necessary. It is even possible to optimize the simulation using AI, which can assist you in identifying the necessary stimulation for a specific image.”

“We are now also using the simulator to give people an idea of where this research could go and what to expect when the first treatments are carried out in a few years. Using VR glasses we can simulate the current situation with 100 electrodes, which also highlights how limited vision through a prosthesis is: they may be able to find a door, but won’t have the ability to recognize facial expressions. Alternatively, we can show a situation with tens of thousands electrodes and what that will bring us when this technology is developed far enough.’

This video shows an example fragment of the simulator with a normal image (left), an image automatically extracting the edges from the normal image (middle), and the final image with 100 electrodes (right).

Source: eLife