Description/Overview:

The "Eye On Line" (EOL) project is supported by the Agence Nationale pour la Recherche (ANR) and aims to develop and test an original device for writing and drawing with the eyes. EOL combines CNRS researchers, clinicians from the Pitié-Salpêtrière Hospital (APHP), the artist Michel Paysant and the Pertech company. The main objective is to provide patients, who do not have the possibility to move and talk, a communication tool based on the mobility of the eyes. The novelty of the under- research device is to allow the direct realization of smooth and continuous eye movements, recorded by a camera near the eye, which then renders the performed path. Based on a perceptual illusion produced by eye movements, this device differs from the current devices which only enable the selection of predefined information presented on a screen. This new approach thus offers a greater freedom of expression, a graphic and an emotional identity: signatures, drawings and soon music, will be produced directly by the user.

The EOL device will be developed over 2 years and a commercially prototype presenting an intuitive utilization and an easy implementation will be born.

A first step aims to train healthy volunteers since the motor skills' acquisition to control eye movements is not immediate. A second step, in partnership with the Department of Neurology of APHP, will propose testing sessions to patients with Amyotrophic Lateral Sclerosis (ALS) - a disease causing progressive paralysis of the limbs and face - to determine whether they are able to control the device and use it to communicate with the surrounding environment.

At the same time, computer developments carried out by Julien Diard at Grenoble (at LNCP, Joint University-CNRS laboratory) will allow automatic recognition of traces as well as the correction of errors or imperfections..

Michel Paysant a contemporary artist working in the hinge art/science and producer of Eye Drawings (see his pictures on this site and his paintings at the Georges Pompidou Museum) explores the possibilities of this new device and in particular its ability for graphical expressiveness. The Pertech company (directed by Jérôme Baujon) develops accurate and lightweight eye tracking devices adapted to the specific constraints of EOL system.

EOL device operation:

The originality of the EOL is to appeal to the users' oculomotor abilities, in particular those of the ocular pursuit used in everyday life to track moving targets (these are for example the movements produced when eyes are following the flight of a bird). With EOL, it is possible to convert the eye into a pencil or a pen, turning it to a real tool for action. The difficulty arises from the need to produce smooth movements – like the movement of a pencil's lead. To achieve this performance, Jean Lorenceau relied on a visual illusion that gives the impression that a static image moves when the eyes are moving. In this way, the movements of the eyes, which are usually unperceptive, produce an illusory movement that allows the ocular pursuit control. However, it should be notice that this transformation of eye movements into a communication tool can be a difficult task and requires a learning process (like surfing or tennis, for example). Demonstrations of the device and a space for practice (demo) can be found in this site.(demo).

In practice, the EOL device consists on a screen that presents the illusion effect and a camera mounted near one of the eyes to record its movement continuously. The computer manages the whole device (visual stimulation and recording of eye movements). If desired, an automatic recognition software enables eye traces' processing - which are sometimes clumsy- into easily readable signs (numbers, letters, words, etc. ..). The system also provides the ability to produce sounds that reflect eye movements, thus allowing production of sound and music.

Global goals

image 2

The EOL project aims to test and develop a writing device using the smooth pursuit eye movements for the production of letters, words and drawings

The scientific and technical obstacles are the follow:

* Optimize the stimulation parameters for the endogenous generation of pursuit eye movements, reducing fatigue and attention load of the subject

* Test a learning protocol that allows a rapid mastery of the device in a population control

* Maximize the expressiveness by integrating the gaze available features (pupil micro movement) and automate the segmentation of the recorded traces, the recognition of words and texts and its restitution using different media supports

* Produce a prototype integrating all functions into a device with a single and lightweight software suitable for routine use in non-stress situations

* Test the device in patients with amyotrophic lateral sclerosis

The work program

image 3

1. Project coordination and implementation of a collaborative website

2. Study of the optimal conditions for driven pursuit control

3. Definition control group

4. Exploration of emotional expressiveness capabilities of the device and the intra- and inter-individual variability

5. Design of alphabets adapted to the intrinsic constraints of the oculomotor system and to the different individual styles of handwriting.

6. Automation of the processes for data acquisition and treatment (segmentation, recognition, rendered graphics)

7. Integration of software features on a single, lightweight and portable equipment

8. Tests with a prototype in patients with amyotrophic lateral sclerosis (ALS).

9. Writing reports, articles, presentations to hospital services and general diffusion

The scientific, technical and economic impact

image 3

The design and implementation of a Driven Pursuit prototype of the EyeOnLine project has scientific, economic and social potential important impacts. In scientific terms, this new paradigm will aid the understanding of oculomotor learning mechanisms and their links with the development of cognitive skills (motor control, attention, emotion, writing, reading). Pursuit Driven may be an innovative way of remediation and cognitive increase in patients who have lost motor function of the upper limbs or in healthy subjects, enabling greater personal expression, action and initiative freedom and an ease communication between patients and their relatives. The Driven Pursuit can be also extended to the artistic activities providing a new form of graphic design based on the intrinsic dynamics of micro-saccades and saccades, smooth pursuit and the pupillary movements. Besides, it can be also used as a method to increase the mastery of eye movements and their use for controlling devices by experts (surgeons, pilots, athletes, ..).