New Statistical Models to Understand and Predict Oculometric Data

Abstract of the Project-Team (OculoNimbus)

In vision, the interaction that man has with its environment is realized by a dynamic exploration of visual regions of interest through eye movements. Understanding the mechanisms responsible for this efficient information sampling opens perspectives for innovation in human-computer interaction, either by imitating human visual exploration in robots or by creating "natural" interfaces for humans.
Eye tracking is a technique for monitoring and continuously recording eye movements. This is a widespread measuring device with increased ease of use in ecological contexts. In the future, this device will be used in broader disciplines than the initial ones (cognitive psychology or clinical applications), to go towards cognitive ergonomics or enrichment of multimedia content, to take just the two most emblematic examples from digital society.

Our project aims to develop new statistical tools for the analysis of eye movement and multimodal data. We plan to organize our research effort around four principal themes. The first theme is concerned with the development of statistical models for multimodal and eye movement data. The three other themes concern: (i) spatiotemporal data segmentation into comprehensive cognitive phases, (ii) analysis of spatiotemporal dependencies to explain within and between individual differences, and (iii) modelling ocular fixations with a higher spatial resolution to understand the functional roles of microsaccades in visual perception.

The objectives of this Project-Team are in line with those of Persyval-lab and more specifically with the axis “Advanced Data Mining”.

• Statistical modeling: Point process and Hidden Markov models (Theme 1)

The focus of our theoretical work will be on adapting and improving existing tools from spatial statistics, especially point process models and spatial Markov chains, to the specific challenges of eye movement data, alone, but also coupled with electroencephalographic signals. In addition, we will leverage existing literature on hierarchical modeling to build models able to quantify within and between individual differences. Read more.

• Cognitive aspects

1. Multimodal data to segment scanpaths on cognitive phases (Theme 2: Read more)
2. Spatiotemporal dependencies to explain within and between individual differences (Theme 3: Read more)
3. Functional roles of microsaccades (Theme 4: Read more)

Participants

• GIPSA-lab (#4):  Simon Barthelmé (CR CNRS), Anne Guérin-Dugué (PR UGA), Nathalie Guyader (MCF UGA), Ronald Phlypo (MCF GINP), Steeve Zozor (CR CNRS)
• LIG (#1): Francis Jambon (MCF UGA)
• LJK (#3): Mariane Clausel (MCF UGA, up to August 2017), Jean-François Coeurjolly (MCF UGA, up to August 2016), Jean-Baptiste Durand (MCF INP), Jean Charles Quinton (MCF UGA)
• LPNC (#4): David Alleysson (CR CNRS), Alan Chauvin (MCF UGA), Benoît Lemaire (MCF UGA), David Meary (MCF UGA)
• Associate participant (#1): Jean-François Coeurjolly (PR UQAM-Montreal-Canada, from Septmber 2016)

Phd Thesis

1. Joint analysis of eye-movements and EEGs using coupled hidden Markov

• Starting in October 2015, in the doctoral school MSTII (download)
• Phd student: Brice Olivier
• Supervisors: Jean-Baptiste Durand, Anne Guérin-Dugué

2. Eye movement modelling

• Starting in November 2015, funding by a "AGIR " grant (2015), in the doctoral school ISCE (download)
• Phd student: Camille Breuil
• Supervisors: Nathalie Guyader,  Simon Barthelmé

3. Micro eye movement modelling through multistable perception

• Starting in October 2016, funding by a university grant (2015), in the doctoral school EEATS (download)
• Phd student: Kevin Parisot
• Supervisors: Steeve Zozor, Ronald Plypo, Alan Chauvin

Post-doc

1. Spatial/Computational Statistics

• Starting in December 2015, ending in August 2016
• Post-doc student: Mélisande Albert
• Supervisors: Simon Barthelmé, Jean-François Coeurjolly

Articles

• Spatial point process (associate partner, J.C. Coeurjolly)

Jean-François Coeurjolly, Jesper Møller, Rasmus Waagepetersen. (2017). Palm distributions for log Gaussian Cox processes. Scandinavian Journal of Statistics, Wiley, 44 (1), pp.192-203. <hal-01163672v4>

Jean-François Coeurjolly, Jesper Møller, Rasmus Waagepetersen. (2017). A tutorial on Palm distributions for spatial point processes. International Statistical Review, Wiley, 83 (5), pp.404-420. <hal-01241277v3>

Jean-François Coeurjolly (2017). Median-based estimation of the intensity of a spatial point process. Annals of the Institute of Statistical Mathematics, Springer Verlag, 69 (2), pp.303-331. <hal-01071605v2>

Jean-François Coeurjolly, Frédéric Lavancier. (2017). Parametric estimation of pairwise Gibbs point processes with infinite range interaction. Bernoulli, Bernoulli Society for Mathematical Statistics and Probability, 23 (2), pp.1299-1334. <hal-01092225v3>

Achmad Choiruddin, Jean-François Coeurjolly, Frédérique Letue. (2018). Convex and non-convex regularization methods for spatial point processes intensity estimation. Electronic journal of statistics , Shaker Heights, OH : Institute of Mathematical Statistics, 12 (1), pp.1210-1255. <hal-01484779v2>

• Joint analysis of eye-movements and EEG

Emmanuelle Kristensen, Bertrand Rivet, Anne Guérin-Dugué. (2017). Estimation of overlapped Eye Fixation Related Potentials: The General Linear Model, a more flexible framework than the ADJAR algorithm. Journal of Eye Movement Research, 10 (1):1-27. <hal-01568579>

Emmanuelle Kristensen, Anne Guérin-Dugué, Bertrand Rivet. (2017). Regularization and a general linear model for event-related potential estimation. Behavior Research Methods, 49(6):2255-2274. <hal-01539867>

Aline Frey, Benoit Lemaire, Laurent Vercueil, Anne Guérin-Dugué. (2018). An Eye Fixation-Related Potential Study in Two Reading Tasks: Reading to Memorize and Reading to Make a Decision. Brain Topography, 31(4):640-660. <hal-01741895v1>

Anne Guérin-Dugué, Raphaelle N. Roy, Emmanuelle Kristensen, Bertrand Rivet, Laurent Vercueil, Anna Tcherkassof. (2018). Temporal Dynamics of Natural Static Emotional Facial Expressions Decoding: A Study Using Event- and Eye Fixation-Related Potentials. Frontiers in Psychology, Frontiers, 2018, 9:1190.〈hal-01837209〉

• Eye movement modelling

Jean-Charles Quinton, Laurent Goffart. (2018). A unified dynamic neural field model of goal directed eye-movements. Connection Science, Taylor & Francis, Embodied Neuronal Mechanisms in Adaptive Behaviour, 30 (1), pp.20-52. <hal-01637024>

International conferences

• Joint analysis of eye-movements and EEG

Emmanuelle Kristensen, Raphaëlle N. Roy, Bertrand Rivet, Anna Tcherkassof, Anne Guérin-Dugué. (2017). Analyzing Emotional Facial Expressions’ Neural Correlates Using Event-Related Potentials and Eye Fixation-Related Potentials. 19th European Conference on Eye Movements (ECM 2017), Aug 2017, Wuppertal, Germany. <hal-01577643>

Anne Guérin-Dugué, Benoit Lemaire, Aline Frey. (2017). General Linear Model to isolate higher-level cognitive components from oculomotor factors in natural reading by using EEG and eye-tracking data coregistration. 19th European Conference on Eye Movements (ECM 2017), Aug 2017, Wuppertal, Germany. <hal-01841508>

• Eye movement modelling

Camille Breuil, Simon Barthelme, Nathalie Guyader. (2017). How redundant are luminance and chrominance information in natural scenes?. Vision Sciences Society Annual Meeting, May 2017, St Pete's Beach, United States. <hal-01836541>

Camille Breuil, Simon Barthelme, Nathalie Guyader. (2017). Luminance modulates color detection threshold in natural scenes. International Colour Vision Society Symposium, Aug 2017, Erlangen, Germany. <hal-01836503>

Camille Breuil, Simon Barthelme, Nathalie Guyader. (2017). Luminance modulates color detection threshold in natural scenes. European Conference on Visual Perception, Aug 2017, Berlin, Germany. <hal-01836518>

Brice Olivier, Jean-Baptiste Durand, Anne Guérin-Dugué, Marianne Clausel. (2017). Eye-tracking data analysis using hidden semi-Markovian models to identify and characterize reading strategies. 19th European Conference on Eye Movements (ECM 2017), Aug 2017, Wuppertal, Germany. 2017. <hal-01671224>

• Micro eye movement modelling through multistable perception

Kevin Parisot, Alan Chauvin, Anne Guérin-Dugué, Ronald Phlypo, Steeve Zozor. (2017). Predictable motion on a Necker cube leads to micro-pursuit-like eye movements and affects the dynamics of bistability.. 19th European Conference on Eye Movements (ECM 2017), Aug 2017, Wuppertal, Germany. <hal-01726513>

Kevin Parisot, Alan Chauvin, Anne Guérin-Dugué, Ronald Phlypo, Steeve Zozor. (2017). A multistable gravitational potential approach to fixational eye movements. European Conference on Visual Perception (ECVP 2017), Aug 2017, Berlin, Germany. <hal-01724178>

National conferences

• Joint analysis of eye-movements and EEG

Jean-Baptiste Durand, Anne Guérin-Dugué, Sophie Achard. (2016). Analyse de séquences oculométriques et d'électroencéphalogrammes par modèles markoviens cachés. 48èmes Journées de Statistique, May 2016, Montpellier, France. <hal-01339458>

• Eye movement and EEG

Jean-Charles Quinton, Laurent Goffart. (2016). A neural field model of the dynamics of goal-directed eye movements. Colloque BioComp, Oct 2016, Lyon, France. <hal-01839803>