PhD Position F/M Multimodal workspace capture and inference for XR collaboration

PhD Position F/M Multimodal workspace capture and inference for XR collaboration Download job offer in PDF format Contract type : Fixed-term contract Level of qualifications required : Graduate degree or equivalent Other valued qualifications : Master in Computer Science or equivalent. Fonction : PhD Position Level of experience : Recently graduated About the research centre or Inria department The Inria Rennes - Bretagne Atlantique Centre is one of Inria's eight centres and has more than thirty research teams. The Inria Center is a major and recognized player in the field of digital sciences. It is at the heart of a rich R&D and innovation ecosystem: highly innovative PMEs, large industrial groups, competitiveness clusters, research and higher education players, laboratories of excellence, technological research institute, etc. Context The candidate will integrate the Seamless team at the at the Centre Inria at Rennes University/IRISA. The Seamless team adopts a multidisciplinary approach in virtual/augmented reality (XR), human perception, human-computer interaction, and human factors, prioritizing the user at the center of the XR revolution. Their work addresses three key challenges: enabling smooth transitions between realities by bridging gaps in perception and interaction for a continuous experience; fostering equal collaboration across realities, ensuring shared awareness and uniform interaction capabilities regardless of individual differences; and advancing implicit, precise evaluation of user experience. This PhD is part of the European Horizon Europe project OmenXR, which aims to develop a real-time, multisensory, and human-centered framework for hybrid collaboration in eXtended Reality (XR). To achieve this, OmenXR will first leverage state-of-the-art image-based rendering techniques to enable on-the-fly photorealistic visual fidelity using consumer-grade Mixed Reality (MR) hardware. Additionally, it will create a multisensory experience that anchors all collaborators in a sensory-coherent environment. Finally, OmenXR will utilize real-time reconstruction and understanding to enable novel forms of hybrid collaboration, integrating remote users, teleoperated robots, and intelligent virtual agents. This PhD project will partially focus on those objectives by enabling the capture and inference of physical spaces for AR/VR collaboration. Assignment Augmented Reality (AR) has been extensively investigated as a tool for remote assistance across various application domains, such as industrial maintenance and home support. In this context, a user operating within an AR-enabled system may solicit guidance from remote collaborators who possess visual access to the physical environment. To facilitate effective collaboration, it is imperative that remote users maintain a comprehensive understanding of the physical workspace, a requirement referred to as workspace awareness. Workspace awareness enables the remote collaborators to understand and assess the environment in which the co-located user is located. A wide range of works have explored to present AR workspaces to remote users [Assaf et al.]. Most of the works focuses on static 3D reconstructions [Kumaravel et al.], virtual proxies [Oda et al.], light fields [Mohr et al.] or real-time video feeds [Fages et al.]. All methods present several trade-offs, based on the size of the reconstructed workspace, the freedom that remote users have to navigate through the reconstructed workspace, the fidelity of the reconstructed workspace or the required preparation and instrumentation. While video provides high fidelity with minimal instrumentation, the navigation capability for remote users is limited and only provide a partial view of the workspace. However, due to the difficulty to ensure free exploration for remote collaborators, workspace awareness still remains an open problem, and it is typically supported either by virtual replicas/reconstructions or video feeds [4]. The appearance of 3D Gaussian Splatting (3DGS) methods [Kerbl et al.] [Meuleman et al.], real-time and high-fidelity reconstruction of physical workspaces is nowadays possible, there is still little research on how the online reconstruction process can be executed in real time without any instrumentation and still ensure a smooth collaboration among different actors. In this context, the first objective of the PhD is to evaluate collaborative incremental reconstruction methods to efficiently support workspace awareness even in the context of partial or uncertain reconstructions, and propose interaction methods to incrementally refine the dynamic reconstruction. Furthermore, workspace awareness is not solely limited to the visual modality, a second objective of the PhD is exploring methods to enrich the reconstruction inferring other sensory modalities, notably haptic information, using the capability of novel view synthesis and vision-based scene understanding methods [Park et al.]. In addition to the technical contributions, specific focus will be devoted the perceptual assessment of the reconstructed models to ensure that 3DGS methods do not introduce perceptual biases. Main activities The first three months of the PhD candidate will focus on thorough analysis of the state of the art on asymmetrical collaboration (theoretical) and on the development of a proof-of-concept VR 3DGS-based rendering system using off-the-shelf algorithms and methods. The next twelve months will be focused on reconstruction fidelity assessment and explore perceptual issues of the 3DGS reconstruction (empirical, methodological). The next twelve months will be focused on the multimodal inference. The final six months will be focused on the writing of the manuscript and to the preparation of the defense, while finishing the initiated contributions. Furthemore, the PhD student is also expected to participate on the different activities that will be organized around the project, project meetings and staff exchanges. References • Assaf, R., Mendes, D. and Rodrigues, R. Cues to fast‐forward collaboration: A Survey of Workspace Awareness and Visual Cues in XR Collaborative Systems, In Computer Graphics Forum, vol. 43, no. 2, p. e15066. 2024, doi. • Bai, H., Sasikumar, P., Yang, J., and Billinghurst, M., A User Study on Mixed Reality Remote Collaboration with Eye Gaze and Hand Gesture Sharing. ACM CHI Conference on Human Factors in Computing Systems. 2020, 1-13, doi. • Clark, H. H., and Wilkes-Gibbs, D. Referring as a collaborative process. Cognition 22, 1 (1986), 1–39, 1986, doi. • Dix, A. Computer supported cooperative work: A framework. In Design issues in CSCW. Springer, 9–26. 1994, doi. • Fages, A., Fleury, C., and Tsandilas, T., Understanding Multi-View Collaboration between Augmented Reality and Remote Desktop Users. ACM on Human-Computer Interaction. CSCW2, 2022, 549, 1-27. doi. • Johnson, J. G., Danilo G., Tommy S., Evan S., and Nadir W. Do you really need to know where “that” is? enhancing support for referencing in collaborative mixed reality environments. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1-14. 2021, doi. • Kerbl, B., Kopanas. G, Leimkühler, T., & Drettakis, G., 3D Gaussians for Real-Time Rendering of Radiance Fields, ACM Trans. On Graphics, Volume 42, Issue 4 Article No.: 139, 1-14, 2023 doi. • Kraut, R. E Miller, M. D., and Siegel, J. Collaboration in performance of physical tasks: Effects on outcomes and communication. In Proceedings of the ACM conference on Computer supported cooperative work. ACM, 57–66. 1996, doi. • Kumaravel. B. T., Anderson, F., Fitzmaurice, G., Hartmann, B., and Grossman, T., Loki: Facilitating Remote Instruction of Physical Tasks Using Bi-Directional Mixed-Reality Telepresence.” ACM Symposium on User Interface Software and Technology. 1-13, 2020 doi. • Meuleman, A., Shah, I., Lanvin, A., Kerbl, B., & Drettakis, G. On-the-fly reconstruction for large-scale novel view synthesis from unposed images. ACM Transactions on Graphics (TOG), 44(4), 1-14, 2025, doi. • Mohr, P., Mori, S., Langlotz T., Thomas, B. H., Schmalstieg, D., and Kalkofen, D. Mixed Reality Light Fields for Interactive Remote Assistance. ACM CHI Conference on Human Factors in Computing Systems., 2020, 1–12. doi. • Oda, O., Elvezio, C., Sukan, M., Feiner, S., and Tversky, B., Virtual Replicas for Remote Assistance in Virtual and Augmented Reality. ACM Symposium on User Interface Software and Technology, 405–415, 2015. doi • Oh, C. S., Bailenson, N. J. and Welch, G. F., A Systematic Review of Social Presence: Definition, Antecedents, and Implications. Frontiers on Robotics and AI, 5:114, 2018, doi. • Park, S. Nam, J. Kim, U. Ju and S. Choi, "GenTouchVR: Generating a Touchable Virtual Reality Environment from a Single Image," in IEEE Transactions on Visualization and Computer Graphics, vol. 32, no. 5, pp. 4385-4395, May 2026, doi: 10.1109/TVCG.2026.3680620 • Piumsomboon, T., Dey, A., Ens, B., Lee, G. and Billinghurst, M., The Effects of Sharing Awareness Cues in Collaborative Mixed Reality. Front. Robot. AI, 2019 6:5. doi. Benefits package • Subsidized meals • Partial reimbursement of public transport costs • Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.) • Possibility of teleworking (after 6 months of employment) and flexible organization of working hours • Professional equipment available (videoconferencing, loan of computer equipment, etc.) • Social, cultural and sports events and activities • Access to vocational training Remuneration 2 300€ per month Apply for this position Share • Facebook • Linkedin • Twitter • Email General Information • Theme/Domain : Interaction and visualization Software engineering (BAP E) • Town/city : Rennes • Inria Center : Centre Inria de l'Université de Rennes • Starting date : 2026-10-01 • Duration of contract : 3 years • Deadline to apply : 2026-09-30 Warning : you must enter your e-mail address in order to save your application to Inria. Applications must be submitted online on the Inria website. Processing of applications sent from other channels is not guaranteed. Instruction to apply Please submit your CV, cover letter, and any recommandations online Defence Security : This position is likely to be situated in a restricted area (ZRR), as defined in Decree No. 2011-1425 relating to the protection of national scientific and technical potential (PPST).Authorisation to enter an area is granted by the director of the unit, following a favourable Ministerial decision, as defined in the decree of 3 July 2012 relating to the PPST. An unfavourable Ministerial decision in respect of a position situated in a ZRR would result in the cancellation of the appointment. Recruitment Policy : As part of its diversity policy, all Inria positions are accessible to people with disabilities. Contacts • Inria Team : SEAMLESS • PhD Supervisor : Argelaguet Sanz Fernando / ferran.argelaguet@inria.fr The keys to success We are seeking a motivated and creative individual with a Master’s degree (or equivalent) in Computer Science, Human-Computer Interaction, Virtual Reality, or a closely related field. The ideal candidate will have in-depth knowledge in Computer Graphics, HCI, and/or VR, with a curiosity for machine learning or signal processing (though not mandatory). Due to the highly collaborative and international nature of the project, fluency in English (written and oral) is essential. We also value strong teamwork, adaptability, and a passion for innovation. About Inria Inria, the French national institute for research in digital science and technology, supports the French government in national research and innovation strategies in the digital field, acting as Digital Programs Agency. Inria leads over 300 research and innovation projects with its 3,500 scientists, engineers, and support staff, in partnership with universities and the digital ecosystem (businesses, entrepreneurs, and public stakeholders). Together, we explore strategic fields such as artificial intelligence, cybersecurity, quantum computing, cloud technologies, digital transformation in healthcare, digital twins, and digital technologies for defence. We develop practical solutions such as software, tech startups, partnerships with national companies, and cutting-edge training programmes. Our goal is to drive scientific, technological, and industrial excellence to ensure France’s digital sovereignty.