Effortlessly create immersive digital twins for training environments with Spatial AI

The IN+SIGHT project simplifies the creation of immersive training environments through 3D digital twins and Spatial AI, making XR-based learning accessible across sectors such as healthcare, emergency services, public infrastructure, cultural heritage, agriculture, and the green transition. By enabling domain experts to independently capture real-world scenarios using smartphones, enrich them with spatially anchored verbal annotations, and convert them into photorealistic 3D models via 3D Gaussian Splatting (3DGS), IN+SIGHT reduces the complexity and cost of XR training development without requiring specialized 3D skills. IN+SIGHT’s approach consists of three key steps: 1. Capturing: Domain experts use smartphones to capture real-world environments and add verbal annotations, creating the foundation for photorealistic 3D models. 2. Authoring: With intuitive editing tools, domain experts enrich the captured data by referencing relevant documents and define learning paths, supported by Conversational AI. 3. Learning: Through Web 4.0-based technical foundations, trainees access the virtual learning scenarios across multiple platforms – mobile devices, PCs, or immersive headsets – ensuring accessibility and flexibility for a broad range of users. Spatial AI gives contextual guidance based on the location within the digital twin. The project employs user-centered design with feedback from evaluations together with use case partners involved in professional training across a wide range of industries and sectors: ÖBB, Samariterbund Linz, Österreichisches Rotes Kreuz NÖ, Bundesverband Rettungsdienst, Samariterbund Österreich, BOKU University, Verein Energiewende, and Belvedere. The R&D partners St. Pölten University of Applied Sciences, AIT, Mopius, and Johannes Ambrosch (Startup) contribute expertise in XR technologies, AI, and software development. IN+SIGHT’s main benefit is allowing organizations to independently produce & continually update customized XR training environments and to roll them out to a broad set of trainees. This is particularly valuable for institutions with significant societal impact. Results and demonstrators will be shared through public events, publications, and a hackathon to encourage broad adoption.