Towards a quantitative data-driven estimation of student engagement in synchronous virtual learning environments

This thesis proposes an analytical model for estimating students’ engagement levels in Synchronous Virtual Learning Environments (SVLEs). The model is grounded in the principles of explainability, integration of data from multiple digital sources, temporal sensitivity, and adaptability to the teaching context. This research aims to contribute to addressing the inherent limitations of SVLEs with respect to the interpretation and monitoring of students’ states throughout the educational experience.
The work is conducted in accordance with the Design Science Research Methodology. Within this framework, a systematic literature review is carried out, encompassing more than 5,700 retrieved studies; legal and ethical considerations associated with this research domain are analysed; 46 digital parameters inherent to SVLEs are identified and standardised; and 9 quantitative models are designed for processing indicators of engagement. These efforts culminate in the proposal of an analytical model for the global estimation of engagement levels in an SVLE. Technological feasibility is materialised through the conceptualisation of a prototype based on the principles of non-invasiveness, explainability, security, and privacy. The evaluation comprises 14 experiments involving 50 real teachers and students.
The results obtained demonstrate that the proposed model is capable of providing engagement estimations that are understandable, explainable, verifiable, and useful, complementing the teacher’s perception and experience. In this regard, the findings indicate that the model can be used to reveal the impact of teaching strategies on engagement, foster greater teacher awareness of students’ states, facilitate adaptive responses to learners’ conditions, and ultimately increase the likelihood of successful educational experiences.
The contributions of this thesis to the state of the art are embodied in the conception of the ENgagement QUAntification (ENQUA) model as an original approach to engagement estimation. This approach is distinguished by its emphasis on explainability, its reliance on a broader variability of data sources for inference, the incorporation of temporal sensitivity as a fundamental property, and the adaptation of its behaviour to the specific needs of the teaching context. The work demonstrates the feasibility of a comprehensible and operational estimation of engagement, capable of complementing teachers’ perceptions and supporting pedagogical decision-making in SVLEs.