Automated Inference of Social Anxiety From Behavior in Social Virtual Reality: Cross-Sectional Observational Study.

Background
Social anxiety often manifests through behaviors such as reduced gaze to the eyes and lower speech volume. While these markers have been examined in face-to-face interactions, large-scale assessments remain challenging. Social virtual reality (VR) offers a promising alternative by enabling naturalistic interactions in which behavior can be captured at scale, but it remains unclear if people show naturalistic behavior in such artificial environments.Objective
We examined whether behavioral and physiological markers associated with social anxiety in real-life interactions similarly emerge in dyadic social VR. We additionally examined whether these patterns overlap with patterns linked to the broader constructs of psychopathology and verticality.Methods
In this cross-sectional study, 128 participants (105 females, 22 males, 1 diverse; age 18-51 years; mean age 22.60, SD 3.57 years), recruited from a university student population, engaged in a 30-minute avatar-mediated dyadic conversation in social VR while physically located in separate rooms. We assessed gaze toward the partner's eyes, smiling, and speaking behavior by using the VR headsets' built-in eye trackers, face trackers, and microphones, and assessed high-frequency heart rate variability (HF-HRV) by using an electrocardiogram.Results
Relationships between traits and behavioral and physiological measures were analyzed using linear mixed-effects models (α=.05). Social anxiety was linked to reduced gaze toward the partner's eyes while speaking (β=-.20, 95% CI -0.35 to -0.04; t126=-2.51; P=.01), quieter speech (β=-.18, 95% CI -0.35 to -0.01; t126=-2.12; P=.04), and reduced HF-HRV (β=-.23, 95% CI -0.39 to -0.08; t119=-3.00; P=.003). These findings were not entirely specific to social anxiety, as Pearson correlations revealed similar patterns for social anxiety and psychopathology (r=0.94, 95% CI 0.75-0.99; t7=7.61; P<.001), whereas verticality was linked to an overall opposite pattern (social anxiety: r=-0.92, 95% CI -0.98 to -0.65; t7=-6.14; P<.001; psychopathology: r=-0.83, 95% CI -0.96 to -0.38; t7=-3.98; P=.005).Conclusions
In dyadic interactions in social VR, social anxiety was associated with behavioral and physiological modulations similar to those observed in face-to-face interactions, indicating heightened social stress and submissiveness even in avatar-mediated communication. Patterns were similar for heightened psychopathology and reversed for verticality, indicating that these traits may lie on a shared social-behavioral spectrum. Extending previous research focused on face-to-face interactions or reactions toward artificial agents displayed in VR, this study is the first to provide a comprehensive account of the behavioral and physiological modulations associated with social anxiety in avatar-based human-human interactions. Since social VR setups allow researchers to assess a rich set of behavioral data as a byproduct of the setup's core functionality, the technique opens novel possibilities to detect social stress, track therapeutic progress, or tailor interventions to individual behavior when interactions take place in VR.