Nonverbal synchrony in virtual reality

English version

Autoři: Yilu Sun ^aff001; Omar Shaikh ^aff002; Andrea Stevenson Won ^aff001
Působiště autorů: Department of Communication, Cornell University, Ithaca, New York, United States of America ^aff001; College of Computing, Georgia Institute of Technology, Atlanta, Georgia, United States of America ^aff002
Vyšlo v časopise: PLoS ONE 14(9)
Kategorie: Research Article
doi: https://doi.org/10.1371/journal.pone.0221803

Souhrn

How might nonverbal synchrony naturally evolve in a social virtual reality environment? And how can avatar embodiment affect how participants coordinate nonverbally with each other? In the following pre-registered between-subjects experiment, we tracked the movements of pairs of users during a collaborative or competitive task in immersive virtual reality. Each conversational partner controlled either a customized avatar body or an abstract cube that responded to their movements. We compared the movements of the actual user pairs between the two conditions, and to an artificial “pseudosynchrony” dataset composed of the movements of randomly combined participant pairs who did not actually interact. We found stronger positive and negative correlations between real pairs compared to pseudosynchronous pairs, providing evidence for naturally occurring nonverbal synchrony between pairs in virtual reality. We discuss this in the context of the relationships between avatar appearance, task success, social closeness and social presence.

Klíčová slova:

Engineering and technology – Human factors engineering – Man-computer interface – Virtual reality – Computer and information sciences – Computer architecture – User interfaces – Biology and life sciences – Psychology – Behavior – Collective human behavior – Interpersonal relationships – Creativity – Anatomy – Musculoskeletal system – Body limbs – Neuroscience – Cognitive science – Cognitive psychology – Social sciences – Sociology – Social research – Medicine and health sciences – Arms – Research and analysis methods – Mathematical and statistical techniques – Statistical methods – Test statistics – Physical sciences – Mathematics – Statistics

Zdroje

1. Tarr B, Slater M, Cohen E. Synchrony and social connection in immersive Virtual Reality. Scientific reports. 2018;8(1):3693. doi: 10.1038/s41598-018-21765-4 29487405

2. LaFrance M, Broadbent M. Group rapport: Posture sharing as a nonverbal indicator. Group & Organization Studies. 1976;1(3):328–333. doi: 10.1177/105960117600100307

3. Bailenson JN, Yee N, Merget D, Schroeder R. The effect of behavioral realism and form realism of real-time avatar faces on verbal disclosure, nonverbal disclosure, emotion recognition, and copresence in dyadic interaction. Presence: Teleoperators and Virtual Environments. 2006;15(4):359–372. doi: 10.1162/pres.15.4.359

4. Lugrin JL, Landeck M, Latoschik ME. Avatar embodiment realism and virtual fitness training. In: Virtual Reality (VR), 2015 IEEE. IEEE; 2015. p. 225–226.

5. Roth D, Lugrin JL, Galakhov D, Hofmann A, Bente G, Latoschik ME, et al. Avatar realism and social interaction quality in virtual reality. In: Virtual Reality (VR), 2016 IEEE. IEEE; 2016. p. 277–278.

6. Garau M, Slater M, Vinayagamoorthy V, Brogni A, Steed A, Sasse MA. The impact of avatar realism and eye gaze control on perceived quality of communication in a shared immersive virtual environment. In: Proceedings of the SIGCHI conference on Human factors in computing systems. ACM; 2003. p. 529–536.

7. Llobera J, Spanlang B, Ruffini G, Slater M. Proxemics with multiple dynamic characters in an immersive virtual environment. ACM Transactions on Applied Perception (TAP). 2010;8(1):3.

8. Equilibrium theory revisited: Mutual gaze and personal space in virtual environments.

9. Gallese V. Embodied simulation: From neurons to phenomenal experience. Phenomenology and the cognitive sciences. 2005;4(1):23–48. doi: 10.1007/s11097-005-4737-z

10. Ramseyer F, Tschacher W. Nonverbal synchrony in psychotherapy: coordinated body movement reflects relationship quality and outcome. Journal of consulting and clinical psychology. 2011;79(3):284. doi: 10.1037/a0023419 21639608

11. Won AS, Perone B, Friend M, Bailenson JN. Identifying anxiety through tracked head movements in a virtual classroom. Cyberpsychology, Behavior, and Social Networking. 2016;19(6):380–387. doi: 10.1089/cyber.2015.0326

12. Recognizing emotion from dance movement: comparison of spectator recognition and automated techniques. International journal of human-computer studies. 2003;59(1-2):213–225. doi: 10.1016/S1071-5819(03)00050-8

13. Balaam M, Fitzpatrick G, Good J, Harris E. Enhancing interactional synchrony with an ambient display. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM; 2011. p. 867–876.

14. Bailenson JN, Swinth K, Hoyt C, Persky S, Dimov A, Blascovich J. The independent and interactive effects of embodied-agent appearance and behavior on self-report, cognitive, and behavioral markers of copresence in immersive virtual environments. Presence: Teleoperators & Virtual Environments. 2005;14(4):379–393.

15. Condon WS, Ogston WD. Sound film analysis of normal and pathological behavior patterns. Journal of nervous and mental disease. 1966. doi: 10.1097/00005053-196610000-00005 5958766

16. LaFrance M. Nonverbal synchrony and rapport: Analysis by the cross-lag panel technique. Social Psychology Quarterly. 1979; p. 66–70. doi: 10.2307/3033875

17. Tickle-Degnen L, Rosenthal R. The nature of rapport and its nonverbal correlates. Psychological inquiry. 1990;1(4):285–293. doi: 10.1207/s15327965pli0104_1

18. Wiltermuth SS, Heath C. Synchrony and cooperation. Psychological science. 2009;20(1):1–5. doi: 10.1111/j.1467-9280.2008.02253.x 19152536

19. Robinson JD. Nonverbal communication and physician–patient interaction. The SAGE handbook of nonverbal communication. 2006; p. 437–459.

20. Bernieri FJ, Reznick JS, Rosenthal R. Synchrony, pseudosynchrony, and dissynchrony: Measuring the entrainment process in mother-infant interactions. Journal of personality and social psychology. 1988;54(2):243. doi: 10.1037/0022-3514.54.2.243

21. Feldman R. Parent–infant synchrony: Biological foundations and developmental outcomes. Current directions in psychological science. 2007;16(6):340–345. doi: 10.1111/j.1467-8721.2007.00532.x

22. Riley MA, Richardson M, Shockley K, Ramenzoni VC. Interpersonal synergies. Frontiers in psychology. 2011;2:38. doi: 10.3389/fpsyg.2011.00038 21716606

23. Hove MJ, Risen JL. It’s all in the timing: Interpersonal synchrony increases affiliation. Social Cognition. 2009;27(6):949–960. doi: 10.1521/soco.2009.27.6.949

24. Rennung M, Göritz AS. Prosocial consequences of interpersonal synchrony. Zeitschrift für Psychologie. 2016. doi: 10.1027/2151-2604/a000252 28105388

25. Tschacher W, Ramseyer F, Koole SL. Sharing the now in the social present: duration of nonverbal synchrony is linked with personality. Journal of personality. 2018;86(2):129–138. doi: 10.1111/jopy.12298 27977847

26. Kendon A. Movement coordination in social interaction: Some examples described. Acta psychologica. 1970;32:101–125. doi: 10.1016/0001-6918(70)90094-6

27. Schmidt RC, Richardson MJ. Dynamics of interpersonal coordination. In: Coordination: Neural, behavioral and social dynamics. Springer; 2008. p. 281–308.

28. Ramseyer F. Synchronized movement in social interaction. In: Proceedings of the 2013 Inputs-Outputs Conference: An Interdisciplinary Conference on Engagement in HCI and Performance. ACM; 2013. p. 2.

29. Altmann U. Investigation of movement synchrony using windowed cross-lagged regression. In: Analysis of Verbal and Nonverbal Communication and Enactment. The Processing Issues. Springer; 2011. p. 335–345.

30. Delaherche E, Chetouani M, Mahdhaoui A, Saint-Georges C, Viaux S, Cohen D. Interpersonal synchrony: A survey of evaluation methods across disciplines. IEEE Transactions on Affective Computing. 2012;3(3):349–365. doi: 10.1109/T-AFFC.2012.12

31. Boker SM, Rotondo JL, Xu M, King K. Windowed cross-correlation and peak picking for the analysis of variability in the association between behavioral time series. Psychological methods. 2002;7(3):338. doi: 10.1037/1082-989X.7.3.338 12243305

32. Rotondo JL, Boker SM. Behavioral synchronization in human conversational interaction. Mirror neurons and the evolution of brain and language. 2002;42:151–162. doi: 10.1075/aicr.42.13rot

33. Bernieri FJ. Coordinated movement and rapport in teacher-student interactions. Journal of Nonverbal behavior. 1988;12(2):120–138. doi: 10.1007/BF00986930

34. Paxton A, Dale R. Frame-differencing methods for measuring bodily synchrony in conversation. Behavior research methods. 2013;45(2):329–343. doi: 10.3758/s13428-012-0249-2 23055158

35. Schmidt R, Morr S, Fitzpatrick P, Richardson MJ. Measuring the dynamics of interactional synchrony. Journal of Nonverbal Behavior. 2012;36(4):263–279. doi: 10.1007/s10919-012-0138-5

36. Ramseyer F, Tschacher W. Movement Coordination in Psychotherapy: Synchrony of Hand Movements is Associated with Session Outcome. A Single-Case Study. Nonlinear dynamics, psychology, and life sciences. 2016;20(2):145–166. 27033131

37. Shaikh O, Sun Y, Won AS. Movement Visualizer for Networked Virtual Reality Platforms. In: 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE; 2018. p. 681–682.

38. Paxton A, Dale R. Argument disrupts interpersonal synchrony; 2013.

39. Tschacher W, Rees GM, Ramseyer F. Nonverbal synchrony and affect in dyadic interactions. Frontiers in psychology. 2014;5:1323. doi: 10.3389/fpsyg.2014.01323 25505435

40. High Fidelity;. https://highfidelity.com/.

41. Facebook Spaces;. https://www.facebook.com/spaces.

42. Yee N, Ducheneaut N, Ellis J. The tyranny of embodiment. Artifact: Journal of Design Practice. 2008;2(2):88–93.

43. Schultze U. Embodiment and presence in virtual worlds: a review. Journal of Information Technology. 2010;25(4):434–449. doi: 10.1057/jit.2010.25

44. Nowak KL, Fox J. Avatars and computer-mediated communication: a review of the definitions, uses, and effects of digital representations. Review of Communication Research. 2018;6:30–53.

45. Yee N, Bailenson JN. Walk a mile in digital shoes: The impact of embodied perspective-taking on the reduction of negative stereotyping in immersive virtual environments. Proceedings of PRESENCE. 2006;24:26.

46. Won AS, Bailenson J, Lee J, Lanier J. Homuncular flexibility in virtual reality. Journal of Computer-Mediated Communication. 2015;20(3):241–259. doi: 10.1111/jcc4.12107

47. Yee N, Bailenson J. The Proteus effect: The effect of transformed self-representation on behavior. Human communication research. 2007;33(3):271–290. doi: 10.1111/j.1468-2958.2007.00299.x

48. Steptoe W, Steed A, Slater M. Human tails: ownership and control of extended humanoid avatars. IEEE Transactions on Visualization & Computer Graphics. 2013;(4):583–590. doi: 10.1109/TVCG.2013.32

49. Kilteni K, Normand JM, Sanchez-Vives MV, Slater M. Extending body space in immersive virtual reality: a very long arm illusion. PloS one. 2012;7(7):e40867. doi: 10.1371/journal.pone.0040867 22829891

50. Won AS, Bailenson JN, Lanier J. Appearance and task success in novel avatars. Presence: Teleoperators and Virtual Environments. 2015;24(4):335–346. doi: 10.1162/PRES_a_00238

51. Ahn SJ, Bostick J, Ogle E, Nowak KL, McGillicuddy KT, Bailenson JN. Experiencing nature: Embodying animals in immersive virtual environments increases inclusion of nature in self and involvement with nature. Journal of Computer-Mediated Communication. 2016;21(6):399–419. doi: 10.1111/jcc4.12173

52. Kondo R, Sugimoto M, Minamizawa K, Hoshi T, Inami M, Kitazaki M. Illusory body ownership of an invisible body interpolated between virtual hands and feet via visual-motor synchronicity. Scientific Reports. 2018;8(1):7541. doi: 10.1038/s41598-018-25951-2 29765152

53. Waltemate T, Gall D, Roth D, Botsch M, Latoschik ME. The Impact of Avatar Personalization and Immersion on Virtual Body Ownership, Presence, and Emotional Response. IEEE transactions on visualization and computer graphics. 2018;24(4):1643–1652. doi: 10.1109/TVCG.2018.2794629 29543180

54. Won AS, Bailenson JN, Janssen JH. Automatic detection of nonverbal behavior predicts learning in dyadic interactions. IEEE Transactions on Affective Computing. 2014;5(2):112–125. doi: 10.1109/TAFFC.2014.2329304

55. Boker SM, Cohn JF, Theobald BJ, Matthews I, Mangini M, Spies JR, et al. Something in the way we move: motion dynamics, not perceived sex, influence head movements in conversation. Journal of Experimental Psychology: Human Perception and Performance. 2011;37(3):874. doi: 10.1037/a0021928 21463081

56. Sun Y, Pandita S, Shaikh O, Kim B, Won AS. Personalized avatars and self-presence. In: 2018 International Society for Presence Research Annual Conference. ISPR; 2018.

57. Peck TC, Doan M, Bourne KA, Good JJ. The Effect of Gender Body-Swap Illusions on Working Memory and Stereotype Threat. IEEE transactions on visualization and computer graphics. 2018;24(4):1604–1612. doi: 10.1109/TVCG.2018.2793598 29543177

58. Nowak KL, Biocca F. The effect of the agency and anthropomorphism on users’ sense of telepresence, copresence, and social presence in virtual environments. Presence: Teleoperators & Virtual Environments. 2003;12(5):481–494.

59. Won AS, Shriram K, Tamir DI. Social Distance Increases Perceived Physical Distance. Social Psychological and Personality Science. 2018;9(3):372–380. doi: 10.1177/1948550617707017

60. Won AS, Bailenson JN, Stathatos SC, Dai W. Automatically detected nonverbal behavior predicts creativity in collaborating dyads. Journal of Nonverbal Behavior. 2014;38(3):389–408. doi: 10.1007/s10919-014-0186-0

61. Schoenherr D, Paulick J, Strauss BM, Deisenhofer AK, Schwartz B, Rubel JA, et al. Identification of movement synchrony: Validation of windowed cross-lagged correlation and-regression with peak-picking algorithm. PloS one. 2019;14(2):e0211494. doi: 10.1371/journal.pone.0211494 30742651

62. Bourke P. Interpolation methods. Miscellaneous: projection, modelling, rendering. 1999;(1).

63. Mori M. The uncanny valley. Energy. 1970;7(4):33–35.

64. Brenton H, Gillies M, Ballin D, Chatting D. The Uncanny Valley: does it exist and is it related to presence. Presence Connect. 2005.

65. Bailenson JN. Protecting Nonverbal Data Tracked in Virtual Reality. JAMA pediatrics. 2018;172(10):905–906. doi: 10.1001/jamapediatrics.2018.1909 30083770

Nonverbal synchrony in virtual reality

Souhrn

Klíčová slova:

Zdroje

PLOS One

Svět praktické medicíny 3/2024 (znalostní test z časopisu)

Kardiologické projevy hypereozinofilií

Střevní příprava před kolonoskopií

Aktuální možnosti diagnostiky a léčby litiáz

Závislosti moderní doby – digitální závislosti a hypnotika