Machine Learning–Based Audiovisual Phenotyping for Measuring Communication, Shared Decision-Making, and Trust

Shely Khaikin   ¹ , BA ;   Vineet Tiruvadi   ^{2, 3} , MD, PhD ;   Jeffrey Brooks   ³ , PhD ;   Alice Baird   ³ , PhD ;   Anne-Catherine Grela-Mpoko   ¹ , BS, MPH ;   Lindsey Hoffman   ¹ , BS ;   Jadyn Crossley   ¹ ;   Menachem Leasy   ⁴ , MD ;   Jaime Fineman   ⁵ , MD ;   Margot Savoy   ⁶ , MD ;   Laura Igarabuza   ⁴ , MD ;   Anuradha Paranjape   ⁷ , MD ;   Cheryl YS Foo   ⁸ , PhD ;   Michael L Birnbaum   ^{9, 10} , MD ;   Yaara Zisman-Ilani   ^{11, 12, 13} , MA, PhD

¹ Shared Decision Making Laboratory, Temple University, Philadelphia, PA, United States

² Harvard University, Cambridge, MA, United States

³ Hume AI, New York, NY, United States

⁴ Department of Clinical Family and Community Medicine, Lewis Katz School of Medicine, Temple University, Philadelphia, PA, United States

⁵ Department of Clinical Medicine, Lewis Katz School of Medicine, Temple University, Philadelphia, PA, United States

⁶ American Academy of Family Physicians, Washington, DC, United States

⁷ University of Colorado School of Medicine, Aurora, CO, United States

⁸ Center of Excellence for Psychosocial and Systemic Research, Department of Psychiatry, Massachusetts General Hospital, Boston, MA, United States

⁹ New York State Psychiatric Institute, New York, NY, United States

¹⁰ Department of Psychiatry, Vagelos College of Physicians and Surgeons, Columbia University, New York, NY, United States

¹¹ Department of Clinical, Educational and Health Psychology, Division of Psychology and Language Sciences, University College London, London, United Kingdom

¹² Department of Social and Behavioral Sciences, Barnett College of Public Health, Temple University, Philadelphia, PA, United States

¹³ Department of Psychiatry and Behavioral Sciences, Lewis Katz School of Medicine, Temple University, Philadelphia, PA, United States