FaceADE is an iOS app developed in Swift (Apple Inc) to analyze facial features and quantify clinically relevant physical measurements using Apple’s TrueDepth camera [25]. This camera tracks projected IR dots to create a real-time 3D digital mesh of the face that retains accuracy in variable light conditions [26,27]. Apple’s ARKit uses this system to create a 1220-point face mesh with the coordinate system based relative to the position of the head [28]. The FaceLandmarks tool was used to draw curvilinear lines between the predetermined vector points that designate specific facial features of interest on the left and right hemiface [26]. Measurements of these curves were then imputed in the 3D space and then displayed within the app (Figure 1). Facial measures include oral commissure position, mouth area, palpebral fissure height (eye height), and eye area. For this feasibility study, only oral commissure metrics were extracted and analyzed. The 16 facial measurements listed in Figure 1 were exported in a CSV file at 15 frames per second. Dynamic facial movement scores are reported as a percentage relative to the neutral expression, which can be set by the user at any point of the recording by pressing the “neutral expression” button. Patient demographic fields are shown in the lower left-hand corner above the “neutral expression” button. These fields allow for the labeling of patient data when they are exported after recording. In the lower right-hand corner, users can toggle on or off the face mesh, and to the right of the “neutral expression” button, there is a screenshot button to capture images while the video is recording. Measurements were captured at 15 frames per second on a tripod-stabilized iPad, and no video smoothing was applied. Measurements and vector coordinates of each frame of the video were collected to quantify movement during dynamic facial expressions. A walk-through video of FaceADE can be found in Multimedia Appendix 1. FaceADE is available for download on GitHub [29] and in the Apple App Store.

Although the entire matrix of the face was tracked and recorded, the facial feature emphasized in this feasibility study was the lower lip commissure position (mm), defined as the distance from the right or left oral commissure to the facial midline along the lower lip vermillion border (Multimedia Appendix 2). Oral commissure movement is one of the most clinically important features of facial movement and is the focus of many surgical procedures for facial paralysis [30].

Institutional review board approval was obtained (Stanford Institutional Review Board 79624), including an approved consent form and enrollment process. Patients aged 18 years and older of any gender were approved to be enrolled at the Stanford Facial Nerve Center. Eligible patients discussed study participation with a study team member in a private clinic room after their clinical encounter was complete. If they wished to participate in the study, patients subsequently completed the informed consent process. Study team members and clinical staff from the Facial Nerve Center served as controls. All patients and controls were assigned a sequential numerical identifier for data deidentification prior to analysis, and only study team members handled data for collection and analysis.

All images of individuals used in the manuscript are of study team members who provided explicit written consent for the use of their images and reviewed all figures prior to submission.

FaceADE videos were stored on encrypted and HIPAA (Health Insurance Portability and Accountability Act)-compliant iPads (iPad Pro; model number A2759; Apple Inc). Data were transferred from the iPads to a HIPAA-compliant server. Once videos were uploaded to the server, they were deleted from the iPads. All patient images were stored on the same secure server, and all data analysis was performed on a password-protected, encrypted, and HIPAA-compliant desktop computer.

Patient videos were obtained at the conclusion of the clinical encounter in a private examination room. After obtaining patient consent, a ruler was used to make a 1-cm mark with a surgical marker on the patient’s forehead. Patients were seated on a stool, and the tripod with the iPad mounted was placed in front of them. Patients were asked to look directly into the camera during recording. Patients were read standardized face examination prompts (Multimedia Appendix 3) and asked to rest their face between each prompt. The video was then uploaded to a HIPAA-compliant server. Videos were used to obtain still frames of the patient’s neutral and full smile expressions selected from a representative video frame of the face after the patient received the face examination prompt from the recording (ie, “please give a big smile”). The still frames were opened in ImageJ and scaled using the 1-cm mark on the skin. The segmented line tool in ImageJ was then used to measure right and left commissure distance along the lower lip vermillion border. To avoid bias in ImageJ measurements and analysis, the videos had the face mesh turned off, and ImageJ data were entered into REDCap (Research Electronic Data Capture; Vanderbilt University) before FaceADE data were recorded. A study flowchart is shown in Multimedia Appendix 4.

To measure the overall agreement between 2D analysis and FaceADE, the measurements from both facial sides and expressions were pooled, and the intraclass correlation coefficient was calculated using a 2-way random-effects model where both patients and the 2 methods were considered to be randomly selected, and absolute agreement was used instead of the consistency of the trend. The reliability estimate of a single rating was generated. To estimate the potential bias between methods, we performed separate Bland-Altman analyses for each cohort. The bias was defined as the difference between every pair of FaceADE and ImageJ measurements. For each cohort, we used a linear mixed model to account for the measurement of both sides of the face from the same patient. The models included the bias as the outcome measure, facial side and expression as fixed effects, and patient as a random effect with random intercept. We calculated the total variance by adding the within- and between-subject variation generated from the model. The 95% limits of agreement were computed as the bias ±1.96√total variance. Standard Bland-Altman plots were created, with the bias plotted against the average of FaceADE and ImageJ measurements. A horizontal line was drawn at the mean difference and 95% limits of agreement. The scatter dots outside the 95% limits of agreement were identified as possible outliers. All analyses were conducted using RStudio (version 2026.01.2+418; Posit PBC).

Our findings highlight the feasibility of using IR- and video-based imaging to measure oral commissure symmetry and movement in healthy controls and patients with facial paralysis. In this pilot study, we aimed to benchmark FaceADE’s output against gold-standard 2D image analysis and test the feasibility of using this app during normal clinic workflow.

FaceADE has several advantages compared to existing options. Previously published approaches require a measurement scale to be set using an assumed average value, such as a presupposed iris diameter of 11.77 mm or interpupillary distance of 63 mm. FaceADE’s 3D mesh overlay is individualized to a patient’s face and rapidly created using native iOS software and device hardware [18]. Using both video and IR inputs, FaceADE extracts important temporal data such as the velocity of a facial expression and may be able to quantify difficulty maintaining a facial expression over time. Overall, video-based analysis may provide a more complete assessment of facial movement in comparison to 2D analysis of selected still frames. FaceADE captures the entire facial expression attempt, including maximum movement, average movement, and expression duration. The high intraclass correlation coefficient between FaceADE and gold-standard distance measures is encouraging.

FaceADE is designed to fit into existing clinical workflows. The videos take under 2 minutes to record, and the app is run on the existing iOS platform without the need for additional hardware. The method does not require key frame selection, manual adjustment of key points at anatomical landmarks, or other post hoc modification of images. It also allows for the capture and local storage of videos and still-frame images. With these data, clinicians may be able to use this technology to track patient facial function and recovery over time. A future patient-facing version may allow patients to collect data and share them with their health care team if integrated with a HIPAA-compliant storage platform and electronic medical record.

This feasibility study focused only on oral commissure measurements; therefore, the accuracy of tracking other facial features will need to be validated in future work. Of note, measurements obtained from ImageJ are 2D, whereas FaceADE measures movement in the x-, y-, and z-axes over time. This may at least partially account for the higher distance output by FaceADE than by ImageJ. The ARKit technology that FaceADE leverages is developed by Apple Inc [28]. The accuracy of this method is currently limited to Apple’s proprietary technology and algorithms. While we observed a strong intraclass correlation, the limits of agreement in the Bland-Altman analysis were rather wide, suggesting the need for deeper subgroup analysis to identify weaknesses within the ARKit used in FaceADE. Disease severity or other patient factors may contribute to a higher error rate, and this will be a focus of future study.

Additionally, the sample size in this feasibility study was too small to evaluate accuracy and full clinical utility. Critically, we must assess the reliability and accuracy of FaceADE across patients with different disease severities (as judged by clinician experts using validated metrics such as the Sunnybrook Facial Grading System or eFACE scale) and facial anatomical characteristics, test-retest scenarios, and other common clinical variables. This will be the focus of a future validation study.

While we focused on oral commissure movement for this study as a proof of concept, FaceADE collects data from 1220 vectors on the face. In theory, this permits an exhaustive analysis of facial movement that includes all major facial expressions routinely captured during clinical evaluation of facial paralysis and will provide deeper insights into the diagnosis and treatment of this condition.

FaceADE offers a feasible and mobile method to use 3D landmarks to track oral commissure movement and symmetry in healthy controls and patients with facial paralysis. This framework could be expanded to provide clinicians with an easy-to-use system to objectively record facial paralysis severity. Video analysis and 3D modeling allow for potentially more accurate and detailed measurements compared to existing technologies.