Is it all in the Voice? Computerized Voice Analysis Reveals the Dynamics in the Therapy Room

One of the areas of expertise of the Bar-Ilan Psychotherapy Laboratory is Treatment Course Guidance – providing guidance for therapists throughout the course of the treatment, namely: developing tools to guide therapists, identifying factors that can improve the course of treatment and its outcomes, and examining therapeutic tools and therapeutic approaches that can lead to better treatment gains.

Our clinical research laboratory, led by Prof. Dana Atzil-Slonim, is equipped with a variety of advanced systems, including a multi-feature documentation system that records audio, video, and physiological data session-by-session to create a large high-quality database to be used in empirical psychotherapy research. 

In this study, the first of its kind, the researchers developed an automated system capable of analyzing and defining vocal arousal data using models of affective dynamics and examined whether vocal and affective dynamics between the client and the therapist are related to treatment outcomes.

Background: the vocal channel as a tool for measuring therapeutic dynamics

Our voice is one of our primary means of expression. Other than the content of our words, voice quality and voice dynamics also deliver substantial information about our mental and emotional condition. For example, the volume and tone of voice can indicate our level of arousal or emotion regulation.

Emotion regulation occurs on both the intrapersonal and interpersonal levels. Our need of others to help us regulate our emotions exists from infancy. Babies and children often have difficulties regulating their own emotions, and need another person, a parent or caregiver, to help them relax, with the vocal channel being one of the primary means of doing so. This way, as babies develop, they learn to regulate their emotions more independently.

However also as adults, we often need others to help us regulate our emotions. One of the key goals of psychotherapy is to help clients expand their emotion regulation capacity within the affective dynamics that occur in the therapy room between the clients and the therapist. But can the vocal channel be used to quantify the degree to which the therapist and the client influence one another in order to regulate the client’s affective arousal? And what are the dynamics that contribute to better treatment outcomes?  The current research aimed to answer these questions.

Hypotheses: the relationship between vocal arousal levels and successful therapy

In the present study, the researchers sought to examine the relationship between vocal dynamics between the client and the therapist and successful or poor outcomes of psychotherapy.

The basic hypotheses of this study were:

1. The first hypothesis consisted of two sub-hypotheses relating to arousal levels during therapy sessions:

• The researchers expected to see vocal arousal levels, both the client’s and the therapist’s, decreasing toward their baseline levels ("dampening").
• The researchers expected to see the client’s and the therapist's arousal levels "pulled" towards each other, in a way that also results in dampening.

2. The second hypothesis also included two sub-hypotheses, relating to arousal levels during therapy:

• The researchers expected to see an intensifying dampening trend toward baseline levels throughout the course of therapy.
• In addition, they expected to see the client’s and the therapist's arousal levels “pulled” harder toward each other during the course of therapy.

3. The Third hypothesis: This hypothesis also has two parts, and concerns the relationship between vocal arousal and treatment outcomes:

• The researchers expected to see a connection between intrapersonal dampening in vocal arousal and treatment outcomes.
• In addition, the researchers expected to find a connection between interpersonal dampening in vocal arousal and treatment outcomes.

Data collection and analysis for the study

The data used as the basis for this study was obtained from recordings of weekly therapy sessions with 30 adult clients at the community clinic operating at Bar-Ilan University, over the course of a year. The sample consisted of 279 therapy sessions, held by 24 therapists, 22 of whom were undergoing a clinical postgraduate training program and 2 were undergoing a third-degree training program. Therapy sessions were delivered under the guidance of senior clinicians.

The dominant therapeutic approach at the clinic is a short-term psychodynamic psychotherapy treatment model, and the indicators that served as the basis for the analysis of the raw materials included:

1. Outcome rating scale (ORS): This tool is intended for clients to self-rate their functioning levels across four domains – personal, family, social, and general.
    
2. Vocal arousal (VA): Vocal arousal was measured by the researchers using advanced computerized tools. First, by a special algorithm, speech turns were separated and sorted into the therapist's and client's turns. After that, the vocal arousal level was calculated based on a weighted index of three vocal characteristics - vocal volume, pitch, and quality, using an advanced technological tool developed by researchers at SAL Laboratories in California.

For each of the participants in every therapy session – the client and the therapist - the average vocal arousal level was calculated throughout the conversation, which the researchers referred to as the baseline arousal level. At each speech turn, the researchers identified the speaker's vocal arousal level in the first and last parts of the speech turn.

In order to examine intrapersonal vocal regulation, the researchers examined how the arousal level at each speech turn was influenced by the arousal level of the same speaker during their previous speech turn. In order to examine interpersonal regulation, the researchers examined how the speaker’s arousal level at each speech turn was influenced by the arousal level of the other speaker at the previous speech turn.

In addition, by comparison to each speaker's baseline arousal level, the researchers were able to examine whether the arousal intensity had decreased or increased, and whether it was possible to quantitatively measure how hard the client and the therapist were “pulled” toward their baseline levels.

The results: first research hypothesis – Dampening and pulling effects during therapy sessions

The researchers found that in accordance with their hypothesis, significant intrapersonal "dampening" in vocal arousal was recorded during the therapy session for each speaker. That is, both the client and the therapist tended to "calm" themselves down and progress toward their baseline arousal level. The researchers also discovered that this tendency intensified as vocal arousal deviated farther from the baseline arousal level.

In addition, the researchers identified that in accordance with their hypothesis, the client's and the therapist's vocal arousal levels were "pulled" towards each other, and toward their baseline arousal levels. Here, too, the researchers identified that this mutual effect intensifies as the difference between the two parties’ arousal levels increases.

The study results: second research hypothesis – Dampening and pulling effects throughout the course of treatment

To examine the change in dampening throughout the course of treatment, session data was entered into a computation model.

With regard to the "intrapersonal" plane, as can be seen in Figure 1, a close relationship was found between the session number and the clients’ recorded intrapersonal dampening effect. This proved the researchers' hypothesis that clients’ intrapersonal dampening increases as the treatment progresses.

With regard to the "interpersonal" plane, contrary to the research hypothesis, the researchers discovered that interpersonal dampening actually decreased as the treatment progressed. In Figure 2, we can see the increased intrapersonal dampening effect versus the decreased interpersonal dampening effect as the treatment progressed.

The study results: third research hypothesis

To examine the relationship between treatment outcomes and intrapersonal and interpersonal dampening in vocal affective arousal, the outcomes of the therapeutic sessions were entered into the computational model.

Here, too, differences were identified between the intrapersonal and the interpersonal effects: Contrary to their hypothesis, the researchers found no connection between intrapersonal dampening in vocal affective arousal and treatment outcomes, but they did find a close connection between interpersonal dampening and treatment outcomes.

To investigate this relationship further, the researchers examined intrapersonal dampening in therapy sessions that were rated either as successful or unsuccessful. They found that like their hypothesis, the sessions during which significant dampening was recorded were successful, while sessions during which poorer dampening was recorded were unsuccessful. Figure 3 illustrates the relationship between treatment outcomes and intrapersonal and interpersonal dampening.

The significance of the research findings

The research findings show that intrapersonal and interpersonal dynamics occur simultaneously - that is, the affective regulation mechanism depends on internal (client versus themselves) and external (client versus therapist) resources.

The researchers have shown that as the treatment progresses, the degree of mutual "interpersonal" dampening decreases, and the degree of "intrapersonal" dampening increases. That is, as the therapy progressed, the client became less and less dependent on external regulation by the therapist.

These findings are consistent with modern psychodynamic theories according to which the client utilizes the resources they share with their therapist so that their emotion regulation capabilities gradually intensify as the treatment progresses and eventually they internalize these capabilities and make them “their own”. That is, as the treatment progressed, the client became less and less dependent on external regulation by their therapist.

The close relationship the researchers discovered between the clients’ degree of mutual dampening and symptom improvement illustrates the importance of the affective relationship between the therapist and their client, and the way people use the resources of others to regulate their emotions.

Significant value for public health

The vocal dynamics unveiled in this study – the tendency of clients and therapists to return to their baseline affective arousal level, and yet to be "drawn" toward each other's arousal level and baseline affective arousal level, can be of great value to therapists. These findings support the idea that therapists whose arousal levels are synchronized with those of their clients, who simultaneously downregulate their affective arousal and that of their clients, can better help clients to develop better emotion regulation skills.

Other than the research findings, the very methodology developed by the researchers for this research can be used in future studies to shed light on other aspects of the complex therapist-client relationship and help therapists achieve better outcomes with their clients.