The Making of Mind Ease: Our Real World Effectiveness
At Mind Ease, we always carefully evaluate our effectiveness. Find out how our Calm Me exercises help many people with their stress and anxiety!
We have previously written about a large study we ran on our app prototype as well as how we validate new content before it goes into the app. However, we don’t stop there! At Mind Ease, we also continuously measure how effective our exercises are for our actual users. That’s what we will explore in this article.
Our main source of information about our real-world effectiveness is through the Calm Me feature. Anyone who has used our app before will know the prominent Calm Me button in the center. This button offers users 12 different exercises that can effectively reduce stress and relieve acute anxiety.
When you click the Calm Me button, you are always asked to evaluate how you currently feel on three different scales:
- from good to bad
- from calm to worried
- and from relaxed to tense
The range of ratings goes from +3 to -3. That means that if you select “Very good”, you will rate your feelings as +3. “Quite good” would be +2, “Somewhat good” would be +1, and so on. You can also rate that you feel “Neither good nor bad”.
Based on your feelings, our app chooses the best exercise for you. After you complete it, we ask you for this rating again – and this is the magical moment at which we can assess the effectiveness of the exercise.
By comparing your feelings before and after, we measure whether the exercise was helpful and how much.
For example, if you rated that you felt “Very worried” (-3), and after your exercise this has reduced only to “Somewhat worried” (-1), we conclude that your negative feelings were reduced by 2 points.
How effective are the Calm Me exercises?
The number of points by which your feelings move from negative to positive is the main metric we use to determine our effectiveness.
For example, in July, August and September 2022, our users completed 3931 Calm Me exercises. Over all, these exercises improved their mood by a total of 16 805 points.
This means that…
Over the last three months, the average Calm Me exercise improved our users feelings by 4.28 points.
Pretty good right! The effectiveness of these exercises isn’t too surprising for us. After all, we only select the most evidence-based techniques that have been validated by years of psychological research.
These are some of our exercises and their supporting evidence:
- Muscle relaxation has been found to be effective in scientific reviews such as this one or this one
- Mindfulness-based meditation exercises are known to have positive effects on anxiety and depression
- Deep breathing practices are tried and tested tools that have positive effects on stress and anxiety
- Gratitude interventions, where you are asked to focus on the positive events in your life to increase your happiness, can also reduce anxiety and depression
As a team, we regularly review how many points of negative feelings our app reduced. Then we celebrate how our app managed to relieve anxiety and stress for so many people!
Improving our effectiveness over time
Great news is that the effectiveness of our exercises has been improving. For example, this year, the average reduction in negative feelings has been around 79%, thanks to introducing a machine-learning recommender that learns which exercises are the best ones for each user.
In contrast, when we measured our app in 2019, we found the reduction in negative feelings to be 30-57% on average. This was based on a study where we recruited 309 participants and asked them to:
- either complete one Calm Me exercise per day for a period of 5 days,
- or complete a control activity, where they were just asked to read a text about anxiety.
When comparing to the control group, we found that the most effective exercises were Muscle relaxation, Breathing meditation and Calming visualisation.
These insights helped us understand what content is most beneficial for our users and what we should create more of.
Looking into the future
Currently, we are working on another study, measuring our effectiveness in collaboration with the Future of Humanity Institute at University of Oxford.
We will be recruiting several thousand participants and examining how effective our app is as a whole, as well as comparing the effectiveness of individual exercises.
This will provide us with a rigorous evaluation of our effectiveness and valuable feedback on anything we can improve. Hopefully, it will also lead to a peer-reviewed publication in a scientific journal, so that we can transparently communicate our results to our users.
In addition to this, we also continuously listen to our users’ opinions, whether that’s by conducting user interviews (we have done more than 50 this year!), by reading reviews or receiving feedback directly through the app.
For instance, we have learnt that our users sometimes struggle with overwhelm in our Explore section and don’t know which exercise to do. That’s why we have recently created our Homescreen Recommender (which you can see in the title image of this blog post!). This feature picks exercises which are most likely to benefit you, based on the topics you selected during onboarding.
For example, you may have selected that you are interested in Overthinking. As a result, you will now see a recommendation for the Reducing Overthinking activity on your homescreen. This will make it much easier to find what’s right for you!
Another important thing we have recently learnt is that there is significant demand for more audio-only content. That is now one of our main priorities and something that the design, development and content teams are actively working on. Excited for new music, meditation recordings or mini-podcasts? Hopefully you will see them in the app, soon!
In the mean time, we have started producing and uploading calming music for our Youtube channel, where you can listen to relaxing tracks such as Campfire Ambience, Relax and Restore or Wake Up Gently. Special shout-out to our amazingly talented designer Delia, who recorded this beautiful music!
As you can see, our team deeply cares about providing the most effective mental health support for our users. We continuously measure how well we are doing and carefully listen to any feedback provided to us. We believe that direct collaboration with our users is the best way to create an effective tool for calming anxiety and reducing stress in people’s lives.
And if you haven’t yet, download Mind Ease – the best anxiety relief app today!
