Value the quality of health apps: new rating scale

Posted on

Three or five stars in the App Store? What does the subjective evaluation of the user in the app stores say on the objective quality of the assessed health app? Are there defined quality criteria, which can reproduce these evaluations independent of the tester? That was the question of the Australian research group, which has developed the “Mobile Application Rating Scale” (MARS) and which has published first results for validation now.

A total of 29 questions in six categories (A to F) should allow conclusions on the objective quality, the test results are referred to current studies with a total of 60 apps with the star rating of the app users.

Factors that can predict the success of a health app will be interesting for developers and providers of health and medicine Apps equally.

Here are short descriptions of the test categories with the respective core issues:

  • Is it fun to use the App? Does it attract the interest of the user? Is it tailored to the target users? Does it involve the user interactively? (5 questions, user experience)
  • Is it simple and intuitive to use the app? Is the app structure logically? (4 questions, usability)
  • Look and Feel, attractiveness of the visual world, Professionialität of design? (3 questions, user experience)
  • Information & Content: Does the app come up to expectations? Does it explain the achievable goals using the app objectives? Is the information relevant to the user? Are the texts understandable? Is the information convincing and well visualized? Does the app come from a trusted source? Is the effectiveness of the app tested in trials? Is the knowledge evidence-based? (7 questions standards of quality, reliability and evidence))
  • Personal quality assessment: Would the tester recommend the app? How often would he use the app (1 up to more than 50 times?)
  • Would he be willing to buy the app? How would he vote on the 5-star scale? (4 questions)
  • App-specific issues: Impact on health behavior – What does the app? Raise awareness, provide knowledge, change attitudes, motivate to a behavior change, motivate to use more available offers of help, modify behavior to promote health? (6 questions: Health Conducive method concept of the app)

The rating scale classifies the app also based on the following general questions:

  • Technical features of the app. Is a Registration required? Does the app offer a password protection? Does it need network access? Can you send messages? Does it offer different functions for sharing (twitter, facebook, etc.) with others? Note: Additional questions about permissions of the app and privacy policy are not acquired.
  • Focus of application of the app (eg. As behavior modification, physical health, entertainment, depression, etc.)
  • Theoretical support approach of the app (eg. As risk assessment, information, tracking, relaxation, training, etc.)
  • Age of the target users
  • Provider group (University, Governmental Authority, businesses, etc.)

Conclusion: The MARS-rating scale provides important information on quality-determining characteristics of a Health App. Most of the questions are gradual assessments, the answer is influenced by the background and experience of the tester. The calculation of scores weights the category D (Information and content) with max. 7 x 5 pts, the category C (look and feel) with 3 x 5 points.

So far, the MARS scale was tested in a pilot phase with 9 iOS apps and the reproducibility of the results was controlled with50 iOS apps in the final test. We are looking forward to more results.

by Ursula Kramer