In defense of measuring engagement


All lies and jest,

Still a man hears what he wants to hear

And disregards the rest.

The Boxer, Simon and Garfunkle.


Those who study rhetoric describe the "fallacy of exclusion" as:

When only select evidence is presented in order to persuade the audience to accept a position and when evidence that would go against the position is withheld.  The stronger the withheld evidence, the more fallacious the argument.


The rest of us call it cherry picking.


This concept came to mind when someone forwarded me a blog post that rather stridently called out the Axial-Becker’s Patient Engagement Index (PEI) for ranking healthcare institutions on “features” and not clinical outcomes. The post was authored by Jordan Dolin, the founder of a firm that also sells patient engagement solutions.


Engagement to Outcomes

Do readmissions count as an outcomes measure? The Centers for Medicare and Medicaid Services lists them among its outcomes measures here. elaborates further:


The Centers for Medicare & Medicaid Services (CMS) 30-day readmission measures assess a broad set of healthcare activities that affect patients’ well-being. Patients who receive high-quality care during their hospitalizations and their transition to the outpatient setting will likely have improved outcomes, such as survival, functional ability, and quality of life.


The public reporting of 30-day risk-standardized readmission measures is consistent with the priorities of the Department of Health and Human Services’ National Quality Strategy, which aims to: a) improve health care quality; b) improve the health of the U.S. population; and c) reduce the costs of health care.


To be clear, the Patient Engagement Index incorporates readmission rates as part of its ranking methodology. Whether Mr. Dolin’s failure to recognize this fact was an error born of insufficient diligence or an act of deliberate omission, I can’t say. It is also not clear from Mr. Dolin’s post if he understands that the PEI is meant to rank hospitals and not vendors. (He concludes his post with a proposal for an improved vendor ranking.)


Is Patient Engagement Analogous to Consumer Advertising?

As a rule, Axial cheers on any vendor or healthcare institution that is helping to make our healthcare system more effective by engaging patients in self care. We would include Mr. Dolin’s company in that group of fellow travelers. That said, when Mr. Dolin compared consumer advertising to patient engagement, it underscored the fact that not all patient engagement vendors are the same. Here’s the relevant excerpt from Mr. Dolin’s post:


In one sense, you should think of patient engagement like advertising, a process to present messages to a group of people in hope of changing their behavior.

Engagement, like advertising, is a means to an end, so we need to focus on the end rather than the process.



Advertising might be an apt analogy for a firm that is primarily focused on engaging patients through passive content consumption. At Axial, we believe that patient-facing content is a necessary but not sufficient means of sustained patient engagement.


It is true that Axial has a strategic partnership with the Mayo Clinic through which we provide the Mayo Clinic’s health content to our users. When it comes to health content, patients place a premium on trustworthiness. To that end, the Mayo Clinic was a no-brainer partner for us. The content has been an important means of improving the overall health literacy of the patients that we serve. But it is easy to forget that the day-to-day reality of health management requires more than consuming content.


True patient engagement must also include:

  • taking medications as prescribed
  • adhering to nutrition guidelines
  • staying active
  • measuring and monitoring vital signs
  • keeping track of symptoms
  • collaborating with caregivers and healthcare providers


At Axial, we help patients with the day-to-day management of these tasks. PewResearch tells us that 69% of US adults track health in some way. We provide technology and services that makes it easier to do what we know many patients already are trying to do: follow their care plan. 85% of our application’s usage is related to health management. For example, our most popular self care tool is medication management. Our users are able to manage dosage and usage instructions, set reminders, and track adherence. Users of this tool access it 28X per month on average. Why does this matter? Medication nonadherence costs our healthcare system $100B medication annually and is the cause of 10% to 25% of hospital admissions.


The Bottom Line

Despite all the hype, patient engagement remains in its inchoate phase. Some health systems are doing remarkable work while others are taking a "wait and see" approach. The Patient Engagement Index is meant to shine a light on institutions that are leading the way forward.  When we came up with the idea, we saw plenty of hospital rankings, but none focused on patients. If the PEI only measured outcomes, it would add little to what CMS' Hospital Compare already publishes. Instead, the PEI recognizes hospitals that are providing patients with a sufficient breadth of content and tools to support ongoing, day-to-day engagement. We combine this with HCAHPS and readmissions rates to generate our index. 

We can't wait to publish the results in May.