The ROI of Patient Engagement: Value-Based Purchasing

October 1, 2012 marked the beginning of the 2013 federal fiscal year and the commencement of the payment period for Value-Based Purchasing (VBP). As part of VBP, the Centers for Medicare and Medicaid Services (CMS) will begin holding back 1% of the base DRG reimbursement paid to over 3,500 hospitals. Through a series of clinical process and patient satisfaction measures, hospitals will compete to "earn back" the withholding. The top performers will earn a premium over their standard reimbursement. The poor performers will see a reimbursement decrease. 


Background & Program Mechanics

In 2004, CMS began collecting quality and patient experience data from acute care hospitals on a voluntary basis under the Hospital Inpatient Quality Reporting (IQR) Program. Hospitals that participated in the IQR program were eligible for a full annual percentage increase in Medicare reimbursements.

The Patient Protection and Affordable Care Act (ACA) of 2010 requires CMS to establish a hospital Value-Based Purchasing (VBP) program that builds upon the data collected via the IQR program. CMS will establish one overall VBP score for each hospital based on data collected through the IQR program. 12 Process of Care measures account for 70% of the score and 8 Patient Experience measures account for 30% of the score.

The VBP program is self-funded by hospital contributions and, as such, is budget neutral for CMS. The "carve out" amount increases by year:

Fiscal Year Payment Carve Out (%) Payment Carve Out ($)
2013 1% $850,000,000
2014 1.25% $1,000,000,000
2015 1.5% $1,200,000,000
2016 1.75% $1,450,000,000
2017 2% $1,650,000,000


Performance Against Baseline

CMS will compare a baseline period to a performance period to determine a hospital's performance. During the first year of the program, the baseline period is from July 2009 to March 2010. Data collected during the baseline period will be compared to the performance period from July 2011 to March 2012.

CMS has established achievement thresholds for the performance period, which are the minimum thresholds a hospital needs to meet in order to receive any achievement points. Achievement scores are awarded by comparing a hospital's rates during the performance period to the rates of all hospitals during the baseline period. A maximum of 10 points can be achieved for a measure if a hospital meets or exceeds the national benchmark (mean of the top decile of all hospitals). Hospitals that fall below the threshold (50th percentile of all hospitals) level will receive zero points for that measure. Hospitals are awarded between 1 and 9 points for each measure depending on where they fall within the 50th and ~95th percentiles. Improvement scores are awarded by comparing a hospital's results during the performance period to its own results from the baseline period. 

The hospital’s Total Performance Score is determined using the higher of a hospital’s achievement or improvement score for each measure. The following formulas are used to determine achievement and performance points:

Achievement Points =
[9*((hospital performance period score – achievement threshold) / (benchmark – achievement threshold))] + .5

Performance Points =
[10*((hospital performance period score – hospital baseline period score) / (benchmark – hospital baseline period score))] - .5

CMS published some helpful scoring examples here.


Clinical Process of Care Measures

Patient Type What When
Heart Attack AMI-7a: Fibrinolytic therapy received Within 30 minutes of arrival
Heart Attack AMI-8a: Primary PCI received Within 90 minutes of arrival
Heart Failure HF-1: Discharge instructions received At discharge
Pneumonia PN-3b: Blood cultures performed in the emergency department Prior to receiving initial antibiotic in hospital
Pneumonia PN-6: Initial antibiotic selection for CAP in immunocompetent patients  
Surgery SCIP-Inf-1: Preventative antibiotic received Within 1 hour of surgical incision
Surgery SCIP-Inf-2: Preventative antibiotic selection  
Surgery SCIP-Inf-3: Preventative antibiotic discontinued Within 24 hours after surgery
Cardiac Surgery SCIP-Inf-4: Blood glucose under control 6AM after surgery
Surgery SCIP-Card-2: Beta blockers for those who were on beta blockers during perioperative period Prior to arrival
Surgery SCIP-VTE-1: Recommended venous thromboembolism prophylaxis  ordered  
Surgery SCIP-VTE-2: Received recommended venous thromboembolism prophylaxis   Within 24 hours

Since not all clinical measures will apply to all hospitals, CMS will normalize the clinical domain scores by converting a hospital's points earned to a percentage of total points.


Patient Experience Measures (HCAHPS)

Nurse Communication Communication about Meds
Doctor Communication Cleanliness and Quietness of Hospital Environment
Responsiveness of Hospital Staff Discharge Information
Pain Management Overall Rating of Hospital


HCAHPS Overview

HCAHPS stands for "Hospital Consumer Assessment of Healthcare Providers and Systems" and it is a nationally-standardized survey that is sent to patients following a healthcare encounter. Hospitals must submit a minimum of 300 surveys to eligible patients during each reporting period. The results are publicly available on the Hospital Compare website. According to CMS, HCAHPS was designed to:

  • Provide consumers with helpful comparative information on patients’ perspectives of care when choosing a hospital.
  • Create reimbursement incentives for hospitals to improve the quality of care they provide.
  • Enhance public accountability through increased transparency of hospital care quality provided.

The HCAHPS survey measures “how often” six composites of care occur (never, sometimes, usually, or always), along with a question on discharge information and overall hospital rating. The Patient Experience scoring is roughly the same as the Clinical Process of Care Measures; however, hospitals can earn up to 20 consistency points based on how well their single lowest score compares to that of other hospitals.


Patient Experience Scoring

Patient experience scoring is based on the percentage of patients that select the top answer (i.e. "always") to survey questions such as:

"During your hospital stay, how often did doctors explain things in a way that you could understand?"

To illustrate a potential score for that question, let's assume that baseline period showed that, among all hospitals:

  • Patients at the 50th percentile hospitals (i.e. threshold) answered "always" to that question 70% of the time.
  • Patients at the mean of the top decile hospitals (i.e. benchmark) answered "always" to that question 85% of the time.
  • Patients at our sample hospital answered "always" 68% of the time during the baseline period and 76% of the time during the performance period.

According to the VBP scoring formula our (rounded) achievement score is:

9 * (76%-70%) / (85%-70%) +.5 = 4

And our (rounded) improvement score is:

10 * (76%-68%) / (85%-68%) - .5 = 5

Since we are awarded the higher of the two scores, we record a 5 for the "doctor communication" measure within Patient Experience. 


Consistency Points

Once we've determined our score for each of the Patient Experience measures, we calculate our consistency points. We can earn up to 20 points if all of our measures are at or above the 50th percentile. If any of our measures is less than or equal to that of the worst performing hospitals from the baseline period, we get 0 points. If our results fall somewhere in between, we calculate our consistency score based on a formula. To illustrate, let's use the HCAHPS question related to how often the clinical staff communicated information to the patient about their medications. (Note these figures do not reflect the published national floors.)

  • Patients at the 0 percentile hospitals (i.e. floor) answered "always" to that question 20% of the time.
  • Patients at the 50th percentile hospitals (i.e. threshold) answered "always" to that question 60% of the time.
  • Patients at the mean of the top decile hospitals (i.e. benchmark) answered "always" to that question 70% of the time.
  • Patients at our sample hospital answered "always" 40% of the time during the baseline period and 50% of the time during the performance period. 

The formula for calculating consistency points is (hospital's performance period score - floor) / (achievement threshold - floor)

Our score for the medication measure would be:

(50 - 20) / (60 - 20) = .75

Next we determine which measure has the lowest score. The "Lowest Dimension Score" is the lowest performance against the floor out of all Patient Experience measures. Let's assume that our .75 for medications is our Lowest Dimension Score. The next step is to calculate our Total Consistency Score. 

(20 x Lowest Dimension Score) - .5


15 = (20 x .75) - .5


Calculating the Incentive Payment

Now that we have the Patient Experience Score, we can calculate our Total VBP Score as follows:

VBP Score = (Clinical Score x 70%) + (Patient Experience Score x 30%) 

CMS takes the VBP Score and translates it into a payment amount via the Exchange Function. Each hospital’s value-based incentive payment amount for a fiscal year will depend on the range and distribution of hospital scores for that fiscal year’s performance period and on the amount of money available for redistribution.  These six steps are used to determine the payment amount:

  • Estimate each hospital’s total annual base operating DRG payment amount using Medicare inpatient claims data
  • Calculate the total annual estimated base operating DRG payment amount reduction across all eligible hospitals
  • Calculate the linear exchange function slope
  • Calculate each hospital’s value-based incentive payment percentage (also known as percent of base operating DRG earned back)
  • Compute the net percentage change in the hospital’s base operating DRG payment amount for each Medicare discharge
  • Compute the value-based multiplier (also known as the value-based incentive payment adjustment factor)

source: CMS


How to Improve the Patient Experience Score via Mobile

Hospital mobile apps are not just miniature versions of websites. Apps provide a platform for maintaining ongoing engagement with patients. This engagement can be an extremely valuable asset during hospital encounters. (For an overview of mobile adoption statistics, please see our post on mobile ROI via readmissions.) The result can be a better patient experience and improved HCAHPS scores. Here are three specific examples:


Communication with Nurses & Communication with Doctors

There are seven questions in this category on the standard HCAHPS survey. They essentially ask patients to evaluate their care providers on bedside manner, including how well they listened and explained things. A mobile app that has an onboard medical library can be a key teaching tool for the providers and a reference resource for the patients. Here is a sample of diabetes content that is available on Axial's mobile app:


This content can be the basis of a discussion on diabetes, its causes, symptoms, and risk factors between the care providers and the patient. After the providers leave, the patient has access to all the clinical content - right in their pocket. The most effective apps take this engagement a step further and provide patients with a means of managing their recovery and ongoing wellness. Following the same diabetes use case, here's a tracker for glucose monitoring that is within the app:


Discharge Information

A simple way to help here -- that doesn't require complicated calendar integration software -- is to simply give patients easy access to their post-discharge physicians' contact info. A typical scenario might include a PCP and a specialist. With a click-to-call directory of everyone in the health system, their mobile phone becomes a resource for setting appointments -- and adding the providers' numbers to their contacts with one touch. Then, help them get ready for the appointment by reviewing what it entails and why it's important. Here's an example from our app:



Patient Survey Questions Related to Medication 

There are three questions related to medications in this section. When it comes to medicine reconciliation, a patient is not always the most reliable source of information. A phone, however, never forgets. A mobile app is a natural home for medication information. Use the opportunity to help a patient put their medicine information into their phones and set reminders for usage. Here is how it works in Axial's mobile app:



Overall Rating of Hospital

This section's two questions capture a patient's overall experience and asks if the patient would recommend the hospital. If your hospital is leveraging a mobile app to educate patients and help them follow their care plan, you will almost certainly establish yourself as a premier institution in the mind of the patient. Educating patients with good clinical content and empowering them with health tracking tools are essential ingredients of HCAHPS success; however, there are some routine logistical headaches that can turn an otherwise positive patient experience into a negative one. If your mobile app has a click-to-call directory of all hospital resources, then you can save patients a lot of frustration by pointing them to the people they will likely need to call. And don't overlook wayfinding. Hospitals are notoriously difficult places to navigate.



Estimating the Financial Return

Mobile can positively influence 15 of the 25 core HCAHPS questions and 5 of the 8 measurement domains, specifically those that involve the communication with doctors and nurses, discharge planning, medication understanding, and overall rating of the hospital. The published national floors in these measurement domains suggest there is room for improvement.

National Floors:

  • Communication with Nurses (39%)
  • Communication with Doctors (52%)
  • Communication about Medicines (29%)
  • Discharge Information (51%)
  • Overall Rating of the Hospital (29%)


Let's take a look at what HCAHPS improvement means for a sample hospital. First, we'll establish a base case.

Metric Base Case
Beds 500
Annual Revenue $500,000,000
% Medicare DRG Revenue 40%
Medicare DRG Revenue $200,000,000
% of DRG Revenue Withheld via VBP 1%
Amount DRG Revenue Withheld via VBP $2,000,000
Clinical Process of Care Total Score 60
Weight 70%
Weighted Clinical Process of Care Total Score 42
Patient Experience Total Score 30
Weight 30%
Weighted Patient Experience Total Score 9
Total VBP Score 51
Linear Exchange Function Slope 1.84
Hospital VBP Percentage 94%
Incentive Payment $1,876,800
Gain / (Loss) ($123,200)



Metric Improved Case
Beds 500
Annual Revenue $500,000,000
% Medicare DRG Revenue 40%
Medicare DRG Revenue $200,000,000
% of DRG Revenue Withheld via VBP 1%
Amount DRG Revenue Withheld via VBP $2,000,000
Clinical Process of Care Total Score 60
Weight 70%
Weighted Clinical Process of Care Total Score 42
Patient Experience Total Score 70
Weight 30%
Weighted Patient Experience Total Score 21
Total VBP Score 63
Linear Exchange Function Slope 1.84
Hospital VBP Percentage 1.16%
Incentive Payment $2,318,400
Gain / (Loss) $318,400


By improving 5 of 8 patient experience domain scores from middle-of-the-pack to above average, we are able to create $441,600 in value -- this is our return. This return only increases as the VBP program steps up from 1% to 2% of DRG payments over the next 4 years. Even if our return stays flat, we can show a compelling ROI. 


The ROI Calculation

Now that we've determined the "R" in in ROI calculation. It is time to determine the "I" or cost of the investment. ROI is calculated by subtracting the cost of the investment from the gain from the investment and dividing by the cost of the investment. Note that a more complex ROI model might forecast all future returns and costs and discount them back to present value based on cost and risk. This post uses the first year ROI as a proxy for future years.

ROI = (Gain from investment - Cost of investment) / Cost of Investment.

What is the cost of launching and managing a smartphone app? That depends on how you build and manage it.


Here are two scenarios:

1. You build and manage the app via your in-house technology team.

1/4 marketing person

1 iOS developer

1 Android developer

1 HTML5/CSS/Javascript developer

1/2 designer

1/4 QA engineer

While salaries vary depending on where you live, a range for the annual fully-loaded cost (including benefits, etc.) for this scenario might be $300,000 to $600,000. Note that some teams could easily be 2-3X this large depending on the scope of the app. This also assumes that existing "back-end" technical resources can be used for maintaining the app. This is the permanent team that will manage the app on an ongoing basis. With this scenario, we can estimate our ROI.


2. You use an outside firm that specializes in mobile health apps.

Design and development fee + ongoing updates and enhancements. 

Depending on which firm you use, the cost and quality can range widely. If we include the one-time design and set up fees and add them to the ongoing enhancement fees, we'd get a annual average fee of around $150,000 with the first year being higher and the out years lower. 

Let's assume that the health system chooses option #2 based on the lower cost. 

ROI = 194%


Note that this return is for readmissions improvements alone. See how to generate the full 1,560% return here.




Mobile devices are quickly making an impact on how patients manage their health. For a health system, not having a mobile engagement offering 2013 may be similar to not having a corporate website in 2003. Our posts on ROI should help you evaluate the quantifiable value that mobile health can deliver to your organization.