As an art form, benchmarking can polish a healthcare facility’s image and preen its executives’ egos or poke gaping holes in both. As a science, benchmarking can make or break a healthcare facility’s expense and revenue streams – particularly if the data are solid, reliable and measurable.

However, ensuring that the data being collected and analyzed among internal departments and with similar external facilities is reliable and therefore valid and valuable, remains an ongoing quandary.

It’s no secret that benchmarking gleams as a concept and glows on paper but it can gum up administrative, clinical and financial operations when true "apples to apples" comparisons aren’t achieved. Trying to establish and define a benchmark or any of its components has its own hurdles. One of the fundamentals is convincing the myriad players in healthcare to accept a particular standard and its defined parameters, regardless of who originated it or will benefit from it. In short, the intention may be fine but the execution tends to suffer.

HPN: Because benchmarking seems so variable, how can it be meaningful, and therefore, valuable?

Unfortunately, most current bench-marking studies do not provide enough visibility for organizations to rely on the data as a single end point to inform decision making. Benchmarks should really be viewed as one piece of the performance management puzzle. Essentially, they are just one of the many opportunities for organizations to assess performance and should be used in combination with other internally generated performance metrics and indicators. Some institutions tend to start and stop with a benchmark. Given the current variability in most external benchmarks, they should serve as a barometer or directional indicator rather than an end point. 

With all the variables that come into play, such as geographic market service area, patient mix, clinical pathway deviations, and so forth, how can a facility truly find a ‘peer’ institution? In manufacturing and retail, it’s quite a bit more defined and regimented, and therefore easier. Do you agree? Why?

Ultimately, no organization, inside or outside of healthcare, is truly the same.
Benchmarks in other industries appear to be criticized with the same fervor. 

That being said, one of the main supply chain differences between healthcare and other industries is that healthcare providers cannot forecast exactly what supplies they’ll need to treat the next patient that walks through the door. In most other industries, supply chain executives have a much better ability to forecast demand. Also, healthcare is one of the few, if not the only, remaining industries where the purchasing organization lacks control of their own data. For the most part, hospitals are reliant on external supply chain constituents to answer basic questions such as, what I am buying, who am I buying from, and did I pay the right price. These challenges make it even more challenging for healthcare providers to provide reliable data in support of benchmarking activities.

I do think that healthcare over exaggerates the fact that every hospital is different. Despite regional and organizational differences in payer mix, case mix and volumes, most healthcare organizations ultimately operate on the same core financial principles and they generally use a similar mix of supplies. A good benchmarking database should enable hospitals to ‘right size’ the compare group so they are relatively confident they are comparing themselves to someone in the same ball park. I’ve seen current benchmarks that enable a decent ‘apples-to-apples’ comparison, but I think we are a few years away from differentiating between a Granny Smith and a Red Delicious. Understanding this, hospitals should utilize benchmarks with a grain of salt, but can and should use them confidently to provide a directional indication of their current performance. It’s all about making sure that you understand the methodology behind the benchmarks and how they relate to the way your organization calculates the same metrics. This visibility into the benchmarking methodology is critical to evaluate the reliability of the data and ultimately the conclusions you derive about current performance.

Everyone seems to have their own favorite benchmarking mechanism for supply chain management. So is there one universal benchmark that can be applied reliably to any facility, as in a standard? Or are there certain benchmarks that work better than others in specific clinical, financial or operational areas? What are they?

I don’t think there is one universal benchmark that can meet the needs of every institution. It really comes down to relying only on metrics that can be defined and validated with enough confidence to support your needs. Best practice institutions review multiple metrics as a way to think more broadly about how supply chain performance is affected by other macro-level trends. It is important to understand the relationship between the different metrics because most supply chain performance benchmarks almost always rely on some non-supply chain indicator for part of the calculation (i.e., net revenue, operating expense, admissions, discharges). Looking at multiple metrics can help organizations understand how the non-supply chain indicator impacted performance.

How does a materials manager make sure that the benchmark on which he/she relies is meaningful, as in an accurate apples-to-applies comparison? Does such a benchmark truly exist?

There is no way to ensure a 100 percent apples-to-apples comparison unless all data collection can be automated. Not only would this require industry standard definitions for calculating metrics, but it would also require that all benchmark participants are sending automatic feeds of data from a common IS platform. In the meantime, materials managers should be critical in understanding how the benchmarks are developed. Consider questions such as: How many participants similar to my organization were included in the study? How did they define the underlying metrics that support each metric? Did the benchmarking provider audit the results for accuracy or was the data self-reported? Is the information updated and how frequently will it be updated on an ongoing basis?

But it’s possible to automate data that are slightly different, based on subjective determinations, which would undermine the objectivity and reliability of the benchmark, right?

It’s possible to automate collection of things like PO data for the purposes of price benchmarking. As long as the PO line items are compared using the same categorization schema the benchmarks can be quite valuable. 

It is a bit more difficult for performance benchmarks because there is no industry standard definition for the core financial metrics that support the benchmarks. With no standard definition, hospitals are required to interpret the definition provided by the benchmark provider (if even provided) and then assess if their reporting systems can calculate the data in line with the definition. It all comes down to how well the benchmarking provider defines the metrics used to calculate the benchmarks and their ability to audit or test that hospitals accurately report them.

Which benchmarking strategy makes more sense – internal (comparing individuals or departments within a facility or organization) or external (comparing departments, facilities or organizations to one another)? What purpose does either serve?

Both strategies can be valuable for organizations. The key to making any benchmark or metric valuable is to consistently review the same metric or calculation trended over a period of time. Whether the benchmark is internal or external, organizations are best served by creating a repeatable reporting process that utilizes a consistent method for calculating the information. This consistency will help them accurately track and measure trends impacting their business over time. It is also important to think about how you intend to use the benchmarks. For the most part, current benchmarks are most useful in helping to establish goals or to inform opportunities for improvement from more of a directional perspective. 

Unfortunately, all of these terms are used synonymously today. From my perspective
there are three main ways to look at benchmarks and metrics. 1.) Performance benchmarking – Comparison of high level performance metrics across similar peer organizations that can be useful to understand directionally how you organization stacks up. Sample metrics might include supply cost as a percentage of net revenue, supply cost as a percentage of operating expense or supply cost as a percentage of CMI adjusted admissions or discharges. 2.) Price benchmarking – Comparison of specific prices paid for unique items across similar peer organizations. 3.) Internal metrics/indicators – Month over month internal tracking metrics and indicators to evaluate of actual versus goal performance. Metrics might be similar to the performance benchmarks, but can also get more granular to evaluate category management (i.e., market share and volume goals, standardization opportunities, and budget performance), supplier performance (i.e., fill rates, cancelled orders), and purchasing effectiveness (i.e., average number of purchase order lines per order, total POs per month). Most organizations currently participate in some form of performance benchmarking, all are hungry for accurate and updated price benchmarks, and few have the capabilities or infrastructure to support broad and consistent utilization of internal metrics/indicators for materials management and supply chain performance.

A growing number of companies and organizations seem to have ventured into the benchmarking business so how do you determine which one is the most reliable and useful?

Key questions to ask are: How do you get your data? Is it self reported? Do you audit all or some of the figures? Can the data be pulled directly from existing systems or financials versus manually entered? How frequently do you update your informa-tion? How often will we need to submit data? How do we submit data? How many participants are included in the study? How can we filter the data to ensure we compare ourselves to similar institutions?