Nearly three decades ago some well-meaning industrial devotees within healthcare began to experiment with comparative data for healthcare provider organizations. No doubt, it was a well-intentioned initiative designed in a sincere attempt to see which providers were getting the best results for the investment being made by the government or other third-party payers.

However, like so many other business techniques applied to the healthcare community, the simple initiative has blossomed, as in "The Little Shop of Horrors," into a giant organism that must be fed and which is an apparent insatiable consumer of human, material and fiscal resources.

The healthcare industry has lost sight of the fact that benchmarking is not a solution or end to be achieved. Benchmarking is merely one very small fractional element in determining what needs to be done to improve operations within any organization. This has become evident, and is commonly acknowledged behind closed doors with many healthcare leaders throughout the provider community.

A benchmark should be a call to action and provide only minor directional guidance to operational improvement. Benchmarking in any application was never intended to be an absolute value or target that must be achieved, yet this is largely how healthcare providers and related enterprises have attempted to utilize the tool. Like so many other examples of business/industrial processes and tools that have migrated to healthcare, benchmarking was "adopted without being adapted" to the unique circumstances and complexities of the healthcare industry. The perils of this carte blanche adoption before adaptation have been proven repeatedly in healthcare – i.e., TQM/CQI, re-engineering, corporate restructuring, downsizing, outsourcing, etc. All of these efforts have taken the industry by storm and yet none of these has yet yielded anything close to the projected value or savings touted by their crusaders and early adopters.

Benchmarking in the healthcare industry now permeates and consumes the careers of tens of thousands of healthcare workers; not to mention the legions of staff from government agencies, insurance empires, corporate benefits enterprises and academia who gather, examine, dispute and rehash billions of bits of data on virtually every aspect of the industry. These endeavors consume hundreds of millions, perhaps billions, of dollars of resources with little tangible benefit being derived to advance healthcare in this country. One has to ask if these efforts were curtailed, even by a modest amount, and both the energy and related funding was redirected to actually providing care or applied directly to preventive or curative efforts, that the benefits yielded would be so much greater than the meaningless and largely ignored tomes of statistics we currently get for the investment in benchmarking related activities.

Acknowledging that there is perhaps some minimal benefit derived from comparative measurement efforts related to specific clinical outcomes, it is abundantly evident that even within this very limited realm, the accuracy, and therefore the value,
of the comparatives is limited at best. However, the vast expenditures of human, material and fiscal resources, goes well beyond these clinical measurements and now saturates nearly every aspect of every healthcare institution. The return for the huge investment in benchmarking these non-clinical areas is largely meaningless, and, therefore, an absurd waste of dwindling healthcare funding.

The limited value derived from current benchmarking practices stems from two primary factors – gross misunderstanding/misinterpretation/
misapplication of the intent of benchmarking; and the virtually non-existent standards for measurement.

Benchmarking can be defined in a variety of ways, including:

In all cases the intent is to provide a guide, a directional indicator, a hypothetical goal or target toward which an organization or individual should strive.

The reality in the healthcare provider community is that there are components that:
• vary wildly with significant impurities;
• are being mashed together in a less than orderly, let alone standardized fashion;
• are intended to treat a variety of ills rather than a specific disease or symptom.

It’s important to note that these general comments on the futility of incessant benchmarking should not preclude the prudent reader from seeking or using certain "industry standards" as a "touch point" for determining or gauging performance indicators for such things as utilization or consumption analysis. For example, in the realm of healthcare supply expense it is common practice to refer to key result/performance indicators that are commonly published in national journals, such as linen consumption (quantity/occupied bed), or dollar amount of inventory/bed, or supply expense as a percent of net patient revenue, etc. Such standards should be checked periodically (i.e., annually), but provider organizations should not be monitoring and measuring their performance against such standards as a matter of routine operational metrics.

For benchmarking data to be truly meaningful, and for the value of the return on investment to be worthwhile, it requires a significant level of standardized methods, processes, components and precise uniform measurement/calculation methodologies. These criteria are scarcely evident and rarely documented within any single healthcare provider organization, let alone consistent between or among providers of various shapes, sizes, geographies, patient and payer mixes, staffing models, physical plants, fiscal structures, etc.

For example: Benchmark the supply costs per adjusted discharge (SC/AD). Assuming the intent of calculating such a value is a meaningful indicator of supply cost controls (and this is certainly subject to debate), there needs to be a reasonably precise set of very clearly defined and documented components, values, and measurement methods in place.

There are a myriad of component costs that can go into determining the supply costs including direct and indirect product cost factors.

Direct product cost factors include:

• Product mix (individual components as described by physical characteristics, brand names, catalog numbers, sizes, etc., and including but not limited to reusable vs. disposables)
• Vendor/sourcing selection
• GPO contract participation and at what pricing tier
• Cash or Quantity discount eligibility
• Inventory capacities, values and carrying costs for ALL points of storage
• Markups and handling fees by manufacturers and distributor

These direct factors are strongly influenced by other indirect supply cost factors, including:
• Patient services offered – i.e., orthopedics, cardiovascular, OB-GYN, etc.
• DRG comparatives
• Size, age and location of healthcare provider facility
• Storage capacities throughout the facility
• Use of automated supply cabinets or other systems
• Delivery service levels, distances and frequency from suppliers
• Delivery/service frequency for all departments
• Staffing levels, models and patterns for materials functions throughout the facility
• Condition, version and degree of utilization of MMIS or ERP and/or compatibility with other IT systems
• Outsourcing (inclusion/exclusion of supply costs in the management fee)
• Ebb and flow of census levels and acuity mix
• Medical staff size, composition and individual physician referral patterns
• Technology availability and utilization

All of these factors and dozens of other elements can have an impact on supply costs. Despite the requirements and standards for financial recognition of these costs, there is still latitude for frequent variation of just how these costs are defined, calculated and how they are posted in the various financial reports within any one provider organization. When there is an attempt to compare all these variable definitions and elements across multiple facilities or healthcare provider organizations, the complexities are substantially increased, making any comparative relationships virtually meaningless. Meaningful comparisons can only be reached if there is minimal variability in the factors being compared.

As noted in A Better Way to Benchmark by Gardner, Harrity and Vitasek, there are two very fundamental challenges to establishing meaningful benchmarking practices – namely basic communications (i.e., lack of tightly defined, commonly accepted definitions/parameters) and the lack of availability of timely (sufficiently current) widely accepted standards with which to compare ones current operations.¹

So should it be benched?
Compounding the issues reported above is the plethora of alleged credible sources of benchmarking standards. Far too many organizations calculate and publish their "standards," sending mixed messages to the provider and payer organizations’ leadership. In reality the definitions behind these "standards" are rarely revealed, hence the adoption/acceptance of them is equally rare and any comparative efforts cannot be validated. Often, whatever journal a healthcare provider organization leader reads or whichever seminar he/she last attended becomes "THE" authoritative source for setting the new standard for their organization.

Another practice that is blatantly irresponsible is the promotion of totally false, misleading or unrealistic benchmark indicators that are touted by authors or lecturers to inflate their own credibility or to promote their services. One recent example came from an article in a trade journal by a person that sells the use of the tool of value analysis. They blatantly suggested that an arbitrary and totally unobtainable high percent of savings be set as a goal for expense reduction through the use of value analysis. Supposedly, this high figure was used as a motivator to stimulate momentum for the concept. Unfortunately, it is almost assured that some well-meaning senior hospital executives (who are largely unaware of the realities of supply chain expense management) will read that article and assign this grossly inflated indicator as the new, absolute standard for their facilities. They will then flog their materials management and financial staff with this number for untold months; only to have them fall far short of this newly adopted "standard" – not knowing or acknowledging the futility of these efforts.

Frequently, there are multiple sources used to measure different performance characteristics defusing any hope of adopting and adhering to standardized methods. It seems as though everyone is now in the benchmarking business. Various government entities – federal, state and local; public accounting and consulting firms; insurance and other third-party payers; group purchasing and shared services organizations; manufacturers and distributors; various academic institutions and leading think tanks and an array of opportunistic commercial enterprises offer up a cacophony of statistical babble. Some of these enterprises only examine and/or service a very narrow element of the industry, while many others propose to be the ultimate authoritative source for anything statistical related to healthcare. All of these sources use various components, variant statistical methodology, and assorted timeframes in their research and reporting. These variances cause significant challenges to any healthcare provider organization seeking consistently meaningful comparative information. The degree of difficulty is compounded by the fact that few, if any, of the these alleged authoritative sources reveal any details on what base elements they measure, how they do it, and/or what assumptions they make, sample size of their data sources, or other variables they consider.

Industry-wide benchmarking for non-clinical healthcare operational elements should continue in a limited and standardized fashion. The current wide array of benchmarking sources needs to be significantly narrowed to perhaps three sources:

The only benchmarking that is truly meaningful and that will yield real value to any healthcare provider organization is internal benchmarking and process improvement directed at driving the numbers (of what ever is being measured) in the "right direction."

If a provider organization measures any criteria, process or outcome and establishes a measurable value, then it needs to consistently analyze the various elements by using lean thinking, TQI/CQI/TQM, Six Sigma or any other method (du jour) they deem appropriate. They should focus time, energy and resources to improve upon that established value. It is only through the consistent application, using strictly enforced criteria and very clearly defined component values/parameters that meaningful measurement (i.e., benchmarking) can be considered a worthwhile endeavor.

Measurement without action or action without measurement is of little value and generates far less than optimal outcomes.

It is this combination of meaningful measurement and honest/critical analysis coupled with appropriate action for improvement that generates value.

As a major think tank learned 40 years ago in trying to manage the policy and resource allocation controls for the Viet Nam conflict, "(there is) a uselessness of simply trying to mathematically extrapolate the future from the present." ⁴