Connections : Join the Conversation - The blog of Biomet CEO Jeffrey R. Binder

The Use and Misuse of Registry Data

In April, 2009, the AdvaMed Orthopedic Sector issued recommended principles in support of creating a United States total joint registry. To help develop the U.S. registry in accordance with the AdvaMed recommendations, Biomet has donated generously to the American Joint Replacement Registry, (AJRR) currently under development by American Academy of Orthopaedic Surgeons (AAOS). Biomet is also represented on the AJRR Board of Directors.

Registries provide surgeons with helpful implant survivorship data and may provide useful surveillance data to companies and regulatory bodies. In order to ensure the proper use of registry data, it is important to note the critical differences between clinical studies and registries.

Benefits and limitations of registries vs. clinical studies

As compared to clinical studies, registries have the advantage of large patient populations; breadth of experience across large numbers of hospitals and surgeons; and immunity to the perception of bias resulting from investigators' financial ties.

On the other hand, registries are typically limited in their ability to measure endpoints beyond survivorship and to control for differences in patient population, patient risk factors and patient expectations beyond basic demographics. In addition, registry data can be misleading if revision rate data is isolated from factors such as non-device related failures; potential differences in patient populations; significant variance in outcomes among surgeons and sites within the registry; and results from other registries and clinical studies.

For these reasons, registry data should not be used in isolation to make policy decisions related to reimbursement and coverage. Furthermore, registry data should be used for regulatory purposes only if there is a confirmed threat to public health, which can only be determined through a full evaluation of all available data sources and an understanding of the multiple factors that contribute to sub-optimal outcomes.

As surgeons and other stakeholders develop and administer registries, it is critical to ensure that registry data is never misused. The registry recommendations developed by AdvaMed and the orthopedic industry state: "The data points collected for the registry should...not be used for purposes for which the registry is not defined. Registry data alone cannot compare the relative effectiveness of devices."¹ Indeed, only well-controlled clinical studies may appropriately isolate and evaluate the relative performance of devices.

Examples of registry limitations

Hip resurfacing in Australia
As one example with which we are familiar, the Australian National Joint Registry has observed a higher than anticipated rate of revision for the ReCap® resurfacing femoral component as compared to similar implants. Interestingly, the revision rate of the ReCap® hip in Australia was significantly higher than reported in the UK National Joint Registry and based on far fewer reported cases (137 in Australia versus 750 in the UK).^23,24 Additionally, of the eight revisions reported in Australia, five were isolated to one hospital.

This issue raises several interesting questions: To what extent should anyone using registry data from one country also consider data from other registries and available clinical studies? We know that there are numerous potential reasons for varying results across sites: learning curve; technique-sensitive devices; differences in training protocols; differences in patient inclusion criteria; and differences in post-operative protocols. Should a bad experience at one site, which affects the average performance of a device if the sample size is small, lead to potential regulatory actions?

These are more than academic issues – the Australian TGA is now considering recommendations to remove devices with greater than anticipated revision rates from the Australian market.

Inappropriate use of registry data to market devices

Another issue arises when orthopedic companies misuse registry data. One of our competitors has used isolated revision data from the Australian Registry to purportedly show reduced revision risk of its unicompartmental knee vs. Biomet's Oxford® Partial Knee.² There are two significant problems with the advertisement and its claim:

• The advertisement claims that the data are based on an independent study; however, the registry is not a controlled comparative study.
• The advertisement ignores the plethora of world-wide clinical studies and registry data that support the Oxford® Partial Knee's outstanding long-term clinical performance.

Indeed, 16 clinical studies have been published on the Oxford® Partial Knee, comprising 2,847 patients with follow-up ranging from 2 to 15 years. The total number of device-related revisions performed was 77 (2.7% of the total population).⁴

Additionally, Price, et al., provided updated Oxford® Partial Knee data demonstrating 92.3% survivorship at 20 years.³ Given appropriate patient selection and indications, the Oxford® Partial Knee has proven to be quite durable over the long term in numerous clinical studies.⁴

 

In a second instance, the same competitor employed Swedish Registry data to make a misleading claim that its total knee has the "Lowest Relative Risk of Revision when compared to competitive knees."²¹ Based on the data presented, this claim is statistically invalid.* More importantly, it is simply not valid to use data from one registry to make a general point about any implant's relative risk of revision as compared to another implant.

As a further example of how isolated registry data can be misleading, within the Swedish Registry, five of the ten Biomet® Vanguard® Complete Knee failures were infections, with three of these infections from one hospital. Four of the revisions were due to the later implantation of patellar components that were not implanted during the initial surgery.²² With non-device related revisions removed, the Vanguard® Knee's revision rate might well be the lowest in the registry—however, we would still not dare to claim that the registry proves that our implants have the lowest risk of revision. The data simply cannot be stretched that far.

In these cases, the competitor in question violated the very AdvaMed principles that it agreed to support—including a clear statement that registry data alone cannot compare the relative effectiveness of devices.

The dilemmas posed by registry data

Registry data provides a snapshot of isolated performance on a limited number of variables. In the following hypothetical example, Device 1 demonstrates short-term survivorship data that is inferior to Device 2. However, over the long term, Device 1 out-performs Device 2 in survivorship. Looking strictly at registry data, it is conceivable that the registry would suggest removal of Device 1 after only two years on the market.
Surgeons themselves, charged with developing practice guidelines, might come to the same conclusion.



In independent studies, Device 1 might show better scores for pain, function, and patient satisfaction that indicate its applicability to certain patients and predict long-term durability. But the registry would not pick it up.

A second critical dilemma is the variability in results from site to site. As stated above, the results of any procedure vary based on surgeon technique and skill, experience, pre- and post-operative regimen, and patient condition and selection.

Registry data provides an unparalleled view of average performance over a broad population. However, applicability to any given patient is limited. Surgeons and patients must collaborate to develop a personalized treatment plan based on patient conditions and expectations, as well as the surgeons' knowledge of what works best in their hands. Registry data is a useful tool that can inform, but not displace, surgeon judgment.

Conclusion

As surgeons and manufacturers come together to support the creation of registries in the U.S. and other countries, many in the orthopedic community have recognized the need to ensure that registry data is not misused by government officials and private insurers to make unsubstantiated coverage and payment decisions. The best course of patient treatment should be decided by individual surgeons who have properly utilized data from both clinical studies and registries.

Additionally, orthopedic companies should commit to not use registry or clinical study data in an irresponsible or misleading fashion. If the orthopedic community cannot exercise discipline to adhere to scientific standards, we cannot expect parties external to the community to do so.