InTouch
September / October 2017

FDA Quality Metrics Program Continues
Christopher John Potter, PhD
Article

ISPE's new online course on implementing a Quality Manufacturing program considers FDA 2016 draft guidance

At a meeting with ISPE staff and Quality Metrics Team members on 1 August 2017, FDA confirmed their plans to progress a quality metrics (QM) and data-driven surveillance program. FDA will maintain its commitment to the goals communicated in their 2015 and 2016 draft guidances 1 ,2 and associated Federal Register Notices.

The agency acknowledges while industry changes have occurred since the QM program began, its overarching goals remain: quality oversight, continual improvement, informing risk-based inspection scheduling, making inspections more efficient by focusing on higher-risk products, and the potential for indication of drug shortages. The regulators agreed that all of these goals will not be accomplished during the initial program phase.

The agency expressed strong appreciation for the value and learning derived from the ISPE Quality Metrics Initiative Waves 1 and 2,3 ,4 which they indicated had helped shape the QM program. FDA also reinforced its commitment to product-based reporting and standardized definitions.

Waves 1 and 2 were based on data from 28 companies and 83 sites, including contract manufacturing organizations, laboratories, and drug substance manufacturing sites representing a wide range of technologies. All considered themselves in a good state of compliance and already had mature internal quality metric programs.

Definitions

One of the most significant conclusions from Waves 1 and 2 is that harmonized definitions are a challenge. The proposed definitions in FDA’s 2016 draft guidance drew considerable industry feedback.

In comments submitted to the agency, 5 ISPE noted that many of the proposed terms and definitions and metric calculations were:

  • Atypical and different from those commonly used in industry
  • Not sufficiently clear despite exemplification
  • Open to interpretation due to the use of nonstandard definitions

ISPE also commented that different and unclear definitions combined with inappropriate metric calculations can lead to wide variations in data element values, and comparisons of calculated metric values between time periods, sites, companies, and technologies. This, in turn, removes the ability to make logical conclusions from or derive potential relationships between metrics.

ISPE further suggested that the definitions in its Wave 1 and 2 Pilot Programs could be a starting point for industry-wide harmonization.

At the 1 August meeting with FDA, ISPE indicated a willingness to share further recommendations and illustrative examples for certain definitions. For its part, the agency expressed a strong desire to ensure that any definitions it adopted would bring value to both industry and FDA, and drive continual improvement. ISPE proposed that FDA consider these definitions in a pilot designed with industry representatives to clarify requirements and value relative to the burden of standardizing them.

Quality Culture

Perhaps the most important finding from the Wave 1 and 2 Pilots was confirmation that culture is very important and a crucial foundation for quality excellence or good quality performance. For example, quality culture scores had statistically significant relationships with internal (e.g., lot acceptance rate) and external quality outcomes (e.g., complaints and recalls). As a consequence, ISPE has established a very successful quality culture program, which produced its Cultural Excellence Report on 25 April 2017.6 The report also includes practical tools, training, and templates that companies can adopt immediately.

Online Course

FDA has indicated clearly that a reportable QM program is on the horizon.

Using its knowledge of quality metrics programs using standardized definitions, ISPE created Operationalizing a Quality Metrics Program: Critical Success Factors,7 a new online course that explains how to implement a quality metrics* program considering FDA 2016 draft guidance. Course objectives are:

  1. Learn how to develop and standardize a collection process for data from various sites, including definitions, defining data elements to minimize gaming, and understanding how regulatory metrics fit into a wider quality metrics program.
  2. Articulate the need for standardization to acquire real and comparable data.
  3. Apply information gained from harmonized quality metrics.
  4. Understand the relationships and where there may be leading indicators and cultural indicators. These factors can inform understanding of change within a site, and consequently affect inspectional risk.
  5. Learn the value of recurring deviations as a potential leading indicator metric.

The course provides insight into establishing a harmonized metrics program, with industry examples about how a well-thought-out program can provide substantial knowledge that drives continual improvement. It also includes the finding from the Wave 1 and 2 Reports that the recurring deviations metric, although very hard to define across sites and companies, has the potential to be a leading key performance indicator.

The course provides practical steps to implement a metrics program and drive continual improvement; it is an excellent resource for a company at any stage of its quality metrics journey.

* A good review of metrics programs is given in the ISPE Operations Management Good Practice Guide. 8

About the Author

Christopher John Potter, PhD
CMC Pharmaceutical Consultant
Chris Potter graduated from the University of Exeter with a degree in chemistry and completed a PhD at Imperial College London University in organic chemistry...