Developing the Right Measures Begins with Effective Conceptualization

PQA follows a systematic, transparent and consensus-based development process to ensure that measures are important, scientifically acceptable, feasible and usable. This approach produces strong, consistent results, but it also takes time.  

From the initial stages of convening a technical expert panel to the final stages of achieving PQA endorsement, measure development can last 12 to 24 months or longer, depending on the complexity of the measure. As a result, it’s critical that PQA selects the right measures for development. This blog details the first step in that process, measure conceptualization. 

The PQA Measure Development Process

The PQA Measure Development Process timeline

Setting Priorities: The Importance of Conceptualization 

Conceptualization focuses on generating and prioritizing measure concepts for developmentThrough environmental scans, input from our Measure Concept Advisory Groups and public comment, PQA members and other health care system stakeholders help ensure our efforts are focused on high-impact areas of need. Conceptualization activities typically occur on an annual basis to keep PQA’s priorities relevant and consistent with the evolving quality landscape. 

Laying the Groundwork: The Environmental Scan 

The conceptualization process begins with environmental scanswhich informs potential measure concepts by identifying quality goals and evaluating the strength of scientific evidence. They can also identify existing measures that are related to the proposed measure concepts or may be candidates for adaptation. 

PQA’s environmental scans begin with a literature reviewReports like the CMS Environmental Scan and Gap Analysis Reports, and the CMS Program-Specific Measure Needs and Priorities provide a wealth of information on measurement gaps that PQA can address. 

Environmental Scan Source Examples 

Environmental Scan Source Example

Beyond the literature, environmental scans may include targeted interviews with subject matter experts, lessons learned from current measures in development or in use, previously identified priorities and other information sources. 

By mapping opportunities identified in the environmental scan against PQA’s strategic aims and the CMS Quality Strategy, the PQA team creates a list of priority areas and measure concepts to close gaps. Measure concepts are not fully specified but include preliminary drafts of eligible populations, numerator criteria and other basic information to help stakeholders understand what a potential measure might entail. 

PQA members play an important role in conceptualization by sharing ideas for measure concepts with PQA throughout the year. When concepts are evidence-based and follow sound measurement principles, they’re added to PQA’s repository of measure concepts. 

Engaging PQA's Expert Members: Measure Concept Advisory Groups 

After creating the list of potential measure concepts, PQA convenes Measure Concept Advisory Group (MCAG)MCAGs are composed of 20-25 subject matter experts from PQA member organizations and are convened on an ad hoc basis to evaluate measure concepts. MCAG members are selected through a self-nomination process based on their expertise, with an emphasis on ensuring representation of PQA’s diverse members, including measured entitiespatients and caregivers and key stakeholders relevant to quality measurement or a measure’s focus 

Different MCAGs may have different focus areas. One MCAG might focus on pharmacy measurement, while another MCAG focuses on health plan measurement. MCAG members evaluate measure concepts using standardized measurement criteria: 

  • Importance: The extent to which the measure is focused on a priority area, is evidence-based, and can have a positive impact on healthcare quality.  
  • FeasibilityThe extent to which data needed to calculate the measure are readily available for measurement and retrievable without undue burden. 
  • UsabilityThe extent of opportunity for implementation and the degree to which performance results can be used for both accountability and performance improvement. 

MCAG members reach a consensus on recommendations for each measure concept through discussion and surveys. Measures that rate highly on the standardized criteria may be recommended to advance to full development in the near termMeasures that rate lower may be retained for future consideration. 

Sometimes measure concepts are identified as priorities by the MCAG but aren’t ready to begin full development yet. In these situations, measures can enter the pre-development stage, where PQA staff conduct analyses and research to help answer critical questions or develop methodologies to calculate a measure. For example, the PQA Persistence to Basal Insulin measure underwent pre-development to identify and validate an empirical approach to estimating insulin persistence. 

Vetting the Concepts: Public Comment   

Following the conclusion of the MCAG, prioritized measure concepts advance to public comment. Transparency is a pillar of PQA’s measure development process, and public comment helps all stakeholders stay informed. Furthermore, it allows PQA to leverage the knowledge of its members and the public to refine measure concepts. 

The public comment period lasts at least 15 business days and solicits feedback on each of the measure concepts discussed by the MCAG and recommended next steps for each conceptPQA public comment periods are broadly communicated to PQA members and relevant stakeholders to ensure all interested parties have an opportunity to weigh in. 

PQA reviews all comments received and may make changes to the proposed measure concepts based on feedback provided. Comments frequently raise important points or provide valuable new evidence on measure concepts. PQA staff summarize these comments and communicate them back to PQA members to ensure all stakeholders know their voice has been heard. 

Taking into account the MCAG’s input and the results of public comment, PQA finalizes its priorities and begins work on new quality measures. 

Get Involved: Public Comment

If you’re reading this, you can get involved in the conceptualization process. PQA members and the public will have the opportunity to submit comments on measure concepts prioritized for future development in 2022 and beyond through a 20-business day comment period, October 25-November 19 

This comment period includes four measure concepts, intended for health plan performance measurement in the domains of heart failure, opioid use, chronic obstructive pulmonary disease and adherence. For more information, see our October 25 email to members with details and links to measure information and the comment form. 

Share this post:

Comments on "Developing the Right Measures Begins with Effective Conceptualization"

Comments 0-5 of 0

Please login to comment