Log in using OpenID

AANS and CNS Send Letter to NQF Expressing Concerns with the

December 05, 2014
Chris Cassel, MD
National Quality Forum
1030 15th Street, NW
Suite 800
Washington, DC 20005
Re: NQF Measure Endorsement Process
Dear Dr. Cassel:
The undersigned organizations are writing to share our feedback regarding recent improvements
and changes to the National Quality Forum (NQF) measure endorsement process. We applaud
NQF for continually updating its process for quality measure evaluation and endorsement and
have been encouraged by a number of improvements. However, we believe there are still
deficiencies that need to be resolved, particularly around endorsement review timelines, the
implementation of updated evaluation criteria, measure testing and eMeasure requirements.
In this current healthcare environment where payers and purchasers are increasingly emphasizing
paying for value over volume, we believe that the NQF has an important role to play in ensuring
that the portfolio of endorsed measures includes a sufficient number of meaningful measures for
all physician specialties to report, and that those measures contribute to improving the quality of
care physicians are providing to patients. This is even more pressing as the Centers for Medicare
& Medicaid Services (CMS) has begun to remove the first generation of measures from its
physician quality programs, which widens the gap of available measures for specialties in the
CMS measure portfolio.
Endorsement Timeframes
As you are aware, the house of medicine as a whole has been a leading – if not the largest –
contributor of quality measures submitted to the NQF for endorsement. We realize that the NQF
often operates under contractual obligations set by CMS to review measures, which de-facto sets
NQF’s calendar for clinical topic areas and the associated timelines for projects. Unfortunately,
strictly following CMS’ topic priorities and timelines is hindering the ability of some developers
to get their measures evaluated by the NQF within a reasonable timeframe. We understand the
NQF needs to address in any given year those topic areas identified by CMS as priority areas.
However, this has meant significant delays for certain topic areas. For example, the Head Eyes
Ears Nose and Throat (HEENT) project was originally scheduled to take place Fall 2012, and
only recently appeared on the project schedule with measures due Spring 2015.
We acknowledge and are supportive of the NQF moving toward Standing Committees, which
appears to have reduced the endorsement cycle for the topics addressed and led to a more
consistent endorsement process. Still, our own experience and that of other measure developers
Chris Cassell, MD
December 5, 2014
suggests that for some measures the endorsement cycle – from submission to final decision – is
longer than the seven-months NQF has described to its membership.
When NQF launches an endorsement project, measure developers assume that a standing
committee will complete its review and make a determination within the reported average seven
month timeframe. It has come to our attention that NQF may choose instead to review measures
in phases, thereby stretching out the time of review. We seek to understand how the decision is
made to review a submitted measure promptly, as NQF communications suggest is now standard
practice, or to delay a review. We are aware of a recent instance in which a society measure
developer was informed that review of a measure submitted in December 2013 will not take
place until June 2015, having been assigned to a “later phase.” Clearly, such decisions threaten
the support for measure development within the specialty societies. These societies have
committed enormous resources to develop measures and expect prompt review of submissions
within the seven month average time reported by NQF. Society quality staffers are accountable
to their boards and members for forecasting when the endorsement process will be completed,
and when a measure, if endorsed, will be available for use by its members. It is critical that the
NQF provide specific reasons for delays in writing to the developer so that information can be
shared with their boards and volunteers. The credibility of NQF and of society measure
developers depends on transparent communication and clear expectations.
The delayed timeframes for project commencement and phasing of projects resulting in a
delayed measure review have significant implications given NQF’s frequent changes to its
endorsement criteria and submission forms. As a result, a measure may be reviewed against
endorsement criteria that were changed long after its initially scheduled endorsement review and
even longer after the development project began. Such changes in criteria can force measure
developers to return to the drawing board, invest additional resources, and extensively revise
submission materials. This is particularly problematic when new requirements are applied to
measures already in the review queue that have been assigned to a later phase of the project.
A measure submitted for maintenance endorsement in December 2013 for the Cardiovascular
project illustrates the problem. The eMeasure submission and testing requirements had changed
in late October 2013, long after the testing project for the measure had been completed and well
into the submission process. In this instance, NQF graciously agreed to offer some flexibility
and allow for the review of testing results for a project conducted when the old criteria were in
place. While that accommodation was greatly appreciated, it serves to highlight a larger
problem. Therefore, we request that NQF consider phasing in new requirements and submission
forms and build in a reasonable period of time before the changes take effect. If endorsement
criteria change, NQF needs to institute a process for “grandfathering” performance measures
using earlier criteria.
Measure Testing Requirements
In general, we are supportive of stronger measure testing requirements as they ensure
performance measures are reliable and can be appropriately captured in the clinical setting.
However, the recent changes to the testing requirements for electronic measures pose significant
Chris Cassell, MD
December 5, 2014
barriers and additional burdens. In the current environment, measure testing takes approximately
one year. This is outside the time required to develop and specify a measure. Therefore, there is
the need for the NQF to provide advance notice, of two years at a minimum, of changes in
testing requirements, so testing plans can be modified accordingly.
Similarly, under the NQF's current process, the requirement of three electronic health records
(EHR) systems has been incredibly challenging to accommodate. The requirement is imposing
an extremely intense level of resources for testing and we have found that recruitment of sites
with three different EHR systems can take anywhere from three months to one year. This time
does not count toward the anticipated one year to formally test an eMeasure. While we
recognize that testing an eMeasure in one EHR system would not be a proper demonstration of
reliability and feasibility, testing an eMeasure in two EHRs systems at a minimum of three
different practice sites should be sufficient.
Furthermore, we agree with the requirements expressed by the NQF on testing composite
measures in the claims/administration, registry, and paper modalities. Demonstration of
scientific acceptability of each individual measure and the composite as a whole is expected.
However, the guidance and requirements for testing eMeasure composites are not accessible or
tailored to what is achievable to reach endorsement. In addition, there is a need for additional
guidance surrounding the recommended methodology used to demonstrate component measure
fit as a part of the composite measure conceptual construct.
Review of eMeasures Under Consideration for Endorsement
We support the eMeasure for Trial Use Pilot that NQF has initiated and recognize this pilot as a
key pathway to address some of the timing challenges related to submission requirements at the
time of the call for measures. We look forward to continued engagement in the eMeasure for
Trial Use Pilot program in the coming years. However, we would like to highlight challenges
that we have encountered in submitting eMeasures for consideration of endorsement.
As the NQF is aware, the national effort to standardize electronic specification of quality
measures is a process involving continuous improvement. There are several governance groups
that drive the standards, tools, and accepted practices used in the development of eMeasures, as
well as the update schedules for implementing them. Due to these external factors and the
continuous improvement process for these standards, the standards and tools specific to the
development of eMeasures are constantly undergoing updates to improve the adoption of
eMeasures. As a result, due to the length of time between submission of an eMeasure to NQF
and its review by the Steering Committee, it is a challenge to ensure that the version reviewed
reflects the most up-to-date standards, tools and accepted practices for eMeasures. The
challenge is even greater when there are delays in the review process, or in phased projects.
Our experience is that NQF seems unaware of many of the external factors that impact measure
developers and the ability for measure developers to meet the eMeasure requirements. Measure
developers are working diligently to adhere to the national standards while, at the same time,
meeting the requirements that NQF has set forth. Nevertheless, it is unlikely that the timelines
Chris Cassell, MD
December 5, 2014
for submission, review and endorsement of an eMeasure will align with the updates to the
standards and tools. We recognize this is a very technical aspect of measure development and
welcome the opportunity to discuss this further.
Based upon our experiences outlined above, we provide the following specific recommendations
for improvements. The NQF should:
1. Outline the timeline for endorsement from submission to final decision, taking into
account any project phasing that might occur;
2. Base the review process on the criteria that were in place at the time of the submission
rather than the time of review;
3. If testing requirements change, provide two-year advance notice to allow for planning
and completing a new testing project to satisfy the new requirements;
4. For eMeasures, require testing to occur at a minimum of three sites with two EHRs;
5. With stakeholder input, outline the exact requirements for demonstrating scientific
acceptability of an eMeasure composite in terms of feasibility, reliability and validity in
one comprehensive form; and
6. Acknowledge the evolving state and external factors that measure developers face in
developing eMeasures, and exercise flexibility when reviewing eMeasures under
consideration for endorsement.
Next Steps
Without a concerted effort to improve how clinically relevant measures are embedded in the
many and quickly expanding federal quality programs, physicians will be left without an
opportunity to meaningfully participate in these programs, potentially forcing them to leave
Medicare and threatening access to high quality care for Medicare patients. By addressing our
concerns, you can help ensure that more performance measures that matter to patients,
consumers and clinicians will become available.
Thank you for your attention to our concerns. We stand ready to work with the NQF and other
relevant health care stakeholder groups to improve the current quality measure endorsement
American Medical Association
American Academy of Allergy, Asthma & Immunology
American Academy of Dermatology
American Academy of Neurology
American Academy of Ophthalmology
American Academy of Orthopaedic Surgeons
American Academy of Otolaryngology—Head and Neck Surgery
Chris Cassell, MD
December 5, 2014
American Academy of Physical Medicine and Rehabilitation
American Association of Neurological Surgeons
American College of Cardiology
American College of Emergency Physicians
American College of Radiology
American College of Rheumatology
American College of Surgeons
American Gastroenterological Association
American Osteopathic Association
American Psychiatric Association
American Society for Clinical Pathology
American Society for Gastrointestinal Endoscopy
American Society for Radiation Oncology
American Society of Anesthesiologists
American Society of Cataract and Refractive Surgery
American Society of Clinical Oncology
American Urological Association
College of American Pathologists
Congress of Neurological Surgeons
Heart Rhythm Society
Infectious Diseases Society of America
North American Spine Society
Renal Physicians Association
Society for Cardiovascular Angiography and Interventions
Report inappropriate content