Vascular surgeons have a history of going the extra mile to do the right thing for their patients. So went the theme of one of the comments sent in near the close of a recent Society for Vascular Surgery (SVS) Town Hall, during which key figures behind the Society’s first-ever set of appropriate use criteria (AUC)—set to cover the treatment of claudication—laid out some of the mechanics of bringing the upcoming document to the brink of publication.
The seriousness of the undertaking was laid bare at the outset of the gathering by Larry Kraiss, MD, the immediate past chair of the SVS Quality Council, who oversaw the development of the AUC. Revealing the methodology adopted to carve out the criteria as the RAND Corporation method, he duly quoted the group’s definition of appropriateness in care: “An appropriate procedure is one in which the expected health benefit exceeds the expected negative consequences by a sufficiently wide margin that the procedure is worth doing, exclusive of cost.” The RAND/UCLA (University of California, Los Angeles) method involves group decision-making comprised of a systematic literature review, as well as both a writing panel and a multidisciplinary rating panel. Kraiss said one of the recurring questions he faces concerns why there are two types of documents addressing quality of care—the other being clinical practice guidelines (CPGs).
Kraiss outlined how the two should interact: “When there is strong and abundant evidence to support a grade 1a clinical practice guideline either for or against a particular decision, the appropriate use criteria should generally agree with those statements because they are both using the same evidence base. However, in more nuanced clinical scenarios, CPGs may not be able to make strong recommendations given the available evidence. CPGs may therefore deliver weak or conditional recommendations, and it is in these circumstances then that AUCs explicitly recognize the lack of good evidence and mobilize expert opinion to give guidance.”
AUCs step into this void, Kraiss added. The rationale is clear: “AUCs will allow us to formally state the SVS position in the public domain and provide a means to advocate against inappropriate overuse.”
Karen Woo, MD—who co-moderated the Town Hall in early August with SVS Past President Kim Hodgson, MD, and is a member of the seven-person SVS AUC writing panel for the maiden criteria—explained the reasoning behind the selection of the RAND method.
“It’s the only standardized, validated technique for determining appropriateness, and it combines evidence with expert opinion,” Woo said. “It’s often used when we can’t perform an RCT [randomized controlled trial] in every feasible clinical scenario.”
The process behind the method is scientific, Woo continued. First comes the systematic review and meta-analysis. The writing committee creates scenarios. The rating panel then rates these across two rounds on a scale of one to nine. These range from inappropriate (1–3) to appropriate (7–9), with an indeterminate scale in between (4–6). Ultimately, the aim is to achieve a set of criteria that will result in better patient outcomes “when they are adhered to,” added Woo.
Jeffrey Siracuse, MD, the immediate past chair of the SVS Appropriateness Committee and also a member of the writing panel for the intermittent claudication AUC, explained how the SVS arrived at claudication for its first set of criteria. After considering a number of conditions rather than specific interventions, the Society settled on claudication since “it is a perceived need by both the public and our Society,” he said.
Siracuse also broke down the multicameral nature of the rating panel. Eleven of the 15 members were drawn from the SVS membership, with two each brought in from the American College of Cardiology (ACC) and the Society of Interventional Radiology (SIR). The panel was also drawn from a broad geographical spread, a breadth of experience levels in terms of years in practice, and a nearly even split between academic and private or community surgeons.
In terms of the writing panel’s role, Siracuse explained that the team fleshed out relevant variables deemed important, as well as definitions, scenarios and assumptions, all reviewed by the Appropriateness Committee before being sent to the rating panel.
Jessica Simons, MD, the inaugural chair of the new SVS Quality Improvement Committee, walked Town Hall watchers through the arduous process of how the definitions, scenarios and assumptions were constructed.
“The final result was a mind-numbing number of hypothetical patients to consider and ratings to render— greater than 2,200, but I think ultimately it was meaningful and manageable since there was some striking agreement on a few high levels,” she said.
Woo emphasized the importance of the scenario construction portion of the process described by Simons. “I want to highlight how important this part of the process was because everybody has to be speaking the same language so everyone can rate starting from the same place,” she said.
Kraiss provided a short overview of the fruits of the panelists’ labors: the more than 2,200 individual scenarios rated over the course of 15 hours spread across two days.
“A complete and accurate portrayal of the results will require numerous tables and accompanying narrative describing the deliberations of the rating panel as they considered the various scenarios,” he said. “This is beyond the scope of today’s presentation; however, it is important to note again the remarkable degree of consensus achieved by the rating panel, where less than 0.5% of the scenarios met criteria for disagreement.”
Simons said next steps include peer review of the AUC manuscript by the SVS Document Oversight Committee, a public comment period, SVS Executive Board review and the eventual goal of publication in the Journal of Vascular Surgery. Woo, meanwhile, picked up on the theme of vascular surgeons doing the right thing raised by the viewer comment: “Since CMS [the Centers for Medicare & Medicaid Services] is moving towards pay for performance, perhaps if we can show that we’ve developed these criteria, that they benefit patients—that we can somehow capitalize off of that. I don’t know how that’s exactly going to happen, but I think at least—I’ll just speak for myself—I would rather go to my grave knowing that I did the right thing. And I think that was reflected in our panelists.”
