"guidelines are only as good as the evidence that underlies them"
Thursday, August 11, 2011 at 6:29AM
By ANaturalAdvocate
In the news lately has been an article entitled “Scientific Evidence Underlying the American College of Obstetricians and Gynecologists’ Practice Bulletins.” This article rates the evidence behind the obstetrical and gynecological recommendations made by ACOG as A (“good and consistent evidence”), B (“limited or inconsistent evidence”), or C (“consensus and opinion”) - levels used by the College itself. These recommendations “are meant to synthesize the best available data and make practical recommendations for clinicians” and are implemented “[t]o guide physicians in the implementation of best practice.”
The article itself admits the limitations of these recommendations from professional organizations, which are often used as “benchmark[s] for quality,” especially where high-quality evidence may be lacking. Given that these recommendations - which come from practice bulletins on topics from Operative Vaginal Delivery to Shoulder Dystocia and Fetal Macrosomia - “strongly influence the practice of obstetrics and gynecology,” it is imperative that “clinicians must remain mindful of the limitations of guidelines.” The authors do state that these recommendations should not take the place of clinical judgment, however.
“Among the obstetrics recommendations, level A evidence was noted for 24.6% of the diagnostic recommendations, 46.7% of the counseling recommendations, 20.9% of the guidelines for evaluation, 27.4% of the treatment recommendations, and 4.2% of the guidelines concerning mode of delivery. For gynecology, level A recommendations were found for 29.0% of the diagnostic guidelines, 35.0% of the counseling recommendations, 24.2% of evaluation guide- lines, and 38.1% of those recommendations that addressed treatment.”
While obviously the expertise and training of the physicians forming these recommendations must be taken into account in deciding the weight of “consensus and opinion,” it is clear that many recommendations do not have the evidentiary backing that many consumers assume is present.
A press release from the Big Push for Midwives illustrates the practical effects of these recommendations:
WASHINGTON, D.C. (August 15, 2011)—A study published this month in Obstetrics & Gynecology, the journal of the American College of Obstetricians and Gynecologists, found that barely one-third of the organization’s clinical guidelines for OB/GYN practice meet the Level A standard of “good and consistent scientific evidence.” The authors of the study found instead that the majority of ACOG recommendations for patient care rank at Levels B and C, based on research that relies on “limited or inconsistent evidence” and on “expert opinion,” both of which are known to be inadequate predictors of safety or efficacy.
“The fact that so few of the guidelines that govern routine OB/GYN care in this country are supported by solid scientific evidence—and worse, are far more likely to be based on anecdote and opinion—is a sobering reminder that our maternity care system is in urgent need of reform,” said Katherine Prown, PhD, Campaign Manager of The Big Push for Midwives. “As the authors of the study remind us, guidelines are only as good as the evidence that supports them.”
ACOG Practice Bulletin No. 22 on the management of fetal macrosomia—infants weighing roughly 8 ½ lbs or more at birth—illustrates the possible risks to mothers and babies of relying on unscientific clinical guidelines. The only Level A evidence-based recommendation on the delivery of large-sized babies the Bulletin makes is to caution providers that the methods for detection are imprecise and unreliable. Yet at the same time, the Bulletin makes a Level C opinion-based recommendation that, despite the lack of a reliable diagnosis, women with “suspected” large babies should be offered potentially unnecessary cesarean sections as a precaution, putting mothers at risk of surgical complications and babies at risk of being born too early.
“It’s no wonder that the cesarean rate is going through the roof and women are seeking alternatives to hospital-based OB/GYN care in unprecedented numbers,” said Susan M. Jenkins, Legal Counsel of The Big Push for Midwives. “ACOG’s very own recommendations give its members permission to follow opinion-based practice guidelines that have far more to do with avoiding litigation than with adhering to scientific, evidence-based principles about what’s best for mothers and babies.”
The Big Push for Midwives Campaign represents tens of thousands of grassroots advocates in the United States who support expanding access to Certified Professional Midwives and out-of-hospital maternity care. The mission of The Big Push for Midwives is to educate state and national policymakers and the general public about the reduced costs and improved outcomes associated with out-of-hospital maternity care and to advocate for expanding access to the services of Certified Professional Midwives, who are specially trained to provide it.
Media inquiries: Katherine Prown (414) 550-8025, katie@pushformidwives.org
Reader Comments (26)
What percentage of practices exclusive to midwifery meet A level evidentiary guidelines? Zero?
That's a red herring.
I would be interested in hearing your thoughts on the actual Green Journal article, though.
Amy: I couldn't say, because I haven't seen a study discussing it, although I'd be happy to post/review it, if you have one on hand. I agree with Jill that I'd like to hear what you have to say about this particular article.
From Jill, who is having trouble posting at the moment:
"My thoughts, not on the article but on how it might apply to
patients, is that most patients wouldn't care. If they have sought the
care of an OB-GYN, in my opinion they most likely trust consensus and
clinical experience because they placed themselves in the care of an
expert. Only when disagreements arise as to care would it be an issue,
and this would probably still apply only to a tiny percentage of the
patient population."
Amy: "What percentage of practices exclusive to midwifery meet A level evidentiary guidelines? Zero?"
Really?
Amanda, there's a pattern I see online in discussions that could fall within the area of birth politics. People are discussing midwives critically and someone pops in and keeps asking, "Oh yeah, but what about obstetricians and hospital birth? Answer me!" Similarly, when details of the practice of obstetrics or hospital birth is being questioned, someone throws the "BUT WHAT ABOUT OUT OF HOSPITAL MIDWIVES?", which is less relevant than the former in that 99% of births take place in hospitals. I know this pattern of behavior well. I can pull up a bunch of links to posts on which I've pulled that crap.
If anyone wants to discuss home birth with Amy, you know where to find her.
Let's keep it on topic. This is an interesting subject, no?
"That's a red herring."
Okay, so you acknowledge that there are no Level A guidelines to support any practice exclusive to homebirth midwifery. Therefore, it's hardly a red herring.
The press release is a classic in homebirth advocacy because it depends, like most things in homebirth advocacy, on DELIBERATELY MISLEADING lay people.
It sounds like an indictment (and it is meant as an indictment) to say that only one-third of ACOG guidelines meet Level A standards (as opposed to Level B or C). But it doesn't sound so bad when you acknowledge that ZERO practices exclusive to homebirth midwifery meet Level A standards.
... and 0% meet Level B guidelines; and even 0% meet the lowest level, Level C guidelines. That's because there is NO evidence of any kind to support the practices exclusive to homebirth midwifery. You know it, I know it and The Big Push for Midwives knows it. They're counting on the fact that most lay people don't know it.
The ultimate irony? The Level of Evidence for the actual paper that The Big Push is quoting, Level III, is the lowest possible level!!
"Really?"
If there are any, feel free to name them and to demonstrate that they meet the criteria for Level A guidelines.
Although I know the truth in the above article, it is disheartening to see it in print. What it all comes down to is your OB's expert opinion, with limited backing from scientific evidence. As with any one person's opinion, it is biased due to experience and education. Midwives are no different. I've seen homebirth midwives make scary mistakes, and I've seen OBs, in hospital make scary mistakes. It's because we are all human, and fallible. I've also seen fantastic OBs and fantastic homebirth midwives.
What we need is uniform continuing education requirements and oversight. (Navelgazing Midwife wrote a good bit about this recently). Nothing guarantees a standard of care. However, each of the specialties (OB, CNM, CPM, LM, etc) need to have their own, appropriate, evidence-based guidelines. Each set of letters gives the bearer different privileges, and differing levels of care. If you know the scope of your ability you know when transfer of care is required. That goes both up the scale for high risk births, and down the scale for low risk births.
so, I've digressed a bit. The big question becomes - how do you enforce the evidence-based standards of care? Statistics are great, but how do reframe a hospital system such that all of your OBs follow the same standards, when you know there are exceptions to every rule? When sometimes a gut feeling is more important than numbers. When a bit of confidence and faith in a mother can result in a vaginal birth, rather than a c-section? Sometimes quality of care isn't meausred in numbers, but in the happiness of the mother (and father!) who delivered that baby.
One of the reasons I became a Lamaze Educator, as opposed to other flavors of childbirth educator, was because of Lamaze's focus on "evidence-based practices". I teach my moms to ask questions, request studies, get second opinions, discuss birth plans well in advance, etc. Although the burden of proof is on the care provider, mothers still need to ask for the back up data. Trouble is, they usually don't.
"Although I know the truth in the above article, it is disheartening to see it in print."
The press release is a deliberate lie, meant to trick women and it has already tricked you:
Please explain the difference between Level A, B and C guidelines and how that differs from "no evidence."
Please list 5 practices that are exclusive to homebirth midwifery that meet Level A guidelines.
Can you name ANY practices exclusive to midwifery that meet Level A guidelines?
PS: Nothing in Lamaze is based on any scientific evidence and the only people who are unaware of that fact are NCB and homebirth advocates. Don't believe me? Find one of their recommendations that meet Level A guidelines. Hint: There are none.
Amy: Can you explain what you mean by "practices exclusive to midwifery"? I'm trying to sort out what my midwives did that would not also take place in an OBs office (although I can think of many things that would take place in an OBs office that did not take place in my midwives' office).
Regardless, what of this article? Do you disagree? Do you think that this is appropriate? Do you think that the level of evidence supporting a practitioner's recommendations (any practitioner) should be made clear to the patient, wherever possible (obviously emergency situations require different handling)?
Practices exclusive to homebirth midwifery:
Herbs
Chiropractic
Homeopathy
"spinning babies"
The Brewer Diet
Refusing induction for postdates
Home breech vaginal delivery
Home VBAC
Twins at home
Omitting GBS testing
Refusing antibiotics for GBS
Garlic for GBS
Refusing vitamin K injection
Refusing antibiotic eye ointment
There is no scientific evidence for any of these. I could go on, and on, and on.
Most importantly, there is no scientific evidence that American homebirth midwives are safe practitioners. The Johnson & Daviss study is a bait and switch. All state and national statistics show that planned homebirth with a homebirth midwife dramatically increases neonatal death and MANA categorically refuses to release their own death rates.
So you tell me, where's the Level A evidence for any aspect of American homebirth midwifery?
Amy: Very few of those are "exclusive" to homebirth midwivery, in my experience, although I think it's more "allowing for the refusal of X," etc., as the provider is not (generally) the one that does the refusal there.
So, you answer my questions about THIS article that we are discussing HERE.
"Very few of those are "exclusive" to homebirth midwivery"
Nice try, but irrelevant as well as untrue.
I notice that you have not been able been able to find EVEN ONE practice exclusive to to homebirth midwifery that meets Level A guidelines.
I also notice that no one explained what Level A guidelines are. In practical terms, Level A guidelines are based on multiple randomized controlled trials. As the authors of the study note:
"Compared with some fields in medicine such as cardiology, there are few randomized controlled trials in obstetrics and gynecology. Many
disease processes in obstetrics and gynecology are infrequent, present under relatively urgent circumstances, or are only rarely associated with meaningfully poor outcomes, all of which make the conduct of randomized trials either difficult or impractical."
And the authors of the study are NOT criticizing the practice of American obstetrics. Quite the contrary:
..."Although the College’s guidelines provide a reliable source of information based on the best available data, clinicians must remain
mindful of the limitations of guidelines. There is an urgent need to continue to conduct high-quality research in obstetrics and gynecology and to provide the funding to undertake such research."
So the fact that 2/3 of practice guidelines are not Level A merely means that no randomized controlled studies could be performed. I does NOT mean that there is no evidence to support those guidelines.
How many practices exclusive to homebirth midwifery have been tested using multiple randomized controlled trials? ZERO!
The press release does not mean what you thought it meant, but that's okay with the folks from The Big Push. They were trying to trick you and they certainly did.
How can you possibly trust an organization that attempts to prey on your lack of scientific knowledge in order to fool you? Obviously, you can't.
Jill,
Why did you backdate the post? According to Google cache, you published it this morning, not 4 days and 3 posts ago.
[Edit by Jill... Squarespace does not automatically update the date stamp upon posting. ANaturalAdvocate did not know this and it was she who posted it, but because she began the draft on the 11th, it posted as August 11. To change it after the fact changes the URL. Why did you want to know? Odd question.]
This is formulaic. Guarantee Amy is here to use the traffic of this site to stir up interest to a post griping about the TBP press release and counting on a few click-overs [just checked… yep].
Here’s the formula:
Begin with assumption #1: Laypeople are ignorant and don’t understand science.
#2: Complementary/alternative medicine practitioners (and advocates, which is Amy’s online specialty) deliberately try to dupe lay people out of their money at the expense of their health.
#3: CAM practitioners are scam artists, quacks and cheats; laypeople are too uneducated/ignorant to know better.
Amy works hard to promote the narrative that midwives are CAM practitioners and therefore quacks.
There is a glaring out group bias present about both midwives as a group and “home birth advocates” as a group. These out groups are presumed to be homogenous, which is fallacious reasoning. CNMs are midwives and many practice very similarly to doctors.
Similarly, the two groups, obstetricians and midwives and/or advocate are set up as diametrically opposed in order to generate debate.
___________________________________
Regarding the TBP pres release: Hardly sneaky or misleading. Any layperson without access to full text can Google the article and will find the abstract on PubMed:
Scientific Evidence Underlying the American College of Obstetricians and Gynecologists' Practice Bulletins.
Wright JD, Pawar N, Gonzalez JS, Lewin SN, Burke WM, Simpson LL, Charles AS, Dalton ME, Herzog TJ.
Divisions of Gynecologic Oncology and Maternal Fetal Medicine, Department of Obstetrics and Gynecology, Columbia University College of Physicians and Surgeons, and the Herbert Irving Comprehensive Cancer Center, New York, New York.
Abstract
OBJECTIVE:
Clinical guidelines are an important source of guidance for clinicians. Few studies have examined the quality of scientific data underlying evidence-based guidelines. We examined the quality of evidence that underlies the recommendations made by the American College of Obstetricians and Gynecologists (the College).
METHODS:
The current practice bulletins of the College were examined. Each bulletin makes multiple recommendations. Each recommendation is categorized based on the quality and quantity of evidence that underlies the recommendation into one of three levels of evidence: A (good and consistent evidence), B (limited or inconsistent evidence), or C (consensus and opinion). We analyzed the distribution of levels of evidence for obstetrics and gynecology recommendations.
RESULTS:
A total of 84 practice bulletins that offered 717 individual recommendations were identified. Forty-eight (57.1%) of the guidelines were obstetric and 36 (42.9%) were gynecologic. When all recommendations were considered, 215 (30.0%) provided level A evidence, 270 (37.7%) level B, and 232 (32.4%) level C. Among obstetric recommendations, 93 (25.5%) were level A, 145 (39.7%) level B, and 117 (34.8%) level C. For the gynecologic recommendations, 122 (34.7%) were level A, 125 (35.5%) level B, and 105 (29.8%) level C. The gynecology recommendations were more likely to be of level A evidence than the obstetrics recommendations (P=.049).
CONCLUSION:
One third of the recommendations put forth by the College in its practice bulletins are based on good and consistent scientific evidence.
LEVEL OF EVIDENCE:
III.
______________________________
So the question here is not whether some fringe midwives who promote that rainbows shoot out of your vagina as long as you give birth in a lake are practicing according to the best evidence available. In fact, it is no more relevant that complaining about another surgical speciality’s practice bulletins or jumping all over acupuncturists, chiropractors, massage therapists, etc. It’s not relevant at all to the article in the Green Journal. Why work so hard to play distraction games? That smacks of defensiveness and fear.
HOWEVER, why would TBP put out this press release? Katie Prown writes a mean press release, but did they accurately put the study’s findings in context? Does the finding that only “One third of the recommendations put forth by the College in its practice bulletins are based on good and consistent scientific evidence” actually indicate the need for maternity care reform? Can the purported increase in women seeking out-of-hospital alternatives to OB-GYN care for pregnancy and birth really be attributed to that two-thirds of consensus/opinion based guidelines in the practice bulletins? Wouldn’t a press release by a campaign for licensing and access to CPMs be inherently based on a financial conflict of interest?
I was glad ANaturalAdvocate posted this because I am curious as to what it really means that only 25.5% of obstetrics recommendations are based on Level A evidence. Does anyone really care?
Amy: As you can see from the byline, *I* posted this, not Jill. I am still gathering familiarity with the Squarespace formatting and made a mistake. I tried to correct it, fubaring it even more so. Jill was kind enough to correct the issue so that the URL would work properly. I am happy to place a disclaimer at the beginning of the article if you feel that the date-of-draft-start is somehow misleading when attached to the post-that-just-went-up.
As far as your exclusive comment, I don't see how it is irrelevant - especially since you keep screaming about midwives - and it certainly is not untrue. I know, personally and professionally, a number of OBs (so not CNMs, DEMs, CPMs, whatever) who use, offer or recommend some of your examples.
Also - AGAIN - this article is about obstetricians and the lack of solid evidentiary basis for a number of recommendations put out by them. What do you think about that? Do you think that patients have the right to know the evidentiary basis of the recommendations of their providers (OB, CNM, whatever)? Do you think that this is a problem to be solved? Do you think it's a problem at all?
" Any layperson without access to full text can Google the article and will find the abstract on PubMed:
Not good enough. You must read the actual article.
If you did, you would find out:
!. The authors are NOT criticizing obstetrics.
2. Other specialties have a LOWER proportion of recommendations that meet Level A guidelines.
3. This is NOT about whether obstetricians are following science. The authors are quite clear that they believe that obstetricians are following science.
The Big Push for Midwives deliberately misrepresented the article, counting on the fact that NCB and homebirth advocates would never read the article (you didn't), lack the understanding of what the levels mean, have no idea how obstetrics compares to other specialties and, most importantly, would never consider the fact that ZERO % of practices exclusive to homebirth midwifery meet Level A guidelines.
"Also - AGAIN - this article is about obstetricians and the lack of solid evidentiary basis for a number of recommendations put out by them."
How would you know? You haven't read the article.
The press release completely misrepresents the actual article. If you read the article, you would realize that.
This is a perfect example of how NCB advocates spread misinformation. You write about an article you never read, you rely on a press release and then you thoroughly misrepresent what the authors wrote.
READ THE ARTICLE! DON'T WRITE ABOUT PAPERS YOU HAVEN'T READ!
Amy: I have both read the article. The entire article. All of it. All of the words, all of the references, the entire article. I am not just going on an abstract or on a press release. You may note that I quoted phrases from a number of sections of the article. Are we done now? Can you answer the questions I asked, or make some comment about this, or will you merely continue to toss other items into the discussion? I urge you - most strongly - to please stop assuming that anyone not following your particular position has not read (at least a number of) the ACTUAL, REAL-LIFE, FULL-LENGTH academic writings in the area. Thank you.