Jason G. Newland MD, MEd, Associate Professor of Pediatrics, Washington University; Director, Antimicrobial Stewardship Program, St. Louis Children’s …
[ Music ]
>> Our next speaker is
Dr. Jason Newland.
And Dr. Newland served on the planning committee
for this event today.
He's the Associate Professor of Pediatrics
at Washington University Medical School and
Director of the Antimicrobial Stewardship
Program in Children's Hospital in St. Louis.
His former position, he was on the faculty
at Children's Mercy Hospitals and Clinics
in Kansas City where he was the Director of
the ASP program and the Medical Director of
Patient Safety Insistence Reliability.
He said to me before I started, you don't
have to read everything that you have there
and I think he's really serious, so he's over
[ Multiple Speakers ]
[ Laughter ]
>> If I known you were reading all it, I wouldn't
have sent anything.
>> OK.
Well, actually, I don't think you did.
We just made it up.
But it's–
>> It's true.
[ Laughter ]
>> Let me just tell you that he– Let me read.
I won't read where you did all your fellowship
and that kind of stuff.
Oh yeah, yeah, yeah.
His current research spotlights the use of
antimicrobials and the impact of an antimicrobial
stewardship program at a children's hospital.
He's the co-founder of SHARPS.
Some of you may have heard of SHARPS.
Those of you who are in the peds arena would.
Sharing Antimicrobial Reports for Pediatric
He's the co-founder of that collaborative
and its purpose is to improve antimicrobial
use in children comprised of 42 children's
hospital in the United States.
This large national collaborative of stewardship
programs is dedicated to the appropriate use
of antibiotics for hospitalized children.
The other thing that I will tell you is, you
know, he just went from the blue side of the
state over to the red side of the state.
Not talking about politics folks, we're not
talking about politics.
We're talking from the blue royal side over
to the red cardinal side, but he's still a
true blue royal over in the St. Louis area.
Help me welcome Jason Newland.
>> Thank you, Shirley.
[ Applause ]
So, you know, Shirley has– you guys, what
you don't know about this is that the reality
of us all being here is because of Shirley
and because she is so passionate about making
sure we're all ready for this and what we're
going to do is stewardship.
So I just want to give a big round of applause
for Shirley, all your great work in getting
us here and doing this.
Also, we're– I mean we're blessed to have
great speakers and I think a lot of that,
you know, with Rob and Rebecca and Arjin [assumed
spelling] and the crew coming is– it's a
real treat and being someone that's, you know,
practiced on both sides of the state in regards
to Missouri, it's exciting, and this is an
exciting time.
And I also want to just thank you all for
being here because, you know, we are trailblazers,
as I think Arjin mentioned earlier.
I mean there's only one other state that has
a mandate and that's California and I was
shocked to think that he thought that we are–
mandate was more impressive than California's
because they've been doing this since 2009.
And I think a lot of that has to do with what
we're going to be required to do which is
the fact that our data, our antimicrobial
use data is going to be required to go into
the antibiotic use module.
And so, we're– I think that, to him, is obviously
a big deal and it's going to be a big deal
for us.
So, my hope is to kind of talk about data,
how we can use it and many of you guys know
And I hope to add upon what you've already
heard, build upon with Rebecca and others
have talked about today.
I do have some disclosures, had a grant from
Pfizer that started that large national collaborative
we're a part of and then been a diagnostic–
an advisory board member for a rapid diagnostic
So here are my objectives.
You know, I want to describe data and then
less potential effect of strategies both inpatient
and I have the role of talking a little bit
about outpatient.
So I'll give you a couple of interesting studies
around outpatient as well.
But before we get started because I realized
where I thought maybe this– maybe we wouldn't
know this but I want to find out these following
And you're going to show by raise of hands,
I don't have a clicker thing or we won't do
poll everywhere on your phone.
So– And you can– And this question, you
determine what you think is active or not.
I don't– You don't have to go by any definition.
So the first question is my hospital has an
active antimicrobial stewardship program.
All of those who have an active one please
raise your hand for yes.
OK, that's pretty good about a little over
half the room.
How many would say no?
OK, so there's others.
Good, good.
All right, next question.
I regularly look at antimicrobial prescribing
data for my hospital.
How many say yes?
OK, not as many hands.
Not as many hands but OK.
Next, I regular– regularly share our hospital's
antimicrobial use data with frontline providers.
Oh good, we have a few, we have a couple,
that's good.
We're going to come to talk to Alex and Joe.
I was one of these others, good.
All right, so I think that's telling, right?
I mean the reality of it is this question
here is going to be one of our core elements.
So we're all going to have to start figuring
out how we're going to do that better and
I'm hoping that with this presentation you
can have some ideas and hopefully I can even
think of better ideas because that's not been
something I've been strong at in the programs
I've been involved in as well.
OK, so I'm going to go through these different
bullet points throughout my talk.
One, identifying areas for improvement, so
how can we utilize data for identifying areas
for improvement and what kind of data is out
there and this will relate to a lot we heard
about the SAR that Arjin has spoke about and
others have today.
Talk about implementing effective strategies,
you've heard I think a great talk by Rebecca
on a framework of doing that which I really,
really like.
I'm going to steal that from you because I
think it's fabulous, in thinking about how
you strategize, but I'm going to do it in
the contest– context of the idea, say, guideline
as Rebecca kind of showed that.
I'm going to kind of go in a little bit depth
And then lastly is how do you evaluate because
I look at datas of, you know, how do we identify
what we're going to do and then how do we
evaluate what we're doing to demonstrate it
The reality of these programs as you all know
is that our consumers of the programs are
the frontline prescribers.
We're going to be involved and engaged in
what they're doing and we need to show them
that what we're doing is working and what
they're a part of is working.
And so we'll go through some of that and talk
about some process measures and some outcome
OK, so identifying areas for improvement.
Well, first off, there's regulatory management.
You know, there's this sepsis metrics.
That's about stewardship.
You can sell the septic metric with stewardship
because it is the one place where you can
say, "Hey look, we're not always about stopping
We're not always as about saying you can't
do something.
We really want to help you in this."
And that's a key place that stewardship can
definitely be involved in.
The other thing is avoidance of treating acute
There's obviously the surgical care improvement
project and metrics.
All of that are tied to money that stewardship
can definitely be involved in and most of
you probably have already accomplished those
things but again it's something that you can
The second one is benchmarking data.
How do we utilize that?
I'll give you a little example of some studies
that I've done in the past.
But again, you can utilize that benchmarking
data to identify potential areas.
I think that's the key and I think Rebecca
laid it out very nicely to say, "Hey, look,
our metrics right now for benchmarking are
quantitative-based," right?
They're not appropriateness-based.
They're not the optimal therapy-based.
So again, it's not saying, "Hey, I'm higher
than everybody else therefore I'm wrong."
I mean I think that's the key that we have
to remember in these.
And then finally, I think where you find your
passion or the areas that concern you is oftentimes
the areas you're going to be more successful
at because you kind of have an idea, it's
something that you feel strongly at and you'll
have the passion that'll often drive others
to improve.
So I want to go over one concept here that
I think we've talked about and probably many
of you already know this concept.
But I want to kind of show it on a table that
I've stolen from somebody else in the past.
And let's talk about days of therapy and length
of therapy.
And Rebecca mentioned this and– which is
important to think about these distinctions.
So, days of therapy is the total number of
antibiotics and the number of days they're
on antibiotics, right?
So it's giving you two aspects or you think
two concepts there, right?
So amount that you're receiving as well as
amount– how long you're receiving it for.
And then they're just length of therapy, which
just says I don't care how many antimicrobials
you're on but I'm going to count the length,
which matters because you could do a ratio
for either days of therapy over length of
therapy, and that will give you a sense of
how many antibiotics you're using per patient
in some respects.
So you can see here on your– in your respective
I don't know which one to point at so I could
just go around the room, but that one, this
we'll see the– it's too far away, we'll go
in that with that.
So if you look at number of antimicrobials
we had three, duration four, the days of therapy
would be 12, the length of therapy would be
So if we did this example, right, number of
antimicrobials is two, duration of therapy
five, your days of therapy would be 10, yes,
I heard that.
And your length of therapy would be five,
So I think it's important that we remember
these sorts of aspects and they are different.
The metric you will see presented and continue
to be presented is days of therapy per thousand
patient days.
That's the common use metric that we're still
That's what the antibiotic use module is based
But I do believe strongly that the length
of therapy can provide us additional information.
I additionally think, as we think about these
metrics is if you have a program that utilizes,
let's say, piperacillin and tazobactam to
treat perforated appendicitis, piperacillin
and tazobactam obviously those can be considered
a broad-spectrum antibiotic.
In the pediatric world, we like to use ceftriaxone
We had one antibiotic that I would argue as
broader than– or broader than ceftriaxone
But if I make that change, what happens to
your days of therapy?
It doubles, right?
So again, I think that's to demonstrate that
our metric still are in development.
One of which, we need to think about how we're
going to measure our spectrum, right?
How do we have a spectrum score from "oh look,
this hospital is using narrower antibiotics
versus broader" and how does that fit in the
context of appropriateness.
So, this crazy figure.
So, now, you know days of therapy and length
of therapy.
In this figure, this is a study that was kind
of demonstrating what the SAR would be like.
So if you say hey I want to get my SAR which
by the way we all will get, it's going to
give you this ratio of observe over expected.
And in this case, this Ron Pollack [assumed
spelling] out in Virginia basically was looking
at– look at all these hospitals a part of
the– a large academic health centers across
the country, and have length of therapy in
one bar days of therapy.
And these hospitals could see any of those
above one were outside the range of all the
other hospitals.
So the key about the observed to expected
is the expected is all of the data that is
compiled by all these hospitals.
That's what gives you the– ends up giving
you that denominator of expected and then
it's going to give you your observed.
The SAR is going to do the same thing, OK?
So we're going to have that same capability.
The other thing that was done in this study
was– OK, now, I'm going to drill down, right?
So I'm going to take observed to expected
and I'm going to drill it down to areas.
And so, in this example, they have a bone
marrow transplant unit where their days of
therapy had a observed to expected ratio of
2.5, right?
So it's way above what the expected ratio
would be.
It doesn't tell you what's wrong there but
it is– gives you this sense of where you
would go look.
Again, all of this would be what the SAR will
be based on, and it will be based on a lot
of groupings of antimicrobials that I think,
in the end, we'll have to see if it does benefit
us as we get that data back.
And I believe it will.
I think when you have data, as Rebecca showed
nicely, you see that people improve and then
it stimulates you to do more things.
So that's kind of benchmarking in some ideas
of how you could find areas to improve.
So the other thing is obviously implementing
effective strategies.
Now, there's tons of strategies.
We talked about them.
And I'm just going to go through them some
And I loved, again, what Rebecca said and
I am basically just going to repeat what she
said in many respect is one is all of our
institutions are unique.
I think Rebecca showed that very nicely what
the different hospitals she's had and how
they each had their own menu.
But you have to take that into account.
We're not going to all of us say hey you all
have to do, you know, prospective audit with
feedback of every antimicrobial in your hospital,
that's crazy.
Some of the hospitals will quit on us right
then and there.
That's not what we're after.
You have to strategize those– what you–
strategies selected must be tailored to your
What are your barriers?
What's your low-hanging fruit?
If you walk in the stewardship arena long
enough, you will hear low-hanging fruit all
the time.
And the reality of it is it's smart.
It's actually a very smart principle because
you will find those easy wins and people then
buy into it.
I think, again, Rebecca and her studies and
the case studies showed that.
You know that group that showed the reduction
in C. diff.
They were– There was probably another group
that wanted to work with the stewardship program
after that.
Because they saw great benefit of that proscribing
I believe strongly in this third point.
I think the best interventions are the frontline
provider's idea.
Now, mind you, you could think of it and what
you need to get them on board is the fact
that they completely believe in it as well.
The more that you have frontline provider
involved in that improvement efforts, the
more it happens, right?
The stewardship programs are kind of the on
the outside.
They're not necessarily always a part of the
team on the other peoples' eyes.
So if you can be a part of the team through
somebody that's truly on the team is sustainable.
And I think those in the room that have done
this a lot can tell you that's at least been–
that's probably their experience of the great
I also think that, you know, a lot of times
we think upfront sometimes that the physician
is the hurdle.
But, you know, nurse practitioners, you know,
the PAs, those who are working right alongside
these clinicians have a huge influence on
what happens.
And often and sometimes can be more approachable.
But you'll see that that's the way to get
that to that frontline provider with some
successful interventions.
So this has been mentioned.
This is the new guideline that came out in
It was underdevelopment for over three years
and it's the update to the 2007 guideline.
But this– the goal of this guideline was
to be more practical.
So I think all of us can be a– improve our
programs based on what we see in these guidelines,
especially those who are just getting started.
I think this can get– lays out nicely all
the different things you can do.
So I like this I– Amanda Haze [assumed spelling]
who's at VJC kind of really laid it out and
broke up into strong recommendations, weak
recommendations, and I really thought that
was a great way of thinking about it.
So I stole this from Amanda, so thank you.
But here are the strong recommendations and
I think it's really important to look at this
and think about where this comes from and
I'll show you some data about some different–
of these interventions.
So prospective audit with feedback is a tried
and true, and prior authorization also are–
the core strategies had have been in place
and then talked about since the 2007 guideline.
But now the strong recommendations are also
start talking about hey we need to reduce
duration of therapy.
I think we heard Robin speak earlier today
about how that has a significant impact on
C. difficile infection.
And CMS is very much going to be looking at
our C. difficile infection rates and that's
going to be something they're focusing on
in the future.
No doubt and based on what I've seen in both
the new proposed condition and participation
in some other work that I've been work– doing
with them.
In a dedicated PK monitoring programs IV to
PO conversion programs have been shown over
and over that these things work well.
So let's talk about those core strategies
especially for those who are about to be starting
programs that are thinking about "hey, how
am I going to do this?"
So, in our prospective audit with feedback
is hey, I'll let the clinician prescribe the
drug and then I decide what those drugs are.
I'm going to monitor and when I'm going to
come back behind that ordering of the drug
to review it and to make my recommendations.
Now, the big pros are you can get a lot of
information by waiting and allowing some of
them prescribe it.
I also believe it's the more collaborative
approach, right?
You're not allowing– You're not saying hey
you got to call me before you get permission,
That's kind of where that police mentally
gets in regards to the stewardship program
a lot of times.
I think there's a huge con, impaired therapy
is not impacted.
I mean you should know, I should– full disclosure–
I didn't– I should put this on my disclosure
My wife is ID pharmacist who's also a stewardship
pharmacist who is in this audience.
And she conducted a stewardship program in
Olathe Medical Center for years, and I often
heard how frustrated she was on asymptomatic
bacteria, right?
All of it, yes, I know.
This program will not stop you from treating
those asymptomatic bacteria, right?
They're already path.
And now, they've gotten like two or three
dozes, you know, like, oh man, it does, you
know, I can try to get it stopped.
But so, again, I think that's a key aspect
where you would miss.
And often, more resources are really needed,
You have to have someone dedicated to review
these antimicrobials and be watching this
and oftentimes going to talk to a lot of people
and finding those people.
So it does I believe take more effort.
So what about preauthorization formula restriction,
This is that you got to call me to get the
antimicrobials or I'm only going to restrict
that antimicrobial for certain group of people.
I think it definitely impact, you know, impaired
That's the way they get all rid of asymptomatic
It is more antagonistic, right?
I mean there's no doubt when you have someone
want to get something and they have to go
through you, that's going to lead to some
opportunities for interesting conversations,
so to speak.
But you can utilize those to your advantage
I will say that in my initial experience in
stewardship was at the Children's Hospital
of Philadelphia where this was the program
we ran.
And they would call you up and give you all
kinds of different information but you didn't
have a complete set of information.
You did not have all the cultural results.
Sometimes, they were just trying to say what
they could to get the antibiotics.
So I think that's an important aspect of prior
approval that can lead to some maybe bad recommendations
or things that aren't perfect about that.
But again, an effective strategy, and I tell
you these two strategies, again, they're core
And I think you'll find most programs use
a combination.
And the reality of it is not just one or the
other because, you know, we have these new
agents like ceftolozane/tazobactam or ceftazidime/avibactam.
You know, those are prior approval right off
the bat.
Now, because they're new and for the matter
of fact, most clinicians are like, "I don't
know what to do with them."
So, again, you can tailor it and then change
it over time, and I think that's where those
combinations work.
But there's always the debate of which core
strategy is superior.
And there's actually been studies on this,
and this is the one study that compared preauthorization
with prospective audit.
And now you have to realize that the initial
piece of this program was all preauthorization.
And this program, as you can see, was– is
in Pennsylvania and it have been in place
for, you know, many years, 16 years.
Now, what they decided was hey, let's evaluate
what happens when we change the prospective
audit feedback with cefepime, piperacillin/tazobactam
and vancomycin.
And this is how they set it up.
So you'll see in kind of this pre-intervention
which is to your left, they had all the drugs
on prior authorization.
And then to the right, they switched it.
They have prior authorization and there are
three drugs on prospective audit with feedback.
So what they find.
So you'll see that all of a sudden you have
some drugs that are kind of increasing and
those drugs would be all the broad spectrum
as a whole, so that was all the group there,
as well as those drugs they put on prospective
audit with feedback, though those drugs that
they left on prior approval stayed low.
So this is, in some respects, where some people
have really believed very firmly that the–
or the prior approval is possibly a better
Again, I wanted to show you this data as you
think about this.
I still think that there's definitely room
of how you can combine these strategies.
And I think we're still waiting to see more
data because many of us who done primarily
prospective audit with feedback programs have
found great success in building collaborations
and sometimes even more sustainable program
by doing that.
So durations of therapy.
I can't say in– I think durations of therapy
is a huge issue for all of us and where we
really have to focus on what we can do better.
One thing that we found is that just adding
in some stop orders, you've seen just reduction
in therapy.
Now, everyone gets nervous when you say oh
we're going to tell and they put a stop ordering
what happens if they miss it?
I think that goes back to the culture of how
we talk about antibiotics on our rounds.
If any of you got to listen to Alex and his
presentation earlier, the large health system
talk, you know, I love the fact that there
was this checklist that their ICU docs kind
of reviewed that aspect of antibiotics was
discussion of de-escalation, right?
If you're– If that is a part of it, if that
antibiotic timeout is truly happening everyday,
this isn't an issue.
Stop orders for antibiotics should really
be what we're striving for.
And I think that will help with, you know,
just the culture of antimicrobial prescribing.
You can definitely put this as a part of your
prior approval and your prospective audit
We've utilized this with quality improvement
projects, just finding some areas where you
know their durations of therapy are longer
and working on where there's evidence-based–
there are– there's evidence especially in
the adult population where you can definitely
shorten your durations of therapy.
And so here are the numerous studies.
Ventilator-associated pneumonia where they
look that eight versus 15 days, many of you
know that study and eight days was just fine.
And even I saw the new– the VAP HAP guideline
will– said seven days as the length of therapy.
And it wasn't clear to us that they even distinguished
out acinetobacter pseudomonas in that guideline.
Adult community-acquired pneumonia is– I
love the adults for this.
They say five days which is fabulous.
It's based on great data.
You know, if you go Johns Hopkins, I know
for a fact that's one of the biggest things
for them is that all their community-acquired
pneumonia even admitted, as after they show
improvement on afebrile, they treat them for
72 hours afterwards, and it's usually about
a five-day course.
Unfortunately, in pediatrics, we have a lot
of room to improve in that as our national
guideline still says 10 days based off of
expert opinion and no other studies.
So hopefully, that'll change in the future.
And then finally, cellulitis and abscess,
data again suggesting down to five days.
So with that, what about C. diff.?
So you've heard all about C. diff. and I don't
think I have to convince you.
But this is a classic study that we all talk
about regarding C. difficile infections and
the importance of both infection prevention
and control as well as antimicrobial stewardship.
So in this figure, you have two y-axes.
The one on the left is actually the C. diff.
rate which is represented by the blue bars.
And the one on the right is your days of therapy
per thousand patient days which is represented
by that orange line that you see kind of traits
and across that graph and then goes– it decreases.
And what you can see in the left half of that
figure is where that C. diff. rate continues
to climb continues to climb.
They put in some infection control and prevention
And then they added in the antimicrobial stewardship
efforts and you see this signification reduction.
And those are– that antimicrobial stewardship
efforts was to reduce clindamycin second and
third generation cephalosporins, [inaudible]
through I believe prospective audit with feedback.
So again, this has been shown over and over.
This is just one of the many studies that
have been done and obviously Rebecca presented
at one of her hospitals.
So you will see this as in these the adult
population by utilizing these sorts of strategies.
But I think the key here and a huge key is
the fact that infection and prevention and
control must be a part of it.
It's not the stewardship alone.
And again I think over and over you'll talk
about it, right, because if you can eliminate
C. diff., you're eliminating antibiotic use.
Just like in surgical site infection, just
like in the prevention of central line infection,
you're preventing these infections, you're
preventing the use of antimicrobials, and
I think that circle you'll see over and over.
So, it's an overall improvement in care and
overall improvement in the value for our patients
in our hospitals.
So, weak recommendation.
So, here are the weak recommendations.
And when you hear the word "weak", you're
like, oh, those must be terrible, right?
I mean that was first assumption, ah, you're
using weak.
But, remember, weak recommendation doesn't
mean that it's not a good strategy.
I mean they actually do work, it's just the
level of evidence for these strategies are
just kind of these observational studies,
pre/post intervention studies are not these
larger studies that have been for some of
the others strategies.
And I say that because development of guidelines
and protocols and ordinances are very, very
I think– I would assume most people in this
room who've done stewardship will tell you
that they put in together an order set and
they drove their practice by making like the
main antibiotic selection you could– all
you could pick.
They saw great reduction in inappropriate
use and they probably show– showed the improvement
So, I think prescriber-lead review is something
that we all are trying to figure out.
I think there's a lot of difficulties with
that and we kind of talk about it from the
cultural standpoint.
I think more and more we'll figure out about
rapid diagnostics.
I mean it's key to start utilizing them but
rapid diagnostics cannot be done on their
own, microbiology implementing them without
antimicrobial stewardship and others, helping
review and making sure the results end up
in the implementation of the correct intervention.
And finally, education.
You know, education is not sustainable, education
is what we have to do, but it needs to be
done with something else.
You have to be building that with, you know,
doing some prospective audit, adding in other
type of IV to PO, again, but you need to be
I think obviously the core element is going
to have us do that.
And then, obviously, joined commission wants
us to be doing that with our patients and
So, guidelines and protocols.
So, you can standardize the care which I think
obviously improves prescribing and improve
You can focus on that, treatment strategies
with selection and duration.
My experiences suggest clinicians are very
open to changing their selection with what
you recommend.
That's never been hard for us in at least
in pediatric space.
But they are– they hold onto their durations.
Changing a duration is really hard, right?
I mean, because usually your change in duration
is going shorter.
And for some reasons, 10 is way better than
seven or way better than five.
And so, again, you know, that's just the way
where I think clinicians have been hardwired
over time.
There are limitations.
They can take forever to develop.
If you're really going through all the steps
you need from the Institute of Medicine in
regards to what's a great guideline.
But I think that's where you utilize national
guidelines, to use those to make them unique
to your institution.
There's great national guidelines that have
been embedded in and the data has been looked
at over and over that you can utilize it.
Now, you got to make them where you can chew
on them and make it easier to use because
they're usually huge, right?
They're 50 pages, you know, like, how is anybody
going to do that?
So, you have to kind of get a group to kind
of, I would say, you know, clarify and make
it simpler for the frontline clinician to
And then, finally, and something that I think
we do a poor job just nationally and how to
do this is updating.
You know, just the other day, we were on the
Infectious Diseases Society of America's website,
looking at bacterial meningitis and they have
a guideline that was– is from 2004 that's
being updated currently.
So, again, that's a long time.
And what that literature then and with everything
that's happening to now, you have to take
into account.
So, I would suggest that all of those who
are going to think about guidelines and protocols,
you have to think about how you're going to
review those and update those, and using data
to help decide how that goes especially kind
of you're antibiogram data.
So, here's an example.
At Children's Mercy Hospital, we had this
great pharmacist in the NICU who worked with
out great stewardship pharmacist about hey,
they were tired of the way that necrotizing
enterocolitis was being treated in their hospitals.
And I brought this example because I'm thinking
maybe many of your hospitals that might primarily
adult will have NICUs in them.
So, the NICU pharmacist and a neonatologist
worked on this and us, as the stewardship
program, helped them developed with some ideas
we already had.
We were nervous about the fact that meropenem
was being using way more than we thought it
should be used in the neonatal intensive care
We also were wanting them to get off of vancomycin
because, you know, in the NICU, once vancomycin
started, it's hard for them to stop.
And so, that was another goal of ours from
stewardship perspective.
And so, they– we came together and literally
the NICU pharmacist says OK, we're– this
is how we're going do it.
Medical NEC is going to be vancomycin and
Medical NEC with suspect perforation or surgical
NEC, we'll give them vancomycin and gentamycin
and metronidazole and only meropenem if it
was septic shock.
The important part was really this part I
thought was that after 48 hours if their cultures
remain negative, they would deescalate from
vancomycin to ampicillin, all right?
So, we did give them little caveats for the
real very low birthweight babies.
So, it gave the NIC– neonatologist some ways
OK, I'm not so comfortable.
But, again, this was developed in conjunction
with the neonatologist and NICU pharmacist.
This was accepted very quickly within the
So, again, our worry was carbapenems.
So, we looked at our carbapenem data in this
figure on the y-axis.
You have days of therapy per thousand patient
In the x-axis, you have months.
And you can see over time, our meropenem use
had dropped significantly.
And I didn't look at vancomycin but the standardization
within that unit for NEC was very helpful
for us in the end.
So, good practice recommendation.
So, I bring out NICU before and you'll see
this here.
These are good practice recommendations.
Meaning, there wasn't enough data to make
any firm recommendations on what you should
be doing but the thinking is like in nursing
homes and skill nursing facilities, we should
be doing stewardship.
I think that we all realize that especially
in this audience.
And then finally, perform stewardship in NICUs.
And one thing that one can consider is there
are some studies that suggest limiting use
of cephalosporins and carbapenems will reduce
resistance as well as invasive fungal infections
and is likely better for the patient.
So, again, I think these are some ideas that
you can take especially for those with NICUs
and being that I'm a pediatrician I had throw
in some pediatric stuff.
All right.
So, lastly, evaluating.
So, I think, evaluating is key.
I think Rebecca demonstrated this nicely.
Evaluating your programs will help you learn
where you can improve other areas for you
to implement different strategies.
And finally, you need these data to go to
the administration into the C-suite to demonstrate
all the work that you're doing.
And I think it needs to be showing what you're
doing as well as the outcomes from what you're
doing as this is going to be important as
you move forward and likely to maybe even
get more resources.
So, evaluation, there is so many different
things I could put on these charts.
So, I think of process measures being one
of those things that you have to think about
in demonstrating what you aren't doing and
how things are going.
So, if you're doing prospective with feedback
or prior approval, how many reviews are doing?
What kind of actions are you implementing?
What kind of acceptance rate you have, right?
So, are clinicians buying in or is there or
are there certain areas where they're not
as accepting to these recommendations?
I think for those that you're doing guidelines,
how many of them are following the order set
for that guideline or what's the percentage
of use of that order set could be another
process measure.
And then, finally, the outcomes.
So, there's a lots of different things to
think about with outcome measures.
Days of therapy and length of therapy are
not obviously a clinical outcome.
But they are an outcome for your stewardship
Percent antimicrobial appropriateness, in
my opinion, is the gold standard.
And you heard Arjin in– this morning talked
about how difficult that is.
There's a lots of gray areas of antimicrobial
use in our hospitals.
But we have to striving to get at that appropriateness.
Because that's our goal, right, 100% appropriateness
of antimicrobial use.
I think cost and values is key.
I think we're going to be asked for that.
We're going to have to demonstrate that.
Now, I put both those terms up there because
cost you can show by showing reduction in
antimicrobial use.
You can show that possibly in the reduction
of maybe vancomycin levels because you're
doing less vancomycin use.
But what about the value?
How do you account for that C. difficile infection
you prevented that was suggested would cost
up to 10 to 12,000 dollars currently?
How do you add that in?
How do you add in the fact that you picked
up on the staph aureus bacterium were someone
was going to discharge them home with azithromycin.
And they were dialysis patient.
And then they found out endocarditis.
And what's that value?
So, there's a lot of aspects to think about.
It's more than just cost of antimicrobials
that these stewardship programs provide to
our hospitals.
And again, I don't have any– I can show you
this is exactly to do it.
But I think this is where the fields has to
go and for us to think about.
And as you are talking about the program,
one thing you can consider is, what were your
great catches that would have– add a cost
to it?
Oh, yeah, remember that bacterium we caught
and switched from azithromycin to amoxicillin,
they got treated the right way.
I mean that means something to administer
and the legality, right?
So, these are the things you have to think
about us telling those stories and patient
level stories administrators love to hear.
Hospital [inaudible].
You've seen length of stay, readmissions,
having mortality up on the list should be
there because you got to show people that
because you're stopping antimicrobials or
changing, that there's not a negative consequence
to your program.
So they're going to see that data.
Someone will probably ask you that.
So my experience comes from Children's Mercy
This will give you a little– what we've done
with some of that, what we did with some evaluation
and Jean Goldman and Charisma Bateler [assumed
spelling] here currently, and they're running
the program now and have done a fabulous work
in continuing it.
And the program at Children's Mercy was built
on prospective audit with feedback revealing
six days a week, talking about appropriateness
and duration.
And again, we believed in communication.
I haven't said this but I truly believe that
these stewardship programs are social programs.
These are about behavior changes and talking
with people and building relationships for
them to be ultimately effective and sustainable.
We in our program at that time decided that
we were going to have recommendations in chart.
Those only if you agreed upon them.
And again, communication, communication, communication.
If there's anything you remember, remember
And what did we find?
So again, this is about data, so we did.
We were looking data all the time.
And Jean Goldman published this paper here
in 2015, looking at and trying to learn where
our recommendations remained and what our
disagreements were and where our disagreements
And so, you can see, we reviewed lots of patients,
and it's great to show your administrators,
oh yeah, look at all the stuff we're doing,
They like to see that their money is going
to something.
And then you can say hey look, there's a lot
of stop therapy.
So they like to see that you're oh, they're
actually impacting, they're doing things.
We modify therapy such as changing doses–
or excuse me, modifying therapy being escalating,
de-escalating therapies, optimizing therapy
with changing doses and then getting ID consults.
And importantly, we looked at disagreement.
And that was, we had 22% disagreement which
really falls in line with I think most published
papers as well as what Rebecca had mentioned
earlier with her programs with the days on
And the other thing we decided to look at
was OK, what's predictive of having a stewardship
intervention, right?
So where are we making our recommendations.
Now, I show you this slide not to say, OK,
hey, look, here's all these crazy figures
and this is the antibiotics and all the stuff
behind it.
That's not the point here.
The point is look at the antibiotics.
And if you look on the y-axis is the probability
to have an antibiotic recommendation from
our stewardship program.
And you'll see the drug at the bottom with
the most likelihood to have an antimicrobial
recommendation was ceftriaxone and cefotaxime.
Mind you, what is likely the most common antimicrobial
in our hospital to use?
Ceftriaxone and cefotaxime.
So that was still a common drug and we were
still, you know, recommending a lot of things
on that.
And so, we– that– maybe that's just because
we see it all the time but you would think
maybe we had some standards of why we wouldn't
have to keep recommending to stop or to change
the dose.
So what were we missing there in regards to
improving the program?
The other aspect was we did the same thing
for indications.
And I'm just going to take you to the bottom.
And there's community-acquired pneumonia.
What I haven't told you is our hospital at
the time or hospital had a community-acquired
pneumonia guideline in place at this time,
but it was still the most likely diagnosis
for us to make a recommendation.
So we captured this data, we identified another
area which still is a common condition that
we can approve upon.
And so I would suggest and I was thinking
it by the time, if I was in your shoes, I'd
be thinking well, I think I can still go after
common conditions.
And sometimes conditions are easier to talk
about because there's more data about them
as well.
And so, again, I think that's where I might
look as a potential target if I was starting
a stewardship program today, and I kind of
am so that's what we're doing.
My pharmacists are like yeah, you are, right?
So the other thing we looked at was disagreements.
I think you have to evaluate where you're
not on the same page.
I think you have to be thinking about that.
Now, it's OK to disagree.
That's what we do on medicine sometime but
Where is that disagreement and what's driving
So we looked at this.
Not surprisingly, they like to disagree when
we said to stop linezolid or with something
with linezolid.
It was mainly to stop, trust me.
And carbapenems, likely the same argument
was stop.
So where is that?
What's the indication?
You'll see here in the most common indications,
it was ear, nose and throat which I can tell
you was trachitis in the intensive care unit
and community-acquired pneumonia still.
And so where is it?
Why are– Why is that where the disagreements
Who's the groups?
And so, again, I think that's the next step
is to evaluate further where you can improve
upon those disagreements.
So this is all that processed stuff.
I'm just talking about processed data.
So did it really– did the stewardship program
really work?
So I think you have to show this data to people.
And on this figure, again on the y-axis, is
days of therapy per thousand patient days.
X-axis says years but those– that data points
were all in months.
That light gray line is a group of 25 other
children's hospitals across the country.
And the black line was Children's Mercy Hospital.
And you can see when the antimicrobial stewardship
program was implemented.
And among the drugs that we monitored, you
can tell, it dropped.
And I promise you when you guys– those of
you who have started a program and those of
you who are about to start a program, this
will happen to you by walking and starting
to do something.
It happened.
People start buying into this and they realize
it's important.
And this saw– we saw with the stats that
there's 18% average monthly decline.
Now, that's great.
We were excited.
We used to say, hey, we're not bothering people
too much and it's making a difference, but
I– but we don't have clinical outcome data
to say hey it mattered clinically to the patients.
What we did have data was that we surveyed
the frontline clinicians.
So don't forget, they're actually the most
important part of your program.
They have to be bought into this.
And how do you build those relationships so
they are bought in is I think key to make
them sustainable.
So we surveyed our group.
And the top three are basically about quality
doesn't make them use antimicrobials better
and it has an improved patient care and you
see that they all thought that.
But there was a group that said, "Hey, they
take away our autonomy.
They're threatening."
Now, I'm not saying we were threatening, but
we did.
We took away some autonomy.
There's no doubt, they prescribe stuff and
we said, "Hey, you should stop that."
And that's going to be a true feeling that
some of your clinicians will feel.
And so I think that's part of these type of
So we did try to look at clinical outcomes.
And I think this is a struggle for all of
And I'm only bringing up just– I show you
this just to think about ideas of clinical
outcomes you might think about evaluating
in the future.
So we basically captured agreed versus disagreed
and they divided up these patients into different
groups that we thought were distinct.
So you were a medicine patient that didn't
have a lot of chronic care– complex conditions
which is the CCC.
And those who had a complex chronic care condition
and then we divide it into medicine and surgery.
And you can see that the length of stay in
this between those who agreed with our recommendations
or disagreed are about the same for all the
So that's– I was hoping we would see decrease
from those who agreed.
But at least I didn't say, hey, because of
what we're doing, all your– all of these
kids are staying here longer.
So that was good.
But we did have this readmission.
And if you look at readmission, same set up.
There's no readmissions on those who agreed.
Now this is small numbers.
But again, so it's suggestive of these matters,
this has a clinical importance and makes our
patients better.
So, again, something to think about.
Now, again, we talked about economic impact,
and I kind of jumped on my soap box earlier.
There are data that show you can have up to
$3 million within three year of savings.
That's great.
People love to see those numbers.
You need to use cost based on prescription
or administration, not purchasing data.
That's directly from the guideline.
As purchasing date– purchasing data can have
a lot of different things impacting it.
That makes it not as reliable.
The other thing is as your programs have been
developed, and some of you, I know, have been
doing stewardship for like eight years or
longer, it gets hard to keep moving the needle
down in antibiotic use.
So if you're trying to evaluate your cost
reduction comparing your last year to the
previous year, it might be the same if not
You got to be thinking of it in regards to
what the future projected cost would've been
if you would've never done anything.
And that's a key message that's going to have
to be stated to our administrators as we continue
to do these programs.
And again, I talk about this early– earlier.
Consider talking into account outcomes such
as length of stay, prevention of other infections
such as C. diff.
You can think of surgical site infections
and others.
Now, maybe one that you haven't thought off
in a place that you could do stewardship and
start is outpatient– or OPAT, so outpatient
parental antimicrobial therapy.
There's been a number of papers that have
looked at this.
This is just one example of a paper where
they– an OPAT team reviewed all antimicrobials
upon discharge, did number of safety interventions,
I mean, as Rebecca stated, doing the safety
interventions that impact cost as being the
highest priority.
OPAT is it.
I mean you can do a lot of great stuff with
OPAT by making sure they get the right laboratory
data, get the right duration.
Heck, make sure that they're not even on the
OPAT because they don't need it because you've
reviewed that.
This paper actually show that they discharge
delays are avoided for 35 referrals resulting
in 228 hospital days.
OPAT was avoided in 75 patients or the cost
savings of almost a half a million dollars.
Again, it could be a low-hanging fruit if
you have a system to look at OPAT therapy
in your hospital.
This has even been done in pediatrics.
This is a pediatric study where we know that
about 3% of our discharges go home on OPAT.
In this study out of Utah, 78% would've had
a possible recommendation such thing just
IV to oral conversion, so they did never have
to get the OPAT.
So again, these are potentially low-hanging
fruit that are right there at the [inaudible]
discharge that you could do a lot of big impact.
And the last, I don't know five, 10 minutes,
I'm going to talk about ambulatory use of
OK, so to step back, how many people have
clinics associated with their hospitals or
urgent cares or emergency departments that
they need to monitor?
OK, so we got some.
All right, so if we think about this, this
is a whole another bucket.
And I think Alex stated it best that really
what I just talked about and what we've been
mainly talking about in the use of antimicrobials
countrywide in the US represents about 3%,
There's about 3% of overall antimicrobial
use in our country.
And we haven't even talked about agricultural
industry which Alex said let's not even go
there because that's a whole nother couple
of hours of talking.
So the other part is outpatient use.
And this is a recent study that– this was
published, I think in May, 154 million prescriptions
annually in the US.
So that's about 12.5% of our ambulatory visits
were receiving antibiotic.
I mean and it's about half, you know, so 500
prescriptions for 1000 populations, so half
the population is getting antibiotic.
In respiratory conditions, by far and away,
drive it.
It's not even close for the other conditions.
Interesting, sinusitis is the most common.
And in this study, about 30%, 30% were inappropriate.
You know, you recall that Arjin mentioned
that the national goal for reduction inappropriate
antimicrobial use is 50%.
You can see that it should be easy for us
to reduce by 50%.
It shouldn't be a problem with as much of
inappropriate use we have.
So what are some strategies to do this?
Now, again, you're not going to be doing prospective
audit and feedback, right?
Because you can't, they're done.
They're out the door by the time you even
It's prior approval.
Are you really going to make someone call
you before they can leave to get the antimicrobial
Now– And they'll figure out a workaround.
So it's harder.
I mean now you're talking about a lot of behavioral
techniques to do this.
And so, I'm going to show you a couple of
studies that have been done regarding kind
of these behavioral techniques.
So I said audit and feedback in this– the
traditional way of the inpatient setting but
you can do audit and feedback in another sort
of way.
So this is a study from Jeffrey Gerber at
Children's Hospital Philadelphia.
Yes, it's a pedriatic-specific study but can
be pertinent to other– in the adult setting
as well.
And what they do is they divided their hospitals
into everyone getting one hour, or excuse
me, all their clinics which is about 25 clinics
getting one hour of education about the appropriate
use of antibiotics.
And as I've said earlier, education alone
won't always work.
And so, what they say is well, we're going
to give a part of the hospitals, this personalized
audit and feedback which essentially was they
gave on a quarterly basis all their clinicians
a graph that showed their broad-spectrum antimicrobial
prescribing for things like sinusitis, pneumonia,
otitis media.
All three conditions that just amoxicillin
to be treated, right?
So that's essentially what they do.
So what did they find?
Well, it really works, all right?
So– And Jeff Gerber is a really– is a friend
and a really, really smart guy and he'll demonstrate
how intelligent he is here in a second but
what you can see is that that tart– top bar
or top circles all represent the control practices.
And the orange, as you can imagine, is those
that got their, hey, I'm competing with somebody
else essentially, right?
Looking at my– how I did versus everybody
else, all right?
But what happened was that the grant funding
went away.
And so the– they quit sending those audit
and feedback reports about how well they did
versus their peers.
And lo and behold he got his second jam of
That's why I said he's brilliant being in
the academic setting.
Showing that hey look, it wasn't sustainable.
Even though they had taught them, they had
shown them, but once you pull that away, gone.
So interesting, right?
So– But remember this, right?
There's a little bit of competition there
and it worked.
All right, so that's one technique.
So here's another one.
So this is called the nudge.
Now, this was based in adult clinics and it
was in both Massachusetts and California and
it was like, five outpatient adult clinics
in LA and a bunch more I think in another
And based what– they signed a commitment
letter with a photo in the exam room.
And it was a pretty big one.
And it was a one-year observation period total
with interventions and 12 weeks of hot and
cold and influenza season.
So again, it was focused really on respiratory
tract infections.
Now, as you can see, this is what it looked
I'm showing my ugly mug with my crazy picture
when I first started with a lot more hair,
I think a lot more hair.
And you can see, it just says, you know, when
you have a cough or other illness, your doctor
will help you select the best possible treatment.
If an antibiotic would do more harm than good,
your doctor will explain this to you.
And we offer other treatments that are better
for you and so and so forth.
So this is explaining, right, [inaudible].
And this was all a big [inaudible] you can
And what did they find?
Well, the– essentially, what happened is,
the control practices that didn't have the
nudge in there, their antibiotic use increased
by 10% while the intervention, just by having
a poster in a room with somebody's picture
on it decreased by 10%.
So overall, you had about a 20% reduction
by just putting a poster in the exam room
of the provider.
Pretty simple, right?
I mean I'm like OK.
Now, the question is sustainability.
With a poster in there all the time, eventually,
it might wear off, and that's some of the
issues with behavioral economic approaches.
All right, so the other interesting thing
is what they stated in their discussion and
I'll read it to you.
When extrapolated to the entire United States,
the posted commitment letter intervention
could eliminate 2.6 million unnecessary antibiotic
prescriptions and save $70.4 million annually
on drug cost alone.
That's huge, just by a poster.
So, impressive.
So– But there's a newer study by the same
group actually that says hey, let's try some
other techniques.
And these techniques you might relate to a
little bit more as well.
So this is the study where they looked at
both Boston and LA.
And they randomized these clinics to receive
either none of these interventions or one
or two or three.
They do what they called a factorial design
to try to allow you to look at all these different
interventions and they suggestive alternatives.
So essentially is you come in, the doctor
is about to order something or the prescriber
is about to order something, it's a respiratory
tract infection, it says this doesn't need–
a popup comes up on their EMR and says hey
this doesn't require antibiotics, why don't
you prescribe– utilizing ibuprofen, rest,
those sorts of things and tell them that.
The next one was accountable justification.
Now, check this one out.
This one is really impressive.
So let's say they come in with bronchitis
and you try to write for an antimicrobial
or antibiotic.
It would popup and says this as indicated,
you need to justify why you're going to indicate
it, why you're going to prescribe it.
And then if you don't put anything in it,
in the chart, actually, this random chart
whatever you wrote, and also in the chart
says no justification given.
So imagine that, right?
You decided to prescribe and you don't give
your just– it's in the chart so you can see
Then the last one was peer comparison.
Now, again, Jeff Gerber did this, and now
these guys said they did it differently.
They basically divided them up into– they
emailed all the prescribers and said you were
either in the– if you're on the top 10%,
you got an email that says hey, you're in
the top 10% of prescribing.
Everybody else got– basically said, you're
not one of the top prescribers, and that's
what you got.
So you could imagine, right?
Think about a bunch of clinicians especially
doctors getting that, you know they all want
that email.
So what did they see?
So here's the figure.
You have across– so from left is the– that's
the– that's the accountable justification.
The middle one is peer.
And then the third is suggested alternative.
Now, when you look at these, the key is to
see when all these conference intervals separate
out so that you see why they're in-between
the orange line in the– which is the intervention
group and the control.
And essentially, the peer comparison worked
and the accountable justification worked.
So you guys have alternatives.
It kind of worked but it wasn't significant.
So again, I took away from this with these
couple of spaces, maker physicians and clinicians
competing with one another.
I mean it works.
And they will– And you give them some kind
of feedback, they will– they'll try to come
in line, and so something to think about.
So lastly is communication.
So there's data especially for those of you
who have pediatric practices where they've
done studies looking at how well we as clinicians
are at determining if a family wants an antibiotic,
So whenever I've asked this in the audience
is everybody almost raises their hands.
Of course I can tell when somebody wants an
Data will show that, first of, we don't ask.
We don't ask if someone is really expecting
And generally, they don't say they really
want an antibiotic.
They don't– They won't say that.
They might say, hey, you know, Johnny has
a sore throat and he's pulling on his ear
or, you know, Jimmy down the street just diagnosed
with strep throat but they rarely do they
actually say hey, I want an antibiotic.
So– And then when they've done the study,
this was out of Seattle, where this was OK,
let's figure out and they let them go in the
room and then come out and ask and they asked
the parents and then they looked at the end
and says when parents are not expecting antibiotics,
physicians will only write 70% at a time.
When parents were expecting an antibiotic,
right, so this whole thing, like, oh we think
they're expecting, physicians will only write
41% at a time.
And it's definitely been shown that if we
think you want an antibiotic, you're getting
it or you're getting a lot more times than
if I don't think you want it, right?
Because it's the expectation, right?
You want to make somebody happy.
We still want– And you worry about leap frog,
All these pressures that you have, they have
good leap frog scores.
Are you worried about the urgent care down
the street or an ED down the street [inaudible]
just give it and all of the payment is based
off of, you know, how many patients you see
right now.
I mean you can see how this happens.
So what we're going to try to do– actually,
one last thing before I get to that, sorry.
So the same researchers say hey let's look
at how we should talk to our patients.
So, one thought was positive or negative recommendations.
So positive being hey, you know, instead of
an antibiotic, I think, you know, you should
rest and we'll give you, you know, chicken
soup and, you know, being really positive,
never saying you don't need an antibiotic.
This is a virus, you know, being real negative.
That's the negative recommendation.
So she looked at comparing those two and then
lastly looked at giving a contingency plan,
which could be considered as delayed prescribing
for otitis media or saying hey I'm going to
call you back– we'll– call us in two or
three days if you have a problem.
So they looked at this data.
They had 28 pediatricians.
And basically, what they found was, if you
positive only, you get about a 50% reduction.
If you do both positive and negative, so you're
giving them something to do as well as say
hey, this is a not a virus, you got your greatest
Giving a contingency plan or negative alone
did not have any impact.
And so, it seems like maybe we– the script
we should be using has a little bit of both,
you know, trying to help as well as being
very firm about us believing this isn't virus.
So I think this is kind of where we have to
I think in the outpatient setting, you have
to be thinking how we can improve the communication.
So we're on the process of running trial where
we're going to basically ask the families
upfront, are they expecting antibiotic, give
them some education, ask them again, and then
relay that expectation to the prescriber so
they can kind of meet together and talk about
And then give a script of how you're going
to address that.
So with all this, I think one– and I've talked
about a lot of number of different things,
use data to identify as well as use data to
show the improvement, there's a number of
strategies you can use.
I think that's been talked about a lot.
I think you guys will do a great job and–
but I would highly encourage thinking about
how you're going to monitor that data.
And then finally, we have to get to the outpatient
stewardship space.
And so, I encourage, I challenge you to think
about how you might do that not only in the
clinic space but also in your emergency department.
So with that, I will take any questions.
We're right 3:30.
[ Applause ]
OK, break, break.
[ Inaudible Remark ]
Oh, no, no.
Find somebody else.
>> Stay there.
>> Yes.
>> So you talked about using length of stay
as a metric.
And we know there's a bunch of data on ID
consults resulting in longer length of stay
for things like staph aureus bacteremia.
Length of stay is an easy metric to get a
hold of administrators like to see it yet
we know that if an ID doc touches the chart,
there's a good chance, they'll have a longer
length of stay, they'll die less and they
may cost a little more and they'll be readmitted
So how do you use length of stay when we've
got conflicting data showing that die less
if we touch the chart but stay longer.
>> Yeah.
No, I think you have to– I mean obviously
in the staph aureus bacteremia, I mean that
has a lot of factors.
I think one of the issues around length of
stay is always going to be the number of other
factors that influence length of stay.
I think you have to look in conjunction with
other things.
Because if you can show length of stay decreases
and you could put more people in the hospital
after they're out, they will like that, especially
if the outcomes stay the same.
And I think that's the key, right, is how
do you kind of look at this from the other
impact of those outcomes.
Because the reality of it is the clinical
outcomes are going to have to be kind of altogether.
Because if I say, hey, the length of stay
is the same, the readmission rate is less,
they're going to like that in many respects.
So I think that– probably the take home is
that you got to take not just one but how
you're going to do all of them over time.
And again, the real– I think the [inaudible]
we don't have great clinical outcome data
at this point I think to show over and over.
I mean we've shown some resistance.
There's data that showed resistance decreases
but we going– we are going to have to improve
I think our programs to monitor those sorts
of things.
[ Music ]
>> [Background Music] Thank you.
[ Music and Applause ]
[ Music ]

Leave A Comment

Your email address will not be published