00:00:05
our research really uh revolves around a
00:00:09
problem that chief medical officers have
00:00:13
uh they open up the morning paper and
00:00:14
they find that their Hospital's been
00:00:16
ranked very low in some quality of care
00:00:19
report uh and their initial reaction is
00:00:21
always our patients are really sicker
00:00:24
and that's why we look so bad in the
00:00:26
reports uh This research template
00:00:28
matching uh tries to improve the reports
00:00:32
so that there's a fair comparison across
00:00:34
all hospitals and we really tried to
00:00:38
look at a the problem in a different way
00:00:41
using multivariate matching which has
00:00:43
never been done before in looking at
00:00:45
quality to uh be able to fairly compare
00:00:50
hospitals uh and then the chief medical
00:00:53
officer wouldn't be as upset because
00:00:56
he'd probably believe either there was a
00:00:58
problem or he'd look good in the reports
00:01:04
template matching is a new way of
00:01:06
looking at the way hospitals uh uh treat
00:01:09
patients um the usual problem when we
00:01:12
compare hospitals is that we say uh here
00:01:14
are all the patients that you saw and
00:01:17
then if your patients were going to be
00:01:19
going to another hospital or the typical
00:01:21
Hospital how would they do um that's uh
00:01:24
a strange way to compare hospitals
00:01:26
because each hospital has a different
00:01:28
set of patients and so it's really not a
00:01:31
fair way to compare hospitals cuz one
00:01:34
Hospital might have an easier set of
00:01:36
patients we're going to make some
00:01:37
adjustments but they still have a very
00:01:39
different set than another hospital that
00:01:41
might have a much different uh more
00:01:43
difficult set of patients what we
00:01:45
decided to do was create a template of
00:01:47
patients meaning a set of patients in
00:01:49
our case it was 300 patients with
00:01:51
general surgery and orthopedic surgery
00:01:54
and we said let's take a a relevant
00:01:56
template and then match at each Hospital
00:02:00
their patients that would fit the
00:02:02
template so we ended up with 300
00:02:06
patients being matched at 217 Hospitals
00:02:10
now we have very similar patients at
00:02:13
each of the hospitals because they've
00:02:14
all been matched to this cookie cutter
00:02:17
template which is producing very similar
00:02:20
patients across hospitals so now we can
00:02:22
make a very fair comparison because
00:02:24
we're looking at the same in a sense 300
00:02:27
patients at each and every hospital
00:02:30
which is completely different than the
00:02:31
way almost all report cards are made
00:02:33
today where they look at the patients at
00:02:36
an individual hospital then they try to
00:02:39
estimate how would they have done at
00:02:41
other hospitals where each hospital has
00:02:43
different patients so we're completely
00:02:46
changing things around the technique is
00:02:48
and there's another word for it called
00:02:49
direct
00:02:50
standardization um and what we're doing
00:02:52
is saying you have to have a fair exam
00:02:55
the fair exam is how did the hospitals
00:02:57
do on these 300 patients we find the
00:03:00
hospitals 300 patients that look like
00:03:02
the template and that's what we compare
00:03:04
across hospitals so now it's very hard
00:03:07
for a chief medical officer to say our
00:03:10
patients were really easier sicker than
00:03:12
another hospital it's very hard to do
00:03:14
that because they've the patients we're
00:03:15
going to be examining at their Hospital
00:03:18
are the same 300 that are at each and
00:03:20
every other Hospital same in the sense
00:03:22
that we have multivariant matched them
00:03:24
which means that we've matched them on
00:03:26
literally hundreds of characteristics so
00:03:29
that they very very similar and their
00:03:31
risk of doing poorly or their chance of
00:03:33
doing well is very very similar across
00:03:35
all the hospitals what we say in the
00:03:37
paper what we show in the paper is that
00:03:40
the these 300 patients at each and every
00:03:42
hospital are incredibly similar their AG
00:03:45
is the same their rate of diabetes and
00:03:47
heart failure and all the other
00:03:49
characteristics we'd be interested in um
00:03:52
are incredibly similar statistically UND
00:03:56
differentiable to the other
00:03:58
hospitals um but yet what we find is
00:04:01
that the outcomes are very very
00:04:07
different I think that there are two key
00:04:10
takeaways the first is that we can do
00:04:12
the matching uh and what we showed was
00:04:14
that you can get very close matches
00:04:16
across hospitals each Hospital Stamped
00:04:19
Out to this template of patients the
00:04:22
second is that there's just great
00:04:23
variability in the way the hospitals
00:04:25
handled the patients and the outcomes of
00:04:27
the patients and uh I think seeing that
00:04:31
the matches were so close and at the
00:04:33
same time seeing that the outcomes were
00:04:35
so different was uh something that I
00:04:38
think people should realize that there
00:04:39
is different there are differences in
00:04:41
quality across hospitals and that there
00:04:43
are I think better ways to measure them
00:04:45
than what we've been doing up to
00:04:49
now probably the biggest surprise was
00:04:52
that after we did this very close
00:04:54
matching such that all of the hospitals
00:04:57
had an incredibly similar sample of 300
00:04:59
patients there was great variation in
00:05:02
outcomes some hospitals had high
00:05:04
complication rates some had low
00:05:06
complication rates some had high
00:05:08
mortality rates some had low mortality
00:05:09
rates things that when I would see this
00:05:12
in a standard report I might not believe
00:05:15
because I knew the hospitals were seeing
00:05:16
different patients but here after having
00:05:19
matched so closely to see this great
00:05:21
variation in outcomes I think that was
00:05:23
an eye
00:05:28
opener we've given that's some thought
00:05:30
uh We've usually been applying this
00:05:33
technique to the hospital sector but I
00:05:35
think you could easily apply this to
00:05:37
nursing home quality schools um we can
00:05:40
imagine creating a template of students
00:05:42
and stamping the students out across
00:05:44
different schools seeing how the student
00:05:46
outcomes were so yes you could you could
00:05:48
U you could use this technique in other
00:05:50
areas I think the key here is that we're
00:05:53
taking uh a uh from one field which is
00:05:57
from statistics and in particular
00:05:58
multivariate m matching for which Paul
00:06:00
rosenbom is the world's expert in in
00:06:03
that area uh and we're applying it to
00:06:06
Quality assessment and I just don't
00:06:08
think that's been done and it certainly
00:06:10
hasn't been done with a a template which
00:06:12
kind of levels the playing field and
00:06:14
lets you really see if your quality is
00:06:15
different than than
00:06:20
others I I think the biggest problem is
00:06:23
that people um tend to uh believe report
00:06:27
cards based on what's called direct
00:06:30
standardization where we're looking at
00:06:32
different patients at different
00:06:33
hospitals different types of patients
00:06:35
even non-overlapping populations and
00:06:38
saying that we can extrapolate from the
00:06:39
models to say that one Hospital's
00:06:41
quality is better than another I think
00:06:43
that's a misconception I don't think
00:06:44
that the models currently allow us to do
00:06:46
that and that if we had better
00:06:48
techniques and I think this is one of
00:06:50
them uh we'd be able to at least provide
00:06:53
confidence to the reader both the user
00:06:56
in terms of the patient and also the
00:06:58
people at hospitals that are trying
00:06:59
trying to improve safety and quality
00:07:01
that uh there really are differences or
00:07:03
not and I think uh uh that's probably
00:07:05
the the main
00:07:10
point there are many groups that are
00:07:12
grading hospitals the federal government
00:07:15
the it's called Hospital compare the
00:07:16
website that lets you click on a
00:07:18
hospital and compare it to another
00:07:20
hospital uh has been doing this since
00:07:23
2007 um there are many different
00:07:25
organizations uh and and web-based
00:07:28
companies that uh great hospitals uh
00:07:31
they often look to hospital compare
00:07:33
which is the main model for example
00:07:36
Consumer Reports looks to hospital
00:07:38
compare um Hospital compare suffers from
00:07:40
these same problems It suffers from
00:07:42
using indirect
00:07:44
standardization uh uh in the sense that
00:07:46
it's kind of taking what the hospital
00:07:48
the patients the hospital saw and making
00:07:50
an extrapolation about how they would
00:07:52
have done at another hospital um and so
00:07:55
uh I I think that that uh if that could
00:07:57
be changed that would be uh uh a great
00:08:01
benefit to u to the application of
00:08:04
looking at
00:08:08
quality often hospitals have access to
00:08:12
National data sets or comparative data
00:08:14
sets so hospitals could in a sense form
00:08:17
their own templates and then see how
00:08:19
their patients would do at other
00:08:21
hospitals using the same technique but
00:08:23
they'd have to understand the details of
00:08:26
multivariant matching uh we bring to the
00:08:28
table a way to do this in in a very I
00:08:32
think sophisticated way uh through the
00:08:34
work uh of statisticians at at at at
00:08:37
Warton um in particular Paul rosenbom um
00:08:41
but uh they if they could do that
00:08:43
matching then they could better compare
00:08:46
their results to other hospitals and
00:08:48
know whether they're really doing a good
00:08:50
job or
00:08:54
not uh I think a second area that uh we
00:08:57
actually just talked about was this idea
00:09:00
of making your own Boutique template so
00:09:02
a chief medical officer could for
00:09:04
example say uh let's construct uh a
00:09:08
template on these difficult patients
00:09:11
that we see and let's see how other
00:09:13
hospitals would have handled these same
00:09:15
difficult patients I think that Boutique
00:09:17
template is a way that we're is the next
00:09:20
step and that's where we're moving and
00:09:22
then there's other applications of
00:09:24
multivariant matching to looking at
00:09:25
quality suppose you have a very small
00:09:28
Hospital uh you don't match the template
00:09:30
very often because you're so small you
00:09:32
don't have the patience to be able to
00:09:35
match to this overall National template
00:09:37
well we can turn things around and we
00:09:39
can say uh okay let's look at the small
00:09:42
hospitals
00:09:43
patients and then match them to the
00:09:46
whole country so use the whole country
00:09:47
as a kind of a frame of reference uh and
00:09:51
then we have a very very good idea
00:09:53
looking at the outcomes of the small
00:09:55
hospitals patients and their matched
00:09:57
sample how the how the small hospital
00:09:59
did relative to the typical patient in
00:10:02
the country so that's another
00:10:03
application to multivariant uh with
00:10:05
multivariant matching to help improve
00:10:28
quality
