Event ID: 2570738
Event Started: 3/27/2015 8:22:18 AM ET
Please stand by for realtime captions.

Good morning everybody my name is Richard Kronick director of AHRQ. I'm excited for the day, delighted to have you all here and looking forward to your input on a Friday of questions that were dealing with at the agency. I will very soon welcome a number of new members. First, let me welcome and think Beth for agreeing to be the new chair of our national advisory Council. Beth I know many of you know is the director of the Kaiser Permanente Ctr. for effectiveness and safety research.

[ Applause ]

Beth is responsible for strategic direction and scientific oversight of the center. Cut Beth has been a Kaiser four, four years, at RND running most of our AMD health. Internationally known, expert in health services research, I was reading Beth's work when I was a baby.

[ Laughter ]

I do not know how Beth wrote this when she was 12. Member of the Institute of medicine, vice chair of the American Board of internal medicine, order trustees, but on the Academy health board as chair, the Academy health board, seminal work in figuring out about quality of care and many other areas. I am delighted us for agreeing to chair our board.

I am delighted to be here as well. Along with my entering class, I want to welcome [ Indiscernible ], Kevin, Mary, we traveled together lots of circles and Sandy Schwartz. Then Jen is also a new member and she is sadly in Hawaii at this time period gave her quite a hard time for not being with us who don't have time for everybody to go rounds roping at lunch of repeat those people can get to know each other and do all of that. We're going to have [ Indiscernible ] will be here representing Patrick Conway at some point so gives without now I should officially call the meeting to order. I'm following my instructions. The other was all out of order. I want to welcome the next members, participants and visitors of those viewing the web cast. The bios are in your folder if you need to look up who it is around the table. There is a few housekeeping notes, many of you may have gotten these as you came in, if you need transportation after the meeting place on up at the registration desk by the end of lunch. We're all going to have our photo taken in the wellness room. I did not know that photos were -- there doing that here. At the beginning of lunch, make sure that you find your way there. The new members will have an individual photo taken after the group photo. If you ordered lunch, you can pay for and pick up the lunch in the registration area.

I will just know for the folks who are attending from the public if you would like to make a public comment in there are two opportunities coming 11:30 AM and 2:00 PM, make sure you set up at the registration desk. Lets go quickly around the room have people tell us name and your affiliation.

Beth McGlynn, Kaiser Permanente. Jimmy Zimmerman, head of [ Indiscernible ]. Sharon Arnold. David Ballard. Andrea gals are -- Sandy Schwartz. Jane [ Indiscernible ]. Victor [ Indiscernible ] Kevin [ Indiscernible ]. And [ Indiscernible ]. Jed. Paul Ginsburg. Mary [ Indiscernible ]. David Atkins.. Leon [ Indiscernible ]. Mary [ Indiscernible ]. [ Indiscernible ]. Patty. Paul Sherman. There were two members of may be on the phone. Sherry Davidson. Carol.

The first-order business is to review the minutes from the November 7 meeting, the copy is in your folder. See if there are any changes or edits to those minutes. Seeing none, I'm going to send you have all read them carefully. I would like to entertain a motion to approve the minutes. All in favor? Any opposed? Now, Dr. Kronick I'm turning it over to you .

I'm going to spend a little bit of time on general updates, some exciting activities of the agency since we have last been together and then most of this two-hour session on getting your advice it input on a few areas and projects that we are in various stages of working on. Beth welcome the new members who are listed on the slide and I welcome you again and I'm thrilled about your willingness to devote time and energy to the agency, many thanks. On transitions, Greg Baker has a new job as the vice president of pharmacy of premise health and Leon, every time it seems like you getting new jobs.

[ Laughter ]

May congratulate you and wish you luck getting the income to student -- to see the new position. On staff transitions, Eileen Frazier has been the director center for delivery organizations and markets at a RC -- ARC for a while and is retiring. Irene has done an incredible job in building the healthcare cost and utilization Project which I think you are all probably familiar with we have 90% -- 97% of the hospital discharges in the country through voluntary arrangements with 47 states, soon to be 48 jurisdictions, has along with that worked with colleagues to build the quality indicators, ambulatory sensitive conditions used by many folks to understand what is going on in the ambulatory arena inpatient quality indicators used by hospitals all around the country to try to understand quality of care and working on improving it. Software called [ Indiscernible ] network powered by ARC the 13 states are using to create public reporting websites as well as building and managing a group of people who do research on delivery organizations and markets both intramural and extramural grant portfolio. Irene will be sorely missed. She is among other things a new grandmother to be congratulated for many things. And very young. Are recruiting for a replacement, have had a number of alumni interviews and will be doing broader interviews very soon. Not really a replacement but for another director for the Center. I think Irene for the amazing work for the agency.

I think as we have talked about in previous meetings, we're also recruit for the director for center of evidence and practice improvement which is the center that was formed about nine months ago and I had hoped to be able to announce the new center director today. We are probably a couple of days away from that. We have one identified and I think we're very close to public announcement for a very strong candidate. I'm very excited about that, David Myers has been the acting director for the center, worst two hats also as a chief medical officer. He is extremely excited. The Stephanie center also looking forward to permanent -- the staff and the center also look into permanent leadership.

You probably cannot the these numbers, but a slot on our budget, I think since we last met, the 2015 budget was enacted and we did well in the 2015 budget, basically level with 2014 are pretty close to level of 2014 which was more than was requested in the president's budget. And appropriate funds and there was, only Washington insiders would care about this but we were removed from funding on the evaluation tax which is a 2.5% assessment, ledges of various health agencies to budget authority. In some ways this is an Arcadian, why does it matter but people who know budgeting think the budget authority is a more secure source of funding and there has been concerns raised over many years about the evaluation: there's some concern a may go away what would happen so having budget authority is an advantage.

The 2016 president budget that was released in February proposes for the first time in many years an increase in funding for the agency. The president's budget in the last number of years has been for proposing a decrease from the inactive levels and the 16 budget proposes an increase and I will talk about some of the areas of change their. I am assuming you are not a shy group, anyone will ask questions at any point.

Is probably more appropriate will refer to areas of funding, there will be a slight on areas of funding?

Start going to go into details suggest a question. Some accuracy question. As I have been involved in performance measurement over the years and as we're moving to a value-based payment schedule for healthcare, the quality, I know we've had this discussion before by just want to say it again, the quality metrics that we have today, there are huge gaps in certain areas. If any of the money that ARC is putting into fund , has received or there is a way that ARC can make a case to get more money to fund research to identify those gaps and make those gaps smaller would be very helpful and really progressive I believe for our healthcare system.

It is a great comments. That I am going to defer because it actually the first I'm going to ask for input on is what we should be doing around the bring out how to pay for value and that I think will segue, be naturally a part of that.

In the budget for 16, there are three proposals for either new or expanded initiatives. $12 million proposal for a new initiative to improve care for people with multiple chronic conditions. This proposal, it is an issue that has two parts to it. The first part, $9 million of it is to figure out how to get, how to elicit information from patients with multiple chronic conditions about what they want from their care. What their preferences are and what matters to them most and how to communicate that information to physicians in ways that are useful. Patients with multiple chronic conditions, all kinds of problems and part of what is needed for physicians to understand, what is most important, what are the goals of the patient. Is a more important for them to be able to eat dinner with her family, to be able to do the crossword puzzle in the morning and one of the main problems in providing good care here is getting that information in a way that works, communicating it to the clinicians, the physician and the other members of the carotene of being able to act on it. The second part of this initiative is to gather better information as part of the medical panel surge in the configuration of care for people with multiple chronic conditions. We would expand the work we do in the medical provider survey where we go to providers, we survey to back sample of households amount survey many of their providers to get information currently that provider survey gets information on what care was delivered and how much was paid for. We don't gather other information from providers although Steve will talk about an expansion that we are currently working on. This proposal would gather information from the providers of patients with multiple chronic conditions about the configuration of care, whether it be care teams, they doing care plans, what does the care look like. Gather some additional information from patients and be able to make progress on the question that we don't have very much evidence on. What is the relationship between healthcare is organized and deliver for people's multiple chronic to sit -- conditions and outcomes look like for people with MCC.

Sandy?

I think that is great. It has always driven crazy that we look at these conditions and look at -- that is the overwhelming majority of the patients that are sick and cared for. We all know the date on small number of people for large number of [ Indiscernible ] and morbidity in this stuff so I am glad to see that you are doing that. Think that is really important. Thank you.

We are excited we got a lot of support within the Department and Boulevard discussions with folks on the hill seem to be quite a lot of interest in this as well.

I just hope not to make is too simplistic for the first part, when speaking to patients that we teach physicians to have a two way conversation with them and ask them, and start using, learning how to use your decision-making. I think we will get the answers that they need to.

Certainly the focus here is on improving communication, being able to elicit and share information about what patients want. Second initiative here is an expansion of existing work in combating antibiotic resistant bacteria. I think the president yesterday talked about this initiative again. This is a much broader HHS wide initiative in which our part is focused on developing evidence about stewardship oh grams. And how to make these work. I think these are sorely needed and a difficult area. The third is also part of a much broader HHS wide initiative on [ Indiscernible ] drug overdose prevention in our work would be focused on Medicaid assistant trading -- programs. Part of this to be doing and evidence, earning-based practice Center, doing a systematic review to figure out what is known about medication assistance treatment programs but the larger part of this would then be developing evidence for a demonstration program about how these programs can be effectively implemented particularly in rural areas where many providers don't have the support and expertise to implement these programs.

The last initiative I want to mention is actually not in the park -- ARC budget but is in the public health emergency services fund. It is a fund that is part of the office of the secretary. There is a proposal to spend $30 million to replicate the RAID -- RAND health insurance. Actually replicate is not the correct word.

[ Indiscernible - low volume ]

I came in on the wind down. Paul --

Paul were you involved at all?

That was before I got here.

There is a recognition that the main information we have about the effects of cost-sharing on utilization and services comes from the RAND HAE conducted more than 40 years ago. A lot has changed in managed care, network, value-based insurance design and recognition that it would be useful to have updated them better information about the effects of insurance design on utilization and on health outcomes for patients the RAND HAE can cost about [ Indiscernible ] 40 years ago so probably $30 million want exactly replicated. In the other hand were a different world right now with a lot of that cost was paying for the cost of insurance. In the post affordable care act will repent for the cost of insurance for lots of people committing tremendous pride -- progress could be made much less expenditure and the budget does not put the $30 million into ARC budget but says ARC in intimate be working together on bitterness and we're currently long way from having in the president's budget to getting it enacted. Very exciting proposal.

I think that is great. What the biggest hot topics I think today is talking about risk adjustment and if this helps give any evidence basis to how we do credible risk adjustment, I think it is wonderful.

All?

Is a very interesting prospect. Hopefully [ Indiscernible ] are going to do some serious thinking because the complexities in our system make it that much harder to do a meaningful study and the essence of the RAND experiment was it was a randomized controlled trial and obviously trying to get close to but you're not going to do that with $30 million.

We look forward to, we're doing some serious thinking and look forward to any advice and thoughts on this.

What is the timeframe?

Is a proposal for the FY 16 budget. The FY 16 budgets make it enacted before the beginning of FY 16 which would be --

To they say who is behind it? What is the motivation.

There is lots of interest in it, yes.

It comes from the administration so it is not congressional.

Wanted to give you a picture of grant activity at the agency. There is lots of focus often on the investigator initiated portion of our grant activity so the blue slice here, $46 million in FY 14 and a similar number planned for FY 15. These are purely investigator initiated. We have a funding opportunity now that is quite open-ended but covers really the waterfront of health services research. The other slices here have quite a lot of investigator initiation in them that they come from more focused announcements so $33 million in grants and patient safety area, $24 million in health IT and $65 million estimated, that is in FY 15 estimate for the peak or trust fund will be talking about a couple of stores initiatives in the next couple of slides. We have come the point of this slide is substantial activity in the grant area with a quart of it purely investigator initiated, the others some greater amount of focus from us but still grants investigators are telling us what they think make sense to do.

Brief updates and I have covered the next two slides before but I wanted to remind you because they are an important part of what we are moving forward with. We will very soon be announcing awards from a solicitation called -- center outcome research and small and medium-sized practices. For those of you who are bent on the [ Indiscernible ] David Myers discussed this at the last meeting, will be awarding eight grants to grantees who will work with regional collaboratives to try to figure out what kinds of supports small and medium-size actresses dated better incorporate PCOR evidence into their practice with a particular focus on improving performance and improving the outcomes for cardiovascular risk factors, aspirin, blood pressure, smoking in concert with the million hearts campaign that the department is involved in this is a substantial dissemination effort in and as of itself. Were targeting about 6000 physician offices, 9 million people. For us the large-scale dissemination but also crucially important on the evidence development part because even 6000 positions, 2000 primary physicians nationwide of the purpose here is to develop evidence about how to make this work so that we can work with our colleagues at CMS, private payers, health systems to say, here is evidence about how to make progress here. This it will be three-year grants and we are quite excited about it as in all research, it is always, what happens when you get out there? The book into this is, a project were will be awarding grants to three centers of excellence in studying comparative health system performance and disseminated PCOR. The first set is around small, medium-sized practices. These grants are focused on trying to understand the behavior of health systems in disseminating PCOR and how that relates more generally to health system performance and back to Andrea's earlier comment, some of this will be both around defining what health systems are, I think there may be 700 health systems in the United States, can I give you a list of them? No. I tell you they are performing? Not really. The centers will be making progress there as part of understanding what these systems are doing to disseminate PCOR and how that relates more broadly to their performance. Very excited about those.

We're doing some internal work, on a much smaller scale. The centers of excellence that the grants will be up to $2.5 million each year for each center for five years. Substantial investment for the agency. Internal work is being done with a couple of people but we will be doing work with Medicaid claims data and a private database on health systems and trying to figure out what the system performance looks like at least on the Medicare site.

I'm going to shift to work in safety space and I know at our last meeting Jeffrey directs the Center for quality improvement patient safety spend quite a lot of time with you describing the work that we do in producing evidence to try to make healthier, safer, since our last meeting we released a report in early December showing that hospital care was 17% safer in 2013 than it was in 2010. In 2010 we estimated that there were 140 In 2010 we estimated that there were 145 adverse events per at thousand hospitalizations or the estimate in 2013 being 121 adverse events per thousand. Still way too many, 100 and Still way too many, 121 -- is a lot but it was 1.3 million fewer adverse events over the three years compared to what would have been the case at the 2010 level have remained unchanged. Very remarkable improvement or reduction in adverse events. An estimated 50,000 fewer deaths as a result of the 1.3 million fewer events. If you say the 1.3 million estimate I think a very, pretty precise estimates. It comes from, there are three different data sources but almost all of the data, 92% of the data comes from a review of medical records and CMS as part of the IQ are process samples a set of medical records every year, the number of errors between about 18 and 30,000 but pretty decent sized sample of medical records. We as a structured call as part of something called the Medicare patient safety monitoring system in which abstractors review these medical records to see whether one of 21 different adverse events occurred. Adverse events are hospital required infections of various sorts, pressure ulcers, adverse drug events, falls in the protocols are structured protocols with very high interbedded reliability. The protocol hasn't changed over the three years and this showed the reduction from 145 two 121 adverse events per thousand hospitalizations. Estimate of 50,000 fewer deaths is a little less certain. That comes from work that was done actually in 2010 when the partnership for patients was being created. The measuring system was put in place as part of the implementation partnership for patients and a variety of folks here and in other parts of the government reviewed the literature for each of these 21 adverse events to try to say, what is the best evidence about the relationship between the adverse event immortality. Are some of the events the evidence is pretty good in sports some of the events you can imagine the evidence is not what we would want. We did the best we could for each of the events to estimate excess mortality and then the 50,000 comes from taking the reduction for each of these 21 things are multiplying it by the estimated excess mortality. I think as you think about it overall, 1.3 million fewer adverse events which I said is a pretty solid number, were estimating 50,000 fewer deaths which is about 3% so we're saying on average each of these adverse events had excess mortality of 3% which forever I did the hospital required infections it is way low, like to see [ Indiscernible ] infections much higher, for pressure ulcers, which is a fairly big part of the lists, it is pretty good evidence of excess mortality from serious pressure ulcers. I think it is a quite reasonable estimate in a pretty remarkable result. Animist of a lot of not good news that we often get, quite good news, we take credit at the agency for the measurement system certainly. We would not know this if it were not for the work that was done here and for you to begin to the improvement in safety, the work that we have done through the [ Indiscernible ] project over the years and implementing the evidence about how to reduce central line infections and catheter associated urinary tract infections and work we have done another sorts of adverse events.

Certainly we do not know exactly why this improvement has happened. We can only say, we saw that it happened but almost certainly related to the tremendous work at CMS and the partnership for patients, working with the hospital engagement networks across the country in the changes in payment that CMS is going to be very interested in your thoughts on this. As I talk with various people it feels like what I hear is, that the payment changes got the attention of hospital CEOs and CFOs and have changed the approach to looking at safety.

I would say moved it up on the priority list, that was something they needed to pay attention to.

I think there's no question that the HAC initiative has stimulated hospital-based healthcare systems to benefit safety but I think the reduction as measured by the claims space [ Indiscernible ] measure very significant part of that reduction relates to coding issues. I think all of us who work in the hospital care arena across the United States are aware of significant coding issues in our organizations. Animist of addressing some of those issues we also recognize that there is some real care improvement opportunity out there.

That is a good point David. The resin admission is an indicator whether a problem was present on admission it was good had not been so well coded prior to the CMS HAC payment initiatives and that there has been pretty clear evidence of improvement in the sieve president on admission. These that I want to emphasize do not come from claims data. These are from medical review and we did pay attention to whether even in medical records there might be more careful coding of present on admission and it didn't look like it. We, what we found was, I'm trying to remember back, there was, in the medical records we didn't see any increase in POA in the record itself. I don't think this is related to that although it is certainly a good point. Sandy?

Is ARC doing anything in regards to come I'm thinking to this in mission twitchy just about dissemination. It is my sense which means I have no empirical data for what I'm about to say but it is my sense that there is a real so far missed opportunity to figure out the best ways to implement these things in systems. More of a systems management approach, what are the best practices, what are the problems that people run into, things like that. I do not know where that falls between ARC and various other agencies but I think that area might be especially given what you have been going to might be right and it might also fit in with the [ Indiscernible ] dissemination.

Is very central to our work and the work that I mention that in C USP the central unit-based safety project which we have been doing for I think 10 years or more is exactly trying to take, tried to figure out how to effectively implement what is known to work. We be working on catheter associated urinary tract infections made a lot of progress outside the ICU, not so much in the ICU. That project is now working on find to say, how to resolve a problem in the ICUs. We are working, there was a meeting here, a very exciting meeting that Patty was that on Wednesday I think on an early [ Indiscernible - low volume ] resolution programs and hospitals.

What I am wondering about, you guys are [ Indiscernible ] in the mind, you get these things. As you find a finding that is worth doing, is there a standardized approach or mechanism to say, to bring it to the attention of the various program leaders and say, this may be ready for our next grant or directing people or whatever.

In the CD area we have been doing that a lot. In the more general dissemination of PCOR, outcomes research we been doing that some and shared will lead a session right after lunch to focus on exactly that question.

I wanted to comment, where one of the largest healthcare systems with about 100 hospitals in the partnership for patients. We have about a decade of data in this area. We experienced the same drop that you are singing the national data and I think there is a real opportunity to perhaps look at a study around hospitals who have the data that can use themselves as a case-control study of what actually changed in the implementation. We have been tracking all of these measures long before they were attached to penalties. We have not seen the kind of drop that we saw as we partnered with the national effort and we did a lot of time stamping. We also have observed over expected mortality but very similar drop. There is something there. Definitely a correlation [ Indiscernible ] is a different question. A wonderful opportunity.

I appreciate hearing Sandy's comment on this because I think we are at the stage, we know something pretty amazing has happened in we have got some hypotheses about why but don't really know why and how when we are still at 121 --

Is a Springfield phenomenon. What it is isn't exactly sure.

[ Laughter ]

If we were done it wouldn't matter but we are still at 121 adverse events per thousand. We still have a long way to go. It would you good to figure that out. I'm going to move on because want to get to the point, I want to put all of this but we issued a funding opportunity now, recently to fund up to $5 million in new grants for patient safety learning labs. This falls on an app .14 initiative in which we funded five patient funding labs and we will be funding additional work in this area. About going to read to the bullets are but multidisciplinary teams, exciting ways to try to make further progress.

How do efforts like this meld with organizations like joint commission and the private sector that has its own set of tools and data gathering and analysis on how you implement improved patient save the?

That is a really good question. We participate Jeffcoat the director for quality improvement safety, our representative at the joint commission, mostly in sharing of information. We are trying to produce evidence about how to make progress in in the case of central line infections were actually working in 1100 ICUs are trying to implement that progress then we share that evidence, the joint commission is aware of it so is the partnership for patients, we work very closely and they worked with the hospital engagement networks on trying to get that evidence out there. I'm not aware that the joint commission has used this evidence, I don't know if you have anything to add to change what they do in accreditation.

I would say as Rick pointed out there, first will good morning everybody, there is quite a bit of overlap and I think actually synergy in terms of what we are doing and what groups of the joint commission are doing, as Rick noted, only to their primary mission is a accreditation. I think most know about their heavy investment, high reliability as a foundational basis for a lot of their current work. I think probably a nice summary statement from our it is there are lots of different labels for things that may not be exactly the same thing but fundamentally they are the same approaches, certainly different aspects of those, human factors. Also know from interacting with many different systems, hospitals and systems throughout the country, different types of philosophies are more or less appealing to them but again there is a common core in terms of organizational competency as a way that I like to think about it. It is a bit of -- there's quite a bit of variability but think also quite a bit of commonality in terms of what we are doing in the approaches that ultimately we package up and consistent with our mission try to help others understand the evidence. Mashup that exactly answers your question that we come at it from more of a research and operational perspective, certainly again accreditation is memory does fair amount of common foundational elements and what we're both trying to achieve.

I'm going to try to move quickly to the next few slides to get to work really want and need, number of areas were really need your input, I need your input on all of this but open things were struggling with. We are working on [ Indiscernible ] PCOR findings between clinical support I think as many of you know, agent guidelines may come out and then if folks and health systems want to actually implement these guidelines, takes a tremendous amount of work to get from the guideline to what do you do with your THR to get the guideline implemented and will be working with a variety of parties to try to smooth that process and make it less painful antiserum. We recently issued a brief on the use of simulation in improving safety in hospitals response to Ebola and similar kinds of threats. Agency has not been involved in the acute phase of the response to the Ebola crisis but a lot of the tools and evidence that were developed but the use of simulation I think it can be useful and our colleagues at CDC and the assistant secretary for preparedness and response of race -- responded quite well to this project came up fairly recently we will see whether it is actually useful.

With respect to the clinical decision support I'm surprised& And ring with CMS rather than ONC kinds of partnerships which we heard from [ Indiscernible ] last year.

Were very heavily partnered with ONC on this. CMS is in part, there is a [ Indiscernible ] partnership and also contractual partnership that we are using [ Indiscernible - low volume ] called the FFR DC which is a very flexible contract that they have. To be making progress on this. Were heavily involved with ONC on this.

I know you want to get somewhere else but one blind spot we talk about clinical positions part that perhaps could help overcome which is now also in the new rules for meaningful use that just came out for the next stage, the original proposal for the meaningful use spoke about using EMR's to try to capture patient preferences and engagement decision-making that language no longer there. When people refer to critical physicians about their often focusing on supporting the physician clinician decision-making rather than the decision-making that could be happening with patient participation. Perhaps in these artifacts that you are going to implement and testing, consider the possibility of adding language or supporting an idea that maybe there is a priority on the support that tries to engage both the patient and the clinician. These will connect very nicely with your initiative for chronic disease as well.

I wanted to mention the administration wide delivery system reform efforts that Secretary. Burwell announced three weeks or so ago. Maybe a month ago the commitment to delivery system reform and announced goals on payments for Medicare payments of 30% Medicare payments would be in the alternative payment systems by 2016, 50% by 2018, most of the rest going to quality, the FC are billed at the house passed yesterday quite strong support in this direction. The general goals of better care, smarter spending, healthier people, the reformulation of the triple aim with three main streams of focus, one in improving centers the way the providers are paid in a lot of the focus has been there but also on improving and innovating and care delivery and sharing information. Much of the work at the agency is part of this effort. It fits into this effort. Certainly the accelerating implementation of PCOR and small and medium-sized practices centrally part of the improvement and innovation in care delivery as is all of our work in producing evidence to improve the safety of healthcare. We work around providing better information to consumers and clinicians at the point of care, clinical decision support work that I discussed in lots of other work at the agency. Wednesday Thursday kickoff of the learning in action network. Was anybody here a part of that?'s there were I think 3000 people on the webinar. It is an important initiative for HHS more broadly with lots of activity at the agency and support. Very briefly, the US preventive services task force continues to be extremely active. This slide is in your pocket and I'm not going to go through it. Active in terms of recommendations that have either been finalized or are in draft, upcoming. Upcoming also not listed here, sometime this spring expected and up date at the 2009 recommendation on us [ Indiscernible ] for breast cancer and that has been in process for a wild. There has been a draft research plan that received public comment and finalized of the research plan and a draft recommendation is expected this spring and we expect lots of public comment.

The evidence based practice centers have also been extremely active in I'm not going to go through this. I wanted to give Steve Cohen who directs the Center for access and cost of trends a couple of minutes to describe, we received a grant from the Johnson foundation, usually we provide grants for us, this was, we sometimes have received grants in the past but not in the year and a half that I have been here. To expand the provider part of [ Indiscernible - low volume ] and steeple to little bit about this.

Thank you Rick. It is exciting to type at the new arm of the medical panel survey particularly the national resource and heirs of cost coverage access and affordability. As Rick alluded to actually have a medical provider survey but it has been a field goal only to get expenditure and resources of payment information with some diagnosis information as well. With all these dynamic changes in healthcare systems, not having information in terms of practice characteristics and some of the organizational structures that the practices can rely on really seems to be a major Or the survey. [ Indiscernible ] Johnson has provided funding. It is great to be API. I'm usually completely on the other side but this is a novel and allowing us to go for the implementation of this medical organization survey. In essence we will get characteristics that include organization size, practice size, whether or not there is connectivity to accountable care organization, whether or not the practice is certified as a patient centered medical home. Issues in terms of the penetration of HRT. Other aspects that potentially align with what is going on as a consequence of the affordable care act and other financial arrangements. Just to give you a flavor of how things are going right now, we are going to select from our 2015 sample, the household sample which is roughly 15,000 households, 35,000 individuals, a representative national sample of individuals who had ambulatory care with their usual source of care. We will select approximately 6000 as usual source of care providers the will be a subsample of our normal medical provider survey so we have design efficiencies in the process. Through the initial contact we will decide on who is the best respondent to provide information on organizational structure and characteristics. We will either conduct right then and there the phone interview or we will allow for other venues for a data collection which could be by mail, could be by phone, could be potentially webinar. Once we get the data back, let me say and pause, this is not a standalone national survey of medical organizations. Were not going to make separate estimates at the organizational level. This is an interface to move back to the household survey to individuals so we look at other dynamic characteristics in terms of on the physician side, how intervenes with use, access, affordability. In terms of some of the efforts, this questionnaire --

Ask a question? It sounds at this is really an augmentation of your household survey rather than a survey of organizations because it sounds like most of the organizations you will survey will be primary care practices.

For this go-round that is writes. The budget that we have was going in that direction I wanted to put on the table was that, this comes from an effort that actually started in our sister center a delivery organization and markets to develop a questionnaire for precisely the alternative process of a standalone effort. In fact we have an IA with NCHS, Charlie is not here right now but they are actually in the field testing that survey instrument to see how well we have good responses, the item nonresponse and that is informing the questionnaire design. The intention for this [ Indiscernible ] pieces to make it a link component. It will not standalone. That said, my background is biostatistics. There are ways of [ Indiscernible ] estimation if you had certain questions in terms of the multiple contacts people have you can theoretically try to transform it into separate estimates. That was going well beyond the scope of what we intended I have no doubt my staff will look at some of those capabilities. Does I give you some court case on that front.

We're having internal meetings in terms of finding in the questionnaire. It just went out yesterday to a group of external experts, colleagues at NIH, universities and will be getting information back on that. We have had major briefings in the department and I braved the data Council and OMB in terms of our plans to make sure our lead on everybody's on the same page in terms of our intentions and again trying to get as much efficiency in terms of the design coordinating with our current medical provider survey. We're very excited about that and with fingers crossed, a similar photo call as Rick alluded to four 2016 will go forward it will be restricted again budget limitations to individuals as multiple chronic conditions. The targeted sample of organizations would not be restricted to usual source of care. We will go to physician specialties and things that would be most appropriate to inform that analytical effort.

The last update slide is to announce that the ARC research Council overturned used to be called an annual research conference but things to [ Indiscernible ] doing conferences has had a hiatus but things to the work of Jamie Zimmerman, the dish in the typical work of it Jamie Zimmerman we will be holding that conference again in October and are very excited about that as a chance to bring together grantee showcase the work of the agency. Will be doing at Thule -- will be doing actually the 2 1/2 day conference will be sharing Tuesday afternoon with [ Indiscernible ] and they'll be doing their first annual conference following on hours. -- Hours.

I especially need your help here. I have talked a little bit about the paying for value project at a previous meeting and as a reminder, and apologies to those of my colleagues who have heard me say this 10 times, this started as the apple pickers and federal judges project in the question here is, when is it appropriate to use strong financial incentives to try to get people to do what you want them to do and when is it not so appropriate to use strong financial incentive? Apple pickers, ultimate strong incentives they get to taste every public gets picked, federal judges, lifetime 10 years, salary totally divorced from any measure of performance and why is that? It is because we can measure very well what we want, apple pickers to do. The cost of measurement is low, we don't worry much about distortion or gaming and federal judges the opposite people want them to produce high-quality justices efficiently but we're concerned will not be able to measure that very well. We try to measure it well, it will distort their behavior in ways that we will be unhappy with practice project started with trying to understand what parts of medical care -- make sense have strong financial incentives and what part to not. It is a bit over time, we commissioned a set of papers, the titles are listed here and that is actually a co-author on one of them. With September useful advice including the importance of as we are trying to develop a research agenda to answer these questions, to pay attention to the difference between trying to pay larger organizations and trying to pay individual physicians or smaller groups of physicians, the importance of paying attention to the context and underlying payment system. Paying for quality in the context of a capitated payment system or shared savings, we've got very different needs and issues than paying for quality in the context of the fee for service payment system.

As I said, with a set of papers and commentaries that will be appearing in a special issue of health services research scheduled for the fall and we are still trying to refine the research agenda that make sense I will say couple of more things and then open it out. Part of what has become apparent and I think Andrea alluded to this in her comments and in conversations with [ Indiscernible ] folks is the relatively small part of what we want medical care to do that is subject to measurement. I asked you to think of, rent to the following thought experiment. If we imagined a very high-performing health system in the United States and a furlough performing health system, what is the difference in the number of quality adjusted life years that those two systems create? I do not know the answer but I would hypothesize for now or just positive for now. Let's say that it is to quality which, I do not hope that seems high or low but if you had a really good health system and not a good health system it might be that the really good one is producing two more quality adjusted life years than the not good one. Kevin you are frowning.

That is a lot. Given that health care contributes a little.

You think less? May be one quality? Are to see. Art of the problem is we do not know prickly sadist to.

[ Indiscernible - multiple speakers ]

If you are comparing to health systems versus the disorganized care that most Americans still receive, let's say a really high quality performing health system compared to the disorganized care.'s Mecca think that is a better way of thinking about it. May be a general know if it gets one quality may be does not even that. Lets it go for two for now. Then ask, suppose that those systems, take the two systems, one of which performs really well on everything we measure in the other, everything we measure that is part of either a value-based purchasing pay for performance or public reporting together performs really poorly on everything we measure but their average on everything we don't measure. How much of that two qualities which Kevin says is less than two to begin with probably, how much of that to quality difference will be there for the really high and low performance systems on the stuff we measure X I can I do not know the answer but I think it is probably a pretty small part if we started with two, the parts that we are measuring are probably pretty small. That would suggest that you wouldn't want 100% of payment which is not what any of these systems do. You would not want 100% of payments based on the step that we are measuring.

On the other hand, part of the story, I spent quite a lot of time in the up date part on the 17% increase in safety with the reductions of readmissions, we've seen improvements on many things that matter to patients in areas that we measure and it does seem that at least part of, one of the questions if we could figure out a way to answer might be valuable to answer but I look to your comments, is to understand what it is about the, or how to construct an external measurement and either pay for performance or reporting system that will catalyze and encourage internal quality improvement efforts or to what extent can you have the external system be used to encourage healthcare organizations and providers to work on improvement in really productive ways as opposed to external stuff that becomes, if you check the box exercise and it gets divorced and it is an impediment even to resources going into internal improvements.

One area that we are thinking of trying to invest in is trying to understand better what organizations are doing in internal quality improvement, what they are measuring, how those measures relate to the external parts. I see a lot of tense around so I will be quiet.

-- Tense around so I will be quiet.

Kevin?

I think this is such a profound are you are getting into with this because I am really looking forward to reading the HSR supplement because I have a lot of apprehensions about thinking it is all going to be about very targeted incentives for specific things and I think, if you look at, you said control line infections. It is a little bit [ Indiscernible ] but I think the most profound thing is just calling that out. No nurse and physician feels good about killing a patient, right? Once a culture change in that says institutions, I think that is unacceptable. I think in most hospitals, that is a shameful thing to have a patient die of a preventable central line infection work or have a patient fall out of bed and break their hip. I don't think it is motivated may be in a vaguely at the hospital, certainly not at the individual clinician or caregiver level. I think one has to think of incentives in a very general direction away, not trying to go piece by piece of because it is so cute things you can measure I don't think that is what motivates the people. I think if AHRQ is going to go into this area, but that's where the articles in the issue it will say be careful about incentives, it is a whole Daniel pink stereo would like to see that philosophy balance with the pure incentivize everything and that is all that motivates people. I think the global the four service versus global payment has a huge impact on the orientation of the system. I think a few things [ Indiscernible ] but it was earlier thing about comorbidities and your first point, what do patients really want for their overall well-being and functional status and goals. I think you've got to be just that will be great for HRQ to say where is it appropriate, were to get leverage and where do you have to be careful of not over incentivizing.

There's a lot of attention in a number of the papers in one of the commentaries from [ Indiscernible ] two, how do you support and catalyze the intrinsic incentive to do the right thing. What we do not have in the papers really and I am still not all that clear about is, what sort of research if any would be useful in answering that question? We're going to have and use these external measurement systems and I think arguably, some of them have actually catalyze the kind of internal improvement. The safety stuff is a lot easier I think. Nobody wants to do harm. If you can say, this thing you are doing is doing harm, that seems easier then how do you catalyze and do it better and incensed that.

Andy?

I think the payment thing is fundamentally important because we notice going to happen anyhow. We may as well make sure it is done well and I think that is crucial, some people think we know how to do it and it is just a matter of tweaking some financial incentives and everything will get better. What I wanted to bring up, I think it is important really for ARC to pay attention, hate to quote the guy but when [ Indiscernible ] talked about the known unknowns and the unknown unknowns, to not neglect even though it is an area, I am hesitant here because Lincoln once said better be quiet and thought a fool than to open your mouth [ Indiscernible ] a lot of people around here who know a lot more about quality and have spent their careers are working on polity measurement. I don't think we really know how to measure quality. We measure elements of it, sort of like looking for the keys under the light instead of where we drop them for getting back to what Victor was saying before this is really an important part to find out what is important to patients. I think I may be using this as a lever to continue to work with other area or to emphasize that will be important for the thing on how to integrate those to come out to use work that is done on payment as a vehicle incensed, almost a backdoor way to get more bunting or fund more work on how to we really measure and improve quality. The second thing is with regard to, that was triggered was, the appropriateness. Of the really want to minimize central line infections the most import line is only put them in that people need them. We're spending all of our time on trying to minimize the infection rate which is great and the people who about them but we all know that a large portion of the people who get whatever we are talking about don't need it and we know that because every time we stop doing it to people they seem to do just as well or better. I not sure how that fits into the research agenda but I think, still continuing to fund are thinking about again how to maybe incorporate because you know there's going to be a lot of support for the paste up both in Capitol Hill and throughout the system because it is so fundamental. Think about that is getting a way to get support and funding for some of the things that are more problematic given the orbit around the table knows it is important to.

Victor?

On the most current person cited by [ Indiscernible ] today. After Lincoln, Rumsfeld and Springfield.

Victor asked me for a citation it is Stevensville 1966. It was not published in a peer-reviewed journal so I do not know if it is worth anything.

A couple of comments. One has to do with your example jet that was very helpful in the matched in which you are thinking but for instance by focusing on the impact on quality, one of the things that focuses our attention away from the day-to-day expressive the patient. When you get out of the hospital and look at patients minutes over time to clear those in multiple chronic conditions, the healthcare system has a way of establishing a particular size of a print on their lives. With the potential quality targets that suggest a highlight [ Indiscernible ] from the patient is to what extent is still system able to pursue my goals without imposing a burden on my life that we have capacity subside to pursue my lights, hopes and dreams [ Indiscernible ] a patient. That is I have not gotten any thing" where this comes lives is to say we are tried to prevent strokes with antitrust a fibrillation, most of our focus has been on how many strokes can we prevent, how many please can we cause. It only happens for a fraction of people are where everyone the tablet every day -- waited treating those checked every month, does all of our attention has been on strokes and blades. There is a way of thinking about quality in terms of how much healthcare causes burden on people's daily lives. The second aspect which is related is the value lost by quality improvement efforts. In other words the unintended consequences of quality improvement. Again if I see my patients with diabetes, we have had a major effort to try to get that A1 C down, try to get the Staten started, very recently tried to get the Aspen started, all of the efforts getting people to check their blood sugar, bring their logbooks and, get all the self managed support this activity. Although the impact on patients lives in terms of [ Indiscernible ] again and all the unknown. To what extent is now evidence [ Indiscernible ] for instance folks at Yale following -- hypoglycemia and the patients for whom the glycemic targets were never intended to be that way. What is the loss of value to the healthcare system and the consequences of quality improvement still be part of the research agenda.

I think I basically want to build on what Kevin was saying into place. First of all, with respect to financial incentives, I think it is very important if we're going to go in that direction and I know you know this, but I have to say stuff anyway. To be thoughtful about distinction between financial incentives and talking about physicians, financial incentives at the -- and financial incentives at the collective physician level, whatever that is, some cases it is very easy to identify what that is other cases it is not work there are issues that everybody is aware up with respect to designing financial incentives at the individual level that relate to the problems in small numbers, relate to difficulty in attribution when in fact care is increasingly team-based. Also I think, relate to the ethical issues. Are certain forms of financial incentives that I think can be corrupting at the individual level but in a larger group with checks and balances. The second point is having said all that, I guess my reflection on an organization that dealt with most individual incentives and collective group incentive is that into the financial incentives are very small compared with the nonfinancial incentives. Some of that is difficult to measure, some relates to things like mission, the nature of mission, the nature of the goals of the organization sets and whether or not and how those goals follow through in the actual management, the day-to-day management and instructions and all of that that are inside the organization it is hard to get at. It is one of those things as they say, when you see it, you know it. There are some things I think that are measurable. I think the nature and structure of the peer groups within the entity and whether or not but the physicians are essentially functioning as individual players or whether the entity instructs is structured in such a way that like it or not the physicians are structuring as part of a peer group that has an inherent sense of professionalism and self-regulation at the level of groups of physicians, I think that is measurable. I think also the use of performance information and its transparency or lack of transparency and then what happens is the consequence of that. To what degree performs information is shared the on the individual physician to an appropriate your group and then what are the consequences of that. Those are some nonfinancial incentives that I think are [ Indiscernible ] to measurement and there are probably others.

Mary?

I wanted to reflect we held a focus group this week of patients who had had recent hospitalizations and many of them these were older adults with multiple chronic conditions and they were anywhere from five weeks to 12 weeks to hospitalization. We were trying to reflect on the care they have received during the transition and they were constantly telling us how they were just beginning to feel better, they didn't really want to, they didn't have much negative to say about the healthcare delivery but they had a great deal to say about how they were feeling and actually was so difficult to come to the focus group and sit there and be there for 90 minutes and so on. Reflecting on that, I think this conversation around redirecting the conversation to what matters to people as they experienced what we call our care system essential. I was at a meeting last week in this great speaker was at a panel, talked about the need for a serious change. That would be Rick. I wondered, what is the theory of change that is guiding ARC work. I love the fact that the problems that you are thinking about are not limited to cardiovascular disease but now thinking about a population of people with multiple chronic conditions. As we think about moving toward a new theory of change and producing health, I also think that changes the way we think about the context of care so what is the health delivery system, what is a high-performing health system? I think it can -- existent high-performing communities I think we need to be linking how we how we connect our understanding of our understanding of the relationship because Mr. Smith leaves the hospital and it does have a high-performing, does not get to feel good at five weeks or nine weeks or 12 weeks. Those are some of where I think -- I think what you have are all of these parts, the capacity to look at incentives and the relative role of financial versus intrinsic professional, the capacity of looking at the relative advantages of learning health systems and evidence and tools in creating and advancing our theory of change. I would reflect back to you this is a great opportunity to think about what is the theory of change for ARC in a very different future.

[ Captioner's Transitioning ]

Word is health literacy come into this question because it is not only its health and cost literacy for the provider as well for the patient and I do not know where we start with that but I know it is critical for better outcomes and I think it would, I am always amazed that the disconnect the provider has with cost drivers and that effect on patients.

Paul?

This issue that Jay was talking about, the incentive to organizations and then incentive individual petitioners, think it is very important and I want to reflect my thought he was going to mention it. When he was at the AMA he launched a site visit study of medical practices that RAND conducted as part of the study. What was striking is the degree to which physician practices that we interviewed who had financial incentives on the practice to deliver, decided to very explicitly not to pass those incentives through the individual physicians. They just had better mechanisms that Claudia Steiner for was mentioning. Think it is important when government starts thinking about the financial incentives to individual practitioners like Medicare does and position based modifiers for physicians not to go too far with using incentives for individuals practitioners. To the thing I wanted to bring up and I'm not an expert on quality measurement such as listening to people around the table about how limited our progress has been so far. In finding useful measures of quality is that AHRQ as a research agency is probably been involved in funding some measurement developments. Think it would be an ideal time for it to take a backward look at overall research on quality measurements and how well it has done and has it done well, has it done poorly? How can it be refined or redirect it to perform better in the future?

[ Captioner's Transitioning ]

I think it will be an ideal time to take a backward look at quality measurements and how well it is done is it done well? Has it done poorly? How can it be refined or redirected to perform better in the future?

David Ballard.

[ Indiscernible - low volume ] talked a little bit about the role of incentives. I don't know whether the health services research supplement is going to address that issue in any depth. I think it would be helpful for folks to get a sense of how powerful and large they are in magnitude relative to differential payment by Medicare organizations. My long-term incentive is 1X at maximum.

Mine is .06. [ Laughter ]

My point is --

Can you talk more into your microphone?

My point is that within a lot of not-for-profit healthcare delivery organizations there are very large incentives, in our case for all of our employees who have annual goals related to these things. About 10% of the employees have annual compensation risk related to the sorts of things. I think it would be helpful to understand what those practices are across the United States. What are some of those? We want the measures to include things such as HK scores Dolch HCAP scores, mortality -- HCAP scores and mortality scores. The magnitude of these compensation affect -- effects might not be as large. Those effects are much larger than what the federal government is directly driving in terms of influencing the behavior within our own organizations. One other comment I would like to make is, this on to intended consequence. When we implement electronic health records, physician coding dropped off dramatically. We were pleased with our risk adjusted mortality component to our performance-based -- related to patient safety. The [ Indiscernible ] dropped off the face of the cliff with the implementation of electronic health records. It up our performance-based compensation that your related to risk assessed mortality. Doing this work thoughtfully within healthcare delivery organizations is important. It would be hopeful for ARC to try to study this in some depth and help healthcare delivery organizations do this as well as possible

Andrea?

Thank you. I think that we do have some very good, decent polity metrics, for certain chronic disease states and melter -- wellness metrics but they only measure a finite chunk of that care continuum. Patients go home and they are ready to talk because want to get out of the hospital they have the other stressors and barriers to care that we may not even be contemplating in our measurements. That complicates things. We have tried to develop some composites. We have tried to develop composite care sets in diabetes. But looking at that chronic disease as Jean was saying, they are not looking across the care continuum. I think we have done some good workout but it is very nascent and we need to do research to get to where -- from where we are now to the extra high quality.

Thank you. I feel like Mary and Victor read my diary. What I am about to say is underscoring and bolding and italicizing what they said. I think that there is still the need to define value for the patient, for their families, in a way that is more constructive. We haven't quite figured that out yet. I have been in a lot of meetings with specialty societies and they talk about how we say we have this value equation but we are never taking into account what the patient or the families ideas -- family's idea is. There is a lot of were to be done around that. When it comes to quality metrics, I agree. We have good quality metrics but they are not in any way shape or form going to take in the full context of what the patient or family is dealing with. We need to do work on that. That brings me to a point that I haven't heard too much this morning. Paying for value in the context of population health. When I say population health eyelid to relieve me -- I literally mean the intersection of where the health and [ Indiscernible ]

We get to your theory of change aspect and we can organize and think around how we incorporate the needs of the patients, the family, the community come of the providers together. And then improving and sharing the improving -- the quality that is being provided.

As David I have experienced and have it -- have had a chance to observe many structures over the course of my career. Two points, I am wondering whether any of this special issue will look at the impact of a leadership believe on an organization. To speak broadly, I have seen medical group leaders who would seek to emulate Jack Welch and some that would want to emulate a different doctor. That you could do a case study within Kaiser Permanente because when management to -- negotiated with later measurement -- labor-management union members, there was a play out of how they should be plotted that the individual level or union level. That is ongoing.

To say one more thing about that, the incentives were set at the group level and they were set around four health targets. They were set in a stepped way that allowed all employees to participate in the incentive payouts based on the performance of the group. It is an interesting come a different approach to setting incentives that is worked taking a look at. -- Worth taking a look at. The union embrace this along with the organization so that tells you something about it.

[ Indiscernible - low volume ]

It is a different kind of experiment than anything else than I am aware of. Ann?

To build upon comments that have already been made especially around leadership. One of the questions we ask the dissension is if we look at high, middle and low performing hospitals, and ask the question, why do some adopt standards and protocols around hospital acquired conditions much more rapidly than others. It came back to the same central organizing thing, which is leadership in the adoption curve. I think there is so much to be learned around what creates that adoption curve in organizations, where there are not incentives. What we are trying to introduce right now is the notion that, isn't it time to move on because we don't need new science for most of the hospital acquired conditions. We know what prevents them in the first place. We are either not utilizing that for maintaining it properly. When are we going to shift that into a minimum standard of care and move on with the larger more important issues around the patient that have already been cited? I think that is almost in on tap opportunity of learning. Speaking about the hospitals and organizations, a very large part of the cost and utilization of the health care continuum --

David Atkins.

I have the misfortune of representing an organization that will probably be a case story -- case study in the problems of performance measure and paying for value. We looked at axis -- access and implemented a measure poorly and it was tied to incentives that may or may not have contributed to the perception that it was driving bad behavior. I think and paying for value, there are two components. One is do we know how to measure value. I heard three themes of future research. One is making our measures of value more patient centered and I think we are learning that we are at the flat point of the improvement curve where some of our performance measures are promoting overtreatment rather than appropriate treatment. There is a whole agenda in how to make performance measures both more patient centered and value centered, benefit versus the cost of the improvement. There are important questions about where to measure value, provider level, system-level, how to risk-adjusted. Everyone here is aware of controversy of adjusting for SES because of the potential adverse effects on safety net hospitals if you don't. If you hold it accountable for stuff that is bad outcomes that are affected by SES and not care -- then I think there is an interesting question about, are the measures just a biopsy of some function of overall quality? In which case we want to move or change the measures every year. Or are they the things the way -- that we want to change? I think sums -- something such as hospital infection, we want to measure that because we want to specifically change that. For chronic care measures, they may be a biopsy of some overall thing that distinguishes good systems and that systems. We don't necessarily want to keep the same measure forever because of concerns of treating to the test.

The second component of the value question is incentives. There was a whole research agenda about incentives. I think there have been recent interviews and we did a VA one on pay for performance. The literature is mixed about how big an effect does the effect persist after you stop paying for performance. The results are all over the place. I think it is hard to -- not to tease apart the two separate questions about how you measure value. Once you think you are measuring it well enough, how do you tie that to incentives?

Sherman?

I wanted to add to the drumbeat of the importance of studying what works both financially and nine financially -- non-financially to drive collision performance. -- Clinician performance. I think we know less -- not much about what helps patients unless about what drives the physicians. We don't want to push the financial incentives to the individual physicians because of the results of that. We want to have non-behavioral [ Indiscernible ] incentives. That has all kinds of unforeseen consequences as well. We are used to them so I don't even see them. We need to look at the whole spectrum, financial and nonfinancial. Figure out --, and figure out how to use these tools.

Can you make this quick. I want to wrap it up. Victor, --?

Another thing that might be helpful [ Indiscernible - heavy accent ] is the notion of what we're going to do. One thing is to have a metric or a biopsy and the other thing is to have a target. The pragmatists -- John do we -- Dewey pointed out that the reason to have a target is to know what you are shooting for, not to just see the target. We don't want to just mail the bull's-eye. We want to bit better at caring -- get better at caring.

I want to underscore the importance of continuum of care. We have talked a lot about organizations and we tend to focus on hospitals because that is where the money is. I think for a lot of reasons it is going to be important to make sure that the measures that we look at span both -- a pragmatic reason -- the Congressional budget office has the only -- shows that both the payment is the only one that really counts. There is a real reason for the Affordable Care Act to extend bundle pavements -- payments. Medicare is also thinking about extending bundle payments. We are seeing people wanting to use this broadly. It would be great for ARC to get ahead of this . It is the continuum of care. The other thing is the balloon thing. We know that if we put the incentives in one place and we don't look at the other -- when I go around to other hospitals I see more and more things being deferred to the outpatient post discharge that easily could been done in discharge because you get paid for it discharge -- post discharge and you don't get paid for it in the DRG. The last thing is there are a lot of doctors who operate, most doctors operate with multiple organizations. The typical physician in practice doesn't belong to Kaiser or doesn't belong to just one hospital. The real community away from our big systems which is where more medicine is being practice, people are part of multiple systems and multiple insurers. One of the questions is, how was the son in a way that is integrated. As a primary care physician it used to drive me crazy when I did three different guidelines for how to manage asthma. One said you have to do this twice before you can hospitalize. The other one says three times. The other one says look at the moon is prickle salt over your shoulder. [ Laughter ] The integration of this -- sprinkle salt over your shoulder. [ Laughter ] The integration of this is important and it would be important to talk to physicians and others who practice in communities. Right now I don't think that is well represented in this room.

I have one or 2 brief comments. I want to underscore David's comment about the role of the safety net. As we think about all of these metrics and our research and incentives come of the challenges for, at least in my population, they are different. As we think about population health, much of my population lives under a bridge. Some of the control issues around medications and compliance, they are 20 figure out food and water and medication. As we think through this, let's make sure that we think about where the challengers for the safety it institutions are going to be.

I want to thank you all for helpful comments. I got many answers to this first question, what we should be commissioning. I didn't get so many to the second question. Pictures last comment is on the second -- Victor's last comment is on the second question. David talked about whether these measures are -- we are trying to improve performance in and of itself or are they more a biopsy of something else. Safety measures, readmissions come out we are trying to improve performance on them. But for a lot of the rest of what David mentioned, he mentioned this on chronic care, it's probably more about 20 catalyze more general improvement. -- Trying to catalyze more general improvement. I am still not sure what we are doing to try to figure out what kinds of measures or external public reporting will be most useful in catalyzing internal improvement. As you are on your airplanes on the way back home, if you have further thoughts on that, please let me know. I want to mention also, Mary talked about the importance of having a theory of change and -- in mind. I didn't mention on the introduction part, but I should have, encouraged you to think about -- if you think at all about answering this question -- how does change occur? How does quality improvement occur? In healthcare, we have got three mean methods by which this happens, I think. I credit Chery for this observation. First it is accreditation and having some minimum floor, if you are not above the floor, that's no good. The second and the one that I think is the most prevalent is probably general improvement. We develop evidence about how to do something better. And the.-- the desire that Kevin and others of talked about to do good causes clinicians and organizations to take that evidence and adopt it. No one wants to not doing good. Having said that this is the general way in which and prevent happens, we are in the pickle we are in because there is a lot of improvement that hasn't happened. The third is competition. Providers, if they have public reporting on stuff, will work to improve. David talked about some of this, David Ballard. We want to worked to improve to do better on the stuff that is being rewarded. But all three of those are quite different from the way improvement happens in most of the rest of the economy, where it happens mostly through creative destruction. All the five and dime's are gone because we have Walmart. Most of the independent bookstores are gone. That kind of creative discretion -- destruction for many reasons is not something that we have used much in healthcare. I am not suggesting that we should I'm just saying, having some theory in change in mind is important to try to figure out how to answer these questions.

I would say that the other thing -- I want to make a couple of points. We want to pay attention to workflow. I think that is really underappreciated. I was on a call recently. We have a philosophy that I don't think is unique about making the right thing easy to do. I think that very simple construct -- it turns out the flip of that which I heard a great example of, which is making the wrong thing hard to do -- I think we should take a practice and remove it from the physician's easy to order screen and making them hunt to order something that we were trying to get them to stop doing. This decrease the orders from 5000 orders to 1500 orders just by taking it off of an easy to grab picklist. I think there is a lot to be said for paying more attention to understanding clinical workflow. In some ways the adoption of the electronic health record was both a real and missed opportunity to pay attention to this. The way that improvements happen outside of healthcare is you adopt technology but you change the way you worked so that the technology is a facilitator. I think we did a lot of working on electronic health records and hardwiring bad systems as opposed to saying it is a chance for a system redesign. The other thing is, I think a lot of the poinsettia been made suggest that there is a lot of development award to be done in terms of what we should be measuring. I would urge you to put a lot of emphasis on qualitative methods. In some of the readmission worked that we did, we did some [ Indiscernible ] where we actually followed patients home from the hospital with some filming. A lot of the idea is -- that folks had about what was going to workout blown out of the water by just following a few patients home and watching what they really did a what their real lives look like. I think we often rush to measure stuff based on what we think is the right thing to do as opposed to spending the deeper developmental time figuring out what is the problem, in the way that it presents itself.

I think there is a lot to be learned from studying. As I have gone to folks that actually specialize in innovation. Taking time to observe and notice -- to Paul's point -- particularly people who can observe the workarounds that people aren't even aware of. They are so used to finding a way around a no barrier. This gives you some additional insights into things that might be opportunities for improvement. I am going to think these things are all worth paying attention to. We tend to not want to pay attention to them because they are hard, but they are powerful areas.

Excellent.

I am disappointed. I thought we were going to have two different discussions, one of the first one-on-one on the second one. External public reporting is very different than what we were talking about. I cannot say the difference that I have seen, people don't know they are outliers. On self-assessment, people who are poor performers, always think they have done better to -- better than they have. The rest of the people sitting in this room probably know that they could've done better. That external public reporting, we could spend a lot of time on that. I think I have probably said it well and said enough. [ Laughter ]

Thank you Jean.

[ Indiscernible - speaker too far from microphone ]

There is no public comment so we have a little bit of extra time. If people are okay for another 10 minutes, we will take a break after that. The restrooms are outside if you need to take a break before that. I want to talk about three other items. I want to cover briefly one of them which is much smaller. It is quite connected to the last conversation. This is the question of whether there is any value in trying to develop a measure of what an [ Indiscernible ] healthcare system is. And we want to see if there is value in trying to develop evidence between the relationship between a learning health care system and what we might want a health care system to do. Learning health care systems are all of the rage. Why would we want a learning health care system? We would of course. How would we know when we saw one? This graphic comes from worked that Sarah sinker did -- Sarah Singer and also at Kaiser. This is [ Indiscernible ] the Institute of medicine has been working on this for years.. Would there be value and try to actually measure this? Would there be value in trying to understand the relationship between the extent to which healthcare systems are learning and other stuff? In thinking about this, at least as far as I have gotten, is to think about on the research site, maybe the answer is yes. If this were ever to try to be used in either a credit Dacian or payment -- accreditation or payment, it would likely become a check the box exercise and would not be worth anything. But I would still be interested in a response on this.

Victor?

You mentioned three ways in which change could happen. There is a fourth one that has been presented. I think it was the surgeons from Michigan that did this which is collaboration. Not competition, but collaboration. My impression of learning health systems, my best impression has been when different health care systems collaborate with each other to learn together about doing something. I wouldn't want a measure to focus on the individual performance of one system and its components, but I would love to see one that by virtue of beginning to bring the spotlight to collaboration, shows how learning collaboratives learn together more than individual health systems.

Learning health care system elements are already in into QA -- programs for home as well as joint commission. They are already there. Victors point dries me to point out that I HI -- IHI has its own concise three of change that they have been using with the collaborative process for decades.

The short answer is yes and yes. The longer answer is, with the work that the partnership [ Indiscernible ] and hospital engagement, this is on the ground we are learning from each other and we are trying to address these major issues within the healthcare system. I think there is value not only from those types of programs that other ones that we are investing in now. We want to think about, are they actually out -- producing the outcomes that we want to see in the healthcare systems.

In doing the research, we need to know what we are doing is getting us to [ Indiscernible - low volume ]

Are there any other answers to this question? Are there any knows -- nos?

I would want to see how it aligns with IHI. I would find this helpful in my own organization is internal, what are the pieces we're missing. I think it would help to have a conceptual framework and say this is how you can do a self-assessment within your organization to understand how these ingredients [ Indiscernible - low volume ] it could be both within an organization and across the organization. It would be helpful to give us structure. I would frame it as a self-assessment tool but you could also do it is a addictive thing of system performance. I would see it as one of those HR Q types of tools so it doesn't duplicate what is available in other areas.

I see two things, one you can evaluator look for outcomes on an individual health care system. But then you can also look at what's -- what Victor is talking about, multiple systems collaborating toward some sort of change. I am thinking the latter would be more helpful. It's not -- six of one, half-dozen of another.

Am trying to think around the table of who else was on the learning health system study committee. The notion of a measure of a learning health system seems pretty ambitious given that there were real extraordinary challenges in defining what the core components would be of a learning health system. In this case for example, the extent to which patients and families engagement in that system was considered a core component. There was a lot of interest in that, but not necessarily a a lot of evidence to support a. It seems to me that thinking about the framework that you talked about Tom a may be an antecedent to this is to say, what do we know --, maybe an antecedent to this is to say what is adding to value. I will step back to the last question. [ Indiscernible - multiple speakers ] I -- one of the areas of evidence develop that I think is to medically important and for which there was a great vacuum, is now the understanding about incentives for beneficiaries of care. As we're thinking about shared savings programs and all of the ways in which we are trying to -- I don't know what the new acronym is -- get to better stuff. What might be the role? We always thought of incentives in terms of physicians and hospitals. Shouldn't we be thinking now in that theory of change about the extent to which incentives to teams are beneficiaries might benefit the nature of all of this.

I may be digressing a little. This is come up several times. Mary mentioned again whether patient engagement, patient/family engagement, in systems improvement, what is the evidence-based? My sense is there hasn't been a lot. I am curious. This is such a critical area. Is there an -- a research agenda? There was an injured -- really good framework about the individual, larger health policy level --

We are doing work on that. We have developed the patient/family engagement guide for hospitals that was released a year ago about -- two years ago. It is in the ARC mold of , here is a set of things to try to engage. It has for -- four strategies for patients and families. As far as I'm aware, there is not a large research agenda in the relationship a family -- patient/family engagement related to outcomes. Each of the four areas were areas where there was evidence that engagement in these areas resulted in some improvements. We are beginning -- the beginning work of a similar set of questions in the physician office area -- what would be important to do and how would you do it?

[ Indiscernible - speaker too far from microphone ] is there and outcomes oriented research agenda? Here are the tools. Use the ethical thing to do.

Does it actually change patient experience and outcomes?

I don't think there is much of an evidence-based. Spank that would be a great question.

[ Indiscernible - multiple speakers ] -- that would be a great question.

[ Indiscernible - multiple speakers ] we think we have an opportunity to generate evidence about that engagement and what it produces. I agree with you that folks that I have talked about -- talk to that are passionate about that as being the right thing to do, agree there isn't much evidence and we are probably at a point where we could start studying it. Mary?

I was going to briefly add on this point. The project that Rick mentioned, the hospital guide to patient and family engagement, the first step that we undertook with that project was to do a conference of environmental scan of the evidence. It was fairly opportunistic. We chose where the target areas were where we felt like we had enough evidence. That is how we settled on the four strategies. The main point is definitely not contradictory to Beth's point about limited evidence. That is actually what we found too but we didn't let that constrain us in terms of, there is such an interest in this topic. It is so important. We let the evidence pushed us to where we could do things. Things such as, change of six reports -- shift reports, having patience engage in that, the expert panel felt there was enough evidence to give specific implementation steps for that strategy. That is one example.

Both are true. Limited evidence but nonzero evidence, -- but not zero evidence. That is likely the approach we will take as we move into other settings.

Mary?

Can you hear me? Okay. I wanted to add to, not directly related to this question per se but to questions earlier, about the value of measuring -- the value of measuring value. One of the things that I thought was and said was about the appropriateness of care. Some of the value is tied to decreasing unnecessary care and decreasing waste. I think there should be more research on what is appropriate and what is not appropriate. I know there was a push for unnecessary care and unnecessary antibiotic prescribing and so forth and so on. It seems to me that more needs to be done in that area. The other thing I wanted to say was I wanted to support what was said about collaboration. A lot of the work is about collaboration. I think we need to study more about what makes collaboration happen in a particular community. In the sin of soda -- in Minnesota, collaboration happens everywhere. That is not so true in Arizona. [ Laughter ] What makes it happen? I think that will be key to helping us understand how to make this happen more often.

Victor?

[ Indiscernible ] was waiting longer.

That was the order I wrote you down in.

I want to go back to the patient/family engagement topic. Jeff, we use the guide a lot. We work with hospitals but there are so many hospitals out there that don't even know it exists. When we are working with partnership for patients and with the AG and -- HENs and when we are working with partnership, they want the hard data that says patient and family engagement works. Until you show me that, I am not putting any more money into this.. I would challenge us to think about doing the research project on this and doing it so they can see it in front of their faces. They need to see that this is something they need to do.

Oftentimes those requests for evidence are just excuses. [ Laughter ]

But we want to be able to give it to them.

I think one of the challenges that we have is going back to Rick's original question. What are the capabilities that determine -- what are the things that determine if a system is a learning healthcare system? To what extent is the healthcare system able to pick up innovations that are driven by patients and communities? To the extent that they are doing population health, how can they -- to what extents do they engage with the patient/family members and the community at large. I think it is unfair to ask if it works and to what extent it has an impact, because we still don't know how to do it properly. I find sometimes those questions about efficacy come prematurely and can kill something that is just beginning to emerge. I would recommend that we need to understand how it works well, how to make it better, what are the missed opportunities, one of the gaps. I find these patient community driven innovations, a key component that is absent generally from learning health systems.

That has been very helpful. It has been interesting and helpful on both issues. We have our work it out for us. [ Laughter ] -- Cut out for us. [ Laughter ]

Let's take a break.

[ Indiscernible - speaker too far from microphone ]

You are ahead of your self. We will take a 15 minute break.

[ The meeting is on a 15 minute recess. The session will reconvene at 11AM EDT. Captioner on standby. ]

Good morning and welcome to the Council meeting. --

[ Captioner on standby ]

We are going to try to get started. Return to your seats please.

The one thing I wanted to say is that we have had some comments from people listening on the webcast that they are missing your jewels of wisdom. You need to speak into the microphone. We can hear you in the room but if you are in the web Kies -- webcast the microphone is critical to them being able to capture your pearls of wisdom. When we get to the chat part for the committee, make sure that we are able to hear you. It is my pleasure to introduce Irene Fraser who reconvention, she is retiring. Presumably they will have you on

.. She is the director for the delivery [ Indiscernible ] center for delivery, organization on markets.

I think this is my fifth reading as director. People say you are the new director but I am not really new. We have mostly focused on the work that the agency has been doing around producing evidence to make healthcare safer and to improve quality. You will hear in the presentations both this morning and this afternoon, some workaround quality but also some workaround producing -- worked around quality but also some work around producing the broader view of activities at the agency.

I wanted to say it has been terrific over the past 20 years to be able to worked with some of you individually. It has been really nice. It is also great, as I am 2 or 3 days away from retirement, to be able to share off some of the wonderful work that my staff has been doing both in the research that you will hear about and in creating the data and tools that enable others to do research using this data. There have been over 3000 peer-reviewed articles published using the data from HCUP. It covers 97% of discharges in the country, 47, soon-to-be 48 jurisdictions. That part is pretty well known about the data set. What will be focused on today are some of the enhancements to the data and the use of those enhancements. Over the past few years, we have constantly -- been trying to answer the questions from data by this. This is a census of all the data. It is not a set -- survey. It enables you to get at very small subgroups and very rare conditions which has made it so powerful and also longitudinal.. As we started to evolve to meet new needs and are recurrent strategic planning over the years, as we have said, what is coming next, we have started with emergency department a to and amatory surgery data -- and ambulatory surgery data. We now have 36 states with that data. We have been looking to enhance the clinical composition, laboratory values, etc., strengthening the race and ethnicity data so that we could contribute to that are working more precise worked on disparities. We have been looking at revisits, being able to link the data so that you can look at readmission statistics etc. More recently we are in the process of working out ways to be able to utilize not just years of data but more recent quarterly data which is critical as we start to look at the impact of ACA -- the Affordable Care Act. As you can see from the first like, it is not just data. Our goal is to have the data available for internal use but also to be able to make it useful to policymakers such as million lives campaign [ Indiscernible - speaker too far from microphone ] and the partners of for patients -- partnership for patients as well as researchers. In addition to the data bases we also provide a lot of user support in a lot of ways so that more and more people will be using the data.

There are essentially six types of data sets. There is the national inpatient sample which is most commonly used if you can attract national trends. That is inpatient discharge data from a sample of discharges from all of the hospitals that are in our data set.. The nationwide emergency department simple, now that we have enough states we have been able to create an effective national sample of emergency department data, a special data set on kids, children are not frequently hospitalize and so we wanted to be able to go down and look at national trends in pediatric discharges. Then we also had stated data sets. What happens is in each state, they give us their data. We standardize it so that a researcher that is doing research on 10 states will have it all in the same format over time and across states. Each of the types of data that we have contributes to a state database., Inpatient, emergency department and ambulatory surgery be.

What we are going to be talking about today are three studies that have been done into merrily -- intra-merrily using serving -- different stages of the data. Claudia Steiner will present the first. She is a new director of the delivery division -- she has been there for 20 years.. Pam Owens and epidemiologist in the group, is another researcher. They will use statewide ambulatory data. The second is a newer addition to the HCUP group. Pam Owens was also a part of that paper. [ Indiscernible ] is and epidemiologist. The third will be presented by Herbert long who is a -- Herb Wong was also been here as long as me or longer. Another member of the HCUP group will be joining him. Herber -- you heard a little bit about the medical organization survey. That urged out of worked that Herb has been doing with an external advisory group to try to identify how best to serve physicians. It was nice that we were able to take that work and start to integrate it into the maps.

Without any further a do I am going to turn it over to Claudia.

Now we all have a microphone to speak into.

Good morning. I am quite happy to be able to have an opportunity to present this paper. It was published earlier last year but I am going to pause for a minute. As a be sensitive of [ Indiscernible ] and having been with you since you began the center, I want to offer a heartfelt fond farewell, and wish you a lot of happiness as you graduate from ARC and major transitions to what we know are many other adventures. From all of us that are here -- [ Applause ]

By way of background, many of you know this, surgical site infections also known as FSIs -- SSIs, are among the most common healthcare associated infections known as HAIs, that comfort 20 to 30% of HAI's in hospitalize patients. The first phase of this plan paid attention primarily to the inpatient 10 exclusively to the inpatient arena. The second phase which was finally implemented and put -- put a publicly was in 2013 and expanded to the ambulatory surgery area as well as nursing homes and end-stage renal disease areas. The ambulatory surgeries are the most focused and my paper. As part of that paper and that target, it became clear that there is quite a dearth of information following outpatient surgery and more specifically SSIs.

With that in mind, that generated our study objectives, which were to select a spectrum of load to moderate risk surgeries across body systems. Low to moderate risk means the risk of surgical site infection. Most of you know that cataracts are usually done in a post -- impact -- outpatient setting. They have very few infection so that would not make sense to use this as part of the data set. We want to recognize -- represent the sample across the whole [ Indiscernible ] to determine if there was variation in terms of surgical site infections.

You would want them routinely collected. Many are done both inpatient and outpatient so we measure that the operations we left that were at least 50% of not more outpatient surgeries. We can identify the relatively homogeneous population of patients that were at equally low risk for infections and other competitions. In some way we were setting up for. We did not try to get the most risky patients but as we know, patients by and large are lower risk when they do outpatient surgery. I show you -- I will show you the strategy that we used to do the exclusions.

Finally the objective was to determine the current rate of incidence clinically significant posted -- postsurgical infections defined as coming back to the impact -- inpatient or ambulatory surgery setting to have that infection treated.

I am sorry. I missed the logic of picking the low risk surgery. Why would you not pick surgeries where there was a higher risk of infection

We know that most in the outpatient setting are low risk. From our standpoint knowing it wasn't most information, it was a place to start. You could extend it and try to get to the patients that ended up being admitted as -- admitted or observations day. But given that there was none out there come a we thought that was a good place to start. Fair enough?

Do you know what proportion of infections are accounted for by various groups, low risk, to the account for the majority of postoperative infections and serious infections? I don't know that --.

I don't know that may be as we go through that we can make some sense of that.

From our databases we use the outpatient surgery settings. We are going to use our state ambulatory surgery and services database. We were looking for readmissions and revisits for postsurgical infections. In this case, this has been part of the difficulty, you have to start with your incident, your index surgery, you will have to follow patients back to the inpatient setting. You will have to use two different databases. We know the 30 day is the sender. We were curious about whether you would actually see a fair amount of these at 14 days so we did that per thousand operations.

I will show you this in the algorithm in one minute. Remember this is a the patient level. If the patient at the criteria, they may have had more than one surgery during the year. So patients were represented more than once in the index surgeries. We used a complex algorithm that uses both procedures and diagnoses to find the surgical site infections. That is published online. We use the state inpatient database to see when patients would come back. We used eight states because we want to geographic representation. The states not only had the two databases but they also had verified encrypted patient identifiers that support occupations overtime and settings.

-- Over time and settings.

This is the algorithm and I will point out two things. At the very top you will see we had 424,000 operations across all of the operations we chose. The boxes below are showing the strategies that we used to get to Moritz population. For example, though -- a low risk population. For example, if they had more than a two day stay, we would assume there were competitions if they were not discharge back to home. To come down to the group of patients we were the most interested in, from 424,000 we got down to 284,098 amatory operations, which is about 282,000 patients. -- Ambulatory operations, which is about 282,000 patients.

I split these tables across two slides to make the more visible for presenting. I will orient you to but that the sites. They are similarly form elected -- formatted. This was the high water mark. We took a look at the 200. We took a look at the 284,000 9000 -- 284,000 operations wanted to find out how many patients overall came to the ambulatory surgery setting, overall. Not just for infections, overall. You see 20 per thousand at 14 days and about 34 You see 20 per thousand at 14 days and about 34,000 at 30 days -- 34 per 1000 at 30 days. You will see it is that three per 1000 at 14 days and nearly [ Indiscernible ] 80% come back specifically for SSIs. The other thing to note is that the majority are actually coming back for care in 14 days. You do pick up more and sometimes it is 30% more, sometimes 50% more. The majority are being seen at 14 days -- 30 days. Along the left-hand side you will see the operations that we looked at. We tried to get operations across different surgical specialties, orthopedic care, urology, etc. From these particular operations the highest rate for clinically significant size was six point -- 6.4 per 1000 and 1008.9 for 30 days. -- Per 1000 and it was a .9 for 30 days. The removal of gallbladder and six different hernia repairs -- you can see on the left, [ Indiscernible ] hernia had the lowest. We couldn't but the number because it was so few that came back for care. There was variation across the different operations.

In conclusion, --

[ Indiscernible - speaker too far from microphone ] these are ambulatory?

All of the procedures were done in the same day setting.

[ Indiscernible - speaker too far from microphone ]

People are getting comment go hysterectomies?

It is actually about 40% that are done outpatient. When you see the inpatient you need to make sure that you [ Indiscernible - multiple speakers ] when we have done this using all the data, is about 40%.

That is a lot.

That is -- varies whether it is laparoscopic or abdominal. Conclusion, invocations, the rate of clinically significant surgical infections following same-day surgeries was quite low. That is reassuring. This was in the face of other evidence that there was some other document -- documented control practices which contract with in person -- inpatient procedures. With the rate low you still have so many surgeries that are done outpatient, we know that that translates into a significant amount of patient that are experiencing these competitions. We have eight states, we looked at 12 different operations. I don't know how high that gets Pachuca do multiplication and you probably have thousands of patients that are experiencing these competitions. Therefore, quality improvement efforts still remain important.

I am happy to take any other questions.

Just for clarification, where it says contrast, are you talking about control practices per se? Is that what the difference is?

Yes, and I think rate of surgical site infections are higher following inpatient surgeries.

I am sorry. I wasn't reading a properly. What contrast with inpatient operations is the rate. Were you able to quantitate the infection control?

There had been a recent paper prior to when we went through the revise and resubmit. It looked at infection control practices the outpatient arena and that's what we were commenting on. You may have been worried that you would have more surgical site infections following ambulatory procedures if you had a study that said come a infection control practices didn't look like they were so good. But in fact, even in light of that paper that had come out sometime in the year prior, the rates were actually still quite low.

This rate of three per 1000, do you know what the rate is for inpatients? The overall readmission rate for Medicare patients is now what about 170 per 1000, down from 190 per 1000. Claudia was showing the 30 day readmission revisit rate which was 33. That's totally different. I don't know about surgical site infections.

We are talking about oranges -- apples and oranges. You would have to choose the same operations and the ones that were all done impatient to choose this. It is hard to do those comparatives. That is why we tried -- chose not to do some of those comparisons using our data.

What year did this cover?

This is using 2010 data.

I know that when ambulatory service units go through accreditation with any of the accreditation bodies, all of a sudden the infection control processes looks much more like traditional inpatient stop. CMS required them to go through that to bill for Medicare patients at some point but I don't know exact we when that was. That undoubtedly would impact this.

I think there is actually evaluation going on for the second phase of the HHS national action plan. We will try to provide some data to that that will be a little bit more longitudinal. Perhaps we will see that.

David?

I may have missed this in your presentation. Did you look at some of the facility attributes such as, facility level volume, for-profit, not-for-profit status, joint venture status, any of those issues, as to whether or not those facility level organizational structural factors related to infection occurrence?

We did a little side analysis to take a look at.. I don't think it made a big difference but we had a power problem. I don't know if we could show them. The good news is the power problem. [ Laughter ]

[ Captioners Transitioning ]

It typically happens if you take too much for ticketing combination with other substances such as alcohol. Just as a way of reference, in 2010, since a survey on drug use and health indicated that nearly 12,000,000 people in the country were using opioids in the nonmedical fashion. And the CDC says that 17,000 deaths of over use and misuse of opioids and that number continues to rise. This analysis stems from the request that we received in the HCUP project about the approval for these particular drugs. This was to look at the use of the trends and utilization of hospital services for opioids over time. And we focused on the last 20 years. We wanted to explore if there were any demographics were subgroups that might potentially have access you does -- issues with the opioid overuse.

The national inpatient sample, which I read mentioned earlier is a 20% sample of hospital discharges up to 47 states on an annual basis. It serves as the largest discharge category. It is based on the billing data. I would note that this does not include specialty functions. It does not include Veterans Affairs or defense hospital. Our estimates are conservative because we do not have those groups available. We identified appeared over live -- overuse and the nine codes for adverse affects. This does not include illicit opioids like to win. We want to focus on those who were utilized prescription-based drugs. A shift into the results.

Since the early 1900s, we started in 1993 cut the rate of inpatient stays increased by nearly 150% over the 20 care. From 116 -- to nearly 300 400,000 in 2012. That corresponds to a 5% annual average increase. The next few slides will show you some comparative analysis across some select patient demographic and geographic region. You will notice of this first graph, of the far left bar is the males in the far left is the female. Males represented a much larger share of the hospitalization for opioid overuse compared to females. But 2012, this difference between males and females has diminished in 2012. You will see that is a result of females increasing much faster than males.

This with credit by age group. In 1993, there was a -- much higher range from the 25-44 age range. And you will notice as you look in 2012, that disparity no longer exists. About the -- the 45-64 age group has much higher use. The order age groups increase about 9% per year compared to about 1/2% under the age of 44. This has shifted over time. This next graph shows you a regional distribution by large US regions. You see a 1993, this was a larger problem for the northeastern region. In 2012, the race are the highest in the North Eastern region, this is consistent with other information and other data sources for illicit opioids across the country.

The Midwest have the highest rate over this period. About 9% compared to about 6% in other regions. The last thing I want to share, the expected source of payment. This shows the distribution of opioid related states. For all -- or all other hospitalization. The Medicaid population represents a much larger share of expected source of payment for opioid related days. The second thing you will notice is that among the opioid related states, the Medicare population has doubled the representation. This is as we would have expected. As the agent shift, you expect the explicit -- the expected source of payment to shift as well. The Medicaid and Medicare populations are the expected source of payment for these hospital -- hospitalization.

The race has significantly increased in 1990s -- since 1993. Over the last 20 years, it seems that opioid misuse has brought in across the country. And it has become a more widespread issue. It's important that we continue our efforts to continue these rates of hospitalizations. We are continuing our work in this area and looking at the impact of state policy on utilization of the hospital for services. Many states have implemented a prescription drug monitoring service. And I am happy to take any questions.

Have you been able to tell where prescribing patterns is primary care physicians or emergency rooms, cancer doctors?

Unfortunately, in the HCUP data, there is not a diagnosis code that tells you the source of where they got the opioid from. It only reflects the mechanism, if the individual was dependent or of using the drug.

Do you have data on the related diagnoses that would give you a sense of whether they are being treated for opioids with chronic pain? There is a general perception that we are seeing the swing of the pendulum. That 15 years ago pain was the vital sign. And we were under treating pain. And now we are concerned that we have been indiscriminate in our use. And the VA has -- do you have any way of getting what proportion patients are getting opioids as part of their regular treatment versus through divergent or misuse?

That's another good question. And something we can explore. In this analysis we looked at the number of additional diagnoses that were on the discharge record. We did not do a a lot of exploratory analyses to see what those discharge records were. Part of that was coding practices, but in this particular area, there may be something else going on. Individuals are coming in with multiple complex conditions.

The HCUP data is really cool data. My sense is that hospital discharges, most of it is ambulatory surgery data. At the inpatient side, we have this 20 year time period. And this is extremely valuable. On the other hand, they are limited because they do not follow people over the course of the treatment. We need to answer the question, if Medicare and Medicaid claims follow people over time? This can partially be answered with inpatient data. But to really answer it, you need to see people throughout the course of care. And the HCUP data does not have that part of it.

I'm sorry, maybe I missed it, to do talk about the mortality data can measure Mark -- ?.

We don't make estimates. Might reference to mortality was general mortality as referenced by the statistics.

This analysis did not include emergency admission. It was just people who either lived or were severe a not to be admitted to the hospital?

This is restricted to inpatient. But we did look at the emergency department. The rates in the emergency department are much higher as you would expect. And the difference is similar.

Based on your study, you excluded the illicit drug use. I was wondering why?

The first reason why is for the purposes of the Congressional hearing, they were looking to specifically look at prescription drugs. They were debating issues about putting them out on the US market. We can look at the ICD-9 codes that were associated with illicit drug use.

Was there any diagnosis or was that the primary admitting diagnosis?

[ Indiscernible ] be

And the codes distinguish prescription for the non-scripted opiates?

It does. I had a slide with it.[ laughter ]

[ Indiscernible- multiple speakers ]

It's really the primary admitting data. That is quite impressive.

[ Indiscernible- multiple speakers ]

We have the same analysis looking at principles.

When I saw the Scott this is interesting because, I do not work in this area. I have fellows who do. And a lot of investigators and clinical scholars, are working in this area. I think it would be cut this as an important component to what is known, which is a lot more micro oriented. The question would be, -- I would encourage you to think about -- the research implications of this and how this informs the other research which is out there. From two perspectives, what is clinically. We have created our own monster here. To is, a lot of people believed, that these new drugs are not going to be as addictive as they were. And three, there has been a lot of inappropriately -- inappropriate behavior by manufacturers as well. I would encourage you to think of little bit more. I will go back and look at this again. And think about other research opportunities or agendas that will allow these pieces to come together and provide the policies that we need to figure out how to get a handle on this. I think my understanding is, we are on the upslope of the curve. Is that correct?

Yes.

I have a little bit of an overreaction. When you look at pain, it is still scored as one of the lowest and largest opportunities from hospitalized patients. I do not think we know what that means other than in some of the patients we are interviewing and focus groups as they are leaving, they are clearly documenting the experience of pain. And in many cases, they talk about how the provider did not believe them or belittled them for talking about their pain.

Let me clarify, that is clearly true. But is also clearly true that we are using -- according to the pain experts that I work with, we are using drugs inappropriately and using them in inappropriate ways. I do not mean to imply that we have not dealt with pain. This is a really important area. I do not have a dog in this fight, but I feel it's really important.

The group help has had a big group initiative underway in this area. I do not know if the evaluation is done yet. Something that is worth looking into. They have made a major push on this.

Washington state, they made a major push. We were ahead of the curve. But Washington state has an accessible database for all prescriptions so across the state, you can find whatever the source of the prescription. We also had a big focus and primary care are decreasing the use of opiates. Especially those -- I am forgetting the level now -- the level where on a daily basis where you increase the risk of mortality. And have decreased opiate use dramatically. Part of that has been to address the pain. Every one of those patients have a care plan that is available across the system. So that we are reducing use of opiates while paying very close attention to their pain.

It is clearly a bimodal distribution where most of this is not from prescription drug use. That is what you see in the elderly. I do not see that in the Medicare population, or on the streets. Maybe I am personalizing this.[ laughter ] You know that this is being recorded. Yes, -- this is a problem I learned with pregnant women. With birth problems and birth defects. This is a big public health problem. People are increasing and is overlap. Thinking about a coherent comprehensive research strategy that brings together several lines of investigation could be helpful.

We actually have that being done on exactly that. On opiates women and babies for that reason.

I wanted to tie back to our previous conversation quickly that, we inserted at the medical group level, chronic care plans for opioid -- chronic opioid use in one year and got 90% of those patients, chronic care plan.

Question, to what extent, are the databases that are scanning during this one? Including dental practices question mark

This does not include dental care, only inpatient hospital care.

It seems that dental practices have escaped most of the initiatives regarding quality safety. I wonder if that is another opportunity for us to think of tran1's mission in healthcare improvement. Dental health. I have a ton of prescription drugs and hope for pain that we have not used. 100% of those, -- [ laughter ]

The ethics people will be in touch.

It is a safe environment. 100% of the prescriptions came from the dentist office.

If they were admitted and the results of misuse from that source, they would look at that.

We are looking at downstream consequences.

From unintended consequences, from reading the newspapers in the Northeast I have the impression that opioid prescriptions were constrained. People were being forced into illicit drug use. What it compared to prescription drugs? What is the rate of hospitalization for illicit drugs?

I would be guessing. I can check afterwards.

Before we move to the next presentation, I want to acknowledge a third author from AHRQ and then I forgot to mention who is Rick Kronick who is also an author.

Now I wanted introduce Herb Wong. this

Hello and I am an economist at the agency. I am happy to share with you some findings on the working paper entitled young adults and the affordable care act. As you can see from the slide, there is a team here -- a diverse team here. Motivation for this particular study. As all of you know, one of the early provisions of the affordable care act is the expansion of insurance coverage for young adults up to age 26. Under this provision, Adults remain eligible for insurance coverage from their parents health insurance plans. This provision became effective in September 2010. Subsequently, they have been a number of different studies that looked at the impact of this provision. They began on focusing in on what would happen to the insurance rate? Many of these studies look at survey-based information. They found that the insurance rates were increasing roughly 5% to 6%. There is also a different set of studies that starting to hone in on the utilization of healthcare services.

There are a good number of studies that subsequently came out of that body of work. Mental health utilization increased. Inpatient use increased, emergency department you spell. The two most recent studies in this area, actually produced by a colleague of mine in the center for financing and cost trend, Moriya, she found that inpatient use increased. And a follow-up study that she had with some colleagues found that emergency department you spell.

That is the objective of this particular study? The objective is to really expand and complement this existing body of work. We want to estimate the impact of the affordable care act, dependable -- dependent care coverage , on the emergency department and the inpatient rates.

How is it that this expansion may impact emergency department visits and inpatient use? I see three possible effects that are interacting simultaneously. There is a price of fact, with expansion, people are being covered by insurance. That means that their out-of-pocket costs is falling. And that makes ED use and inpatient use a more attractive option. It has become cheaper to go to the hospital and emergency room and then therefore, you would observe a increase of use there. Another potential mechanism that may be working here is the substitution of fact. Again with expansion, people are getting coverage, it's possible that they have greater access to preventive care and they are using that and that preventive care is keeping them out of the emergency department and out of the hospital. One can expect that maybe ED use and inpatient use would fall. A third possible effect is that for this age cohort, maybe they are not using so many resources. It's possible that you do not see a change. Overall, the total effect in ED visit and inpatient use is the sum of all of these effects. Our research questions honed in on looking at the total number of ED visits and inpatient stays. Whether or not there is of different of fact on the conditions leisure market and whether or not there is a change and payer mix? Because the expectation is that we will observe fewer people in uninsured in the ED session and inpatient setting. And for the private insurance you would expect an increase in those things. The data that we use, are the HCUP databases.

You have heard that the HCUP databases are extensive. One of the things that we will take advantage of is that breath. We would use data for the inpatient rounds 436 states. For the inpatient area, 22 states. By using so many different states, we can control for a a lot of things that are going on. Is that are happening in the area that you cannot control from different studies. And you can focus on those different policy changes. The other aspect that we would take advantage of is the longitudinal data. By using multiple years of data, we can better control for pre-intervention trends that may be happening. The methods that we have deployed are the typical methods that are found in the existing studies. We will hone in on that age group between 19-25. We will benchmark it with the 27-31 age group. We will do some analysis by looking at the 23-25 age group. That age group may have a different impact than the younger ones. Then we will follow a traditional approach and the literature. Basically using what is known as the [ Indiscernible ] model approach. What are our preliminary findings?

Looking at the overall use, the 19-25 age group, emergency department use copy found no change. For the inpatient use, excluding pregnancy and delivery discharges, we doubted increase in inpatient use. Looking at different pair groups, holding in on post with no insurance versus private insurance. We observed that for the emergency department setting and the inpatient setting that the number of folks with no insurance declined. Correspondingly, we discovered that the private insurance coverage for those settings increased.

Looking at the 23-25 age group, we found that emergency department use increased for this age cohort and for inpatient space, excluding pregnancy and delivery, we found in increase in inpatient use. What are our quick take away? Cover expansion overall was associated with a decrease in inpatient use which is largely driven by pregnancy and delivery discharges. When you exclude those discharges, inpatient visits increased and is primarily in condition and more likely to affect young adults. Basically in mental health and substance abuse conditions. This is consistent with the existing literature. And overall, without a decrease in emergency department and inpatient use with no insurance coverage but an increase with private insurance. That is my world wind field trip through this particular study. I know that I am between you and lunch. And I am happy to take any questions.

I assume you included whether they were part of Medicaid expansion or not as a variable? And did you look at whether that, in a stratified analysis, changed things?

The great thing about having that great team up there, we all had different roles. I know that [ Indiscernible Name ] will have more detail on how those things are controlled. I will turn it over to him.

We looked at the results in service slowed down. The difference is coming for the Medicaid [ Indiscernible ].

Is important to know that these data goes through 2012, while there were some states it did early Medicaid expansion, from 2014 and beyond, the methods used here are a dry run. They are working now are trying to figure out both the effects of Medicaid expansion as well as marketplace expansion. This particular analysis goes through 2012. I think I mean mentioned that HCUP was able to get quarterly data. Fairly recently this has been an annual process and we have had to wait until six months after the end of the year before any data is available. We have data now from 2006-2014.

A lot of states collect their data quarterly. That data is not the final data. So you have to be careful. They clean it up. For a fair amount of states, the quarter or they are half your data. Nevertheless you can use it for signal. And we have about 15 states that are bringing in quarterly data. Those of you that work with data, sometimes it comes in as quickly as his problem -- promise but sometimes is does not.

With the data that you look at the reasons -- you were able to exclude -- I am curious if you intend to impact what is going on ? The other thing that feels like a cautionary note, I am assuming that the underlying rates are quite small. The percentage increase is large without that necessarily being a big [ Indiscernible ].

If I understand you correctly, one of the things that we can certainly do is to take a harder look at different conditions. I think that the two drivers that we found, the mental health issues and pregnancy discharges, you are right in terms of the numbers being relatively small. When you take a look at the percentage of, you have to be careful how you translate that to volume. That is certainly something to be aware of.

I think I did this talk while health reform was underway. There is strong evidence that people who have insurance coverage are hired users of the emergency department. It seems like it may go in the opposite direction. It would be interesting to untapped that. There is this big mess that ED use is for the uninsured. But that is not consistent with the facts. Not that they don't -- Point well taken.

What thing that will be interesting for the future it is this shows us what the short-term affects for the price effect. Once people have insurance for a long time, you will expect the substitution effect to take affect. That would be interesting to look at in the next round.

As we have a longer time period, you can look at if these things remain or diminish over time. That is one of the advantages of using this database.

Everyone is quiet because they are ready for lunch. And pictures. Right to pick

At this point if you could join us to take our picture, we will go back to the wellness room behind the front desk. You can get your lunch and talk amongst yourselves and we will reconvene at 12-45 -- 12:45 Eastern standard Time.

[ The event is on a break. The session will reconvene at 12:45 EST. Captioner standing by ]

[ music ]

So we are going to attempt to reconvene. It is my pleasure, let me check who from the NAC is on the conference call line . Sherry Are you still with us?

Yes.

What about Carol? I am here.

Excellent.

Welcome. Now it is my pleasure to introduce Sharon Arnold who is the deputy director of AHRQ peered she will talk about the dissemination implementation framework.

Thank you very much. I would like to say that developing this remark was not -- framework was not an individual effort. A lot of people worked on this at AHRQ. Some of those folks are here and I would invite them to come up and join me in answering any questions you have. We are hoping that this is an interactive session. I would love to get your feedback. We will provide you with our initial thoughts while looking for your feedback. While you -- so that you know, we will be starting to reach out to NIH and other stakeholders in refining this framework. We want to get your initial input first.

The affordable care act requires AHRQ in consultation with NIH to broadly disseminate the research findings that are published by [ Indiscernible ] and other research for other clinical research. Just a small charge. [ laughter ]

Just to keep in mind that AHRQ receives about 16% of the PCOR trust dollars. That is about $100 million a year's. A large sum of money but clearly not sufficient enough to disseminate potential he all of the research that is ever going to be developed or has been developed. Or PCOR related stuff. In addition to dissemination we are charged with supporting work on clinical decision support and creating a website that provides easy access to PCOR finding. That is the majority of what we are doing.

The mission of AHRQ is to make research and quality care more accessible and affordable. Our charge is dissemination. Because AHRQ mission is not to push things out, but to make sure that the evidence is understood and used. We are understandably taking a broader view of what dissemination is. The goal of our dissemination and implementation efforts are to make sure that evidence is understand and use. What we want is to develop a process to capture the relevant research to provide a systematic and transparent approach to focus activities. As I said before, our resources are limited. We cannot disseminate all findings equally.. We need to decide where we will devote our resources and attention. We have developed a draft framework that includes the receipt of nominations, a process to analyze and prioritize the findings. We will analyze the evidence in terms of the quality of the evidence and the potential impact to decide what rises to the tight -- top for more dissemination efforts. We will look at the feasibility assessment for implementation and do a prioritization of the findings to decide how we implement and what we do for implementation. And then we will develop a process to disseminate and support implementation of the findings. We also want to advance understanding through evaluation of the dissemination and implementation efforts.

Now I will go through each of those four things. Please feel free to interrupt.

First, received nominations. We could have chosen to take it upon ourselves to scour the literature and make sure that we found all relevant findings. But we thought that it was within the spirit and intent of the law to work with partners to help them -- to help us identify what are the more relevant findings to disseminate. We are proposing to develop a process that would have a NIH and others nominate what they think are the important findings for us to disseminate. Some of the questions we have is given the need to balance how broadly we go out to seek input and how many ideas and research findings we take into consider disseminating with our limited resources and understanding that from dissemination implementation efforts can be quite intensive and expensive. How do we draw this fine line between being inclusive but not overwhelming ourselves with a number of recommendations and asking folks the most important findings and making sure that we can do it justice in implementation? And keeping in mind that the long put this in place through 2019 it is not unlimited. We do not want to create a burdensome process or something that is a relatively short time.

Why are the funders best to identify what is important?

I would welcome your input as to whether that was an appropriate venue to go out. They would have the best sense that was the research proposals like a man. Why they chose certain things to come out there -- what they are tracking. Potentially they will have a better sense. I am very nervous that if we take it upon ourselves to scour the literature, we may be missing things that we did not know where funded.

This is a good time to figure this out. [ Captioners Transitioning ]

They will be tracking with the findings are and they will potentially have a better sense. I am nervous or -- I am nervous that we may miss things that we didn't know were funded.

As Sharon said, we are trying to figure this out. It is a good time to have a conversation. I would also imagine that soliciting input from researchers in the public that Sharon is calling out, at least with PCOR and NIH would be great. I would like their view of what they think should be disseminated unfunded. VA is having funding research. It would be good to have the input from people who have been funding worked but also from researchers themselves and from the public.

PCOR is not allowed to look at cost in the evaluation of the proposals. We need to be very mindful -- I have seen what they have done. They have done small groups and small population samples. I think what we need to do is consider the impact come of the size of the population that the dissemination would affect. We care about costs because, if it is too expensive for a small amount of people, we are going to miss the big prize.

You are queuing up the next set of slides perfectly.

I am not sure. It sounds to me like you are talking about a relatively informal input process as opposed to come a for example, creating a annual -- a panel that would vote or whatever. The matter how you do it, I suggest that in addition to the groups that you have named, having experienced operations people from the healthcare industry who actually had to do this work, particularly as you approach the feasibility question, would be very important.

I will add one more thing and then Victor can respond. We had been asked to produce a dashboard for our organization demonstrating the impact that Kaiser researches had -- has had on practices in our institution. The research community, if you wondered why do were hearing the collective heart attack on Thursday or Wednesday it was, because they were presenting that. One of the pieces that I think is important to keep in mind is, it is pretty where that there is a single study that by it self fundamentally changes practice or should change practice. I think one of the challenges in this work that you will run up against is understanding what the incremental contribution is. I have been using this terminology called premed analytic framework which is understanding how that bit of evidence Jobson with everything else that we know and then collectively how that moves the world forward it is really hard, I think, to find things that, by themselves, weren't -- weren't a fair amount of action -- warrant a fair amount of action.

They often have to justify their research activity by answering the question, what is it that Kaiser has contributed or NIEHS funded that has changed the world -- NIH has funded that has changed the world. Knowledge is cumulative and your work shows that it is the cumulative knowledge that is advancing this. To avoid that potential pitfall of self-interest justification from the funders, which is where my question is coming from, one could look at third-party efforts that exist today. That way there would be no additional cost to the agency. We want to look at significance and importance and newsworthiness of literature. This is work that the knowledge refinery at [ Indiscernible ] University has been doing for decades. They have crowd source to methodologies around North America for the most part. This is the idea of which of these individual papers, many are systematic reviews, how are they interpreting to practice change. Why not potentially use this as another resource for your work.

That's great. I want to post my question. The statute caused that PCOR and NIH, what other stakeholders should we reach out to? We don't want to create a cumbersome process but we want to make sure that we are getting the best ideas. I flipped to the next slide because I heard a theme. We had anticipated this and agreed that we want to look at both the strength of the evidence of the of -- potential impact of the environment scan to look at receptivity to change in thinking about what to move forward, with a strong push for implementation, both as we asked for nominations, and as we assessed the nominations I came in.

I think that we understand and agree that no one finding by it self is something that we want to implement. We want to understand how the findings contribute to a body of evidence as a consistent -- is there enough there to move practice forward? We don't want to promote whiplash with the new finding here and then another over there. We also want to look at the context of the current practice and understand where the finding comes in. Obviously, our efforts for implementation are going to be different, if this is something we are asking practitioners to do. --, Versus if we are asking not -- them not to do something that they have been doing. The financial incentives that are in place, obviously, are important to understand in the context of how that impacts potential implementation. All of these factors are important and we want to look at them because we want to make sure when we move forward to a significant effort on implementation, we are making sure that we spend our money wisely and moving forward on things that have a likelihood of producing impact.

Mary, your card was up. Then it went down and now it is up again.

You partly answered my question, Sharon. First of all on the issue of the nominations and sells, I would -- themselves, I would recommend a final -- funnel where you would welcome multiple stakeholders from PCOR and NIH. You tell us why this work is important. I look the idea of, we are involved in work right now that has -- what is a body of evidence in the 3 or 4 categories in building an organic database, laying out all of the evidence in talking about how each of the studies added to it. At least one strategy is to say, between this period in this period on this theme, I think PCOR has some themes are priorities -- or priorities. This is what the body of worker showing in this is how each of the nominated papers, studies, whatever, findings, contributed. That is one way to pull together, which is what you are saying I think. We want to find these individual study findings in the context of the bigger question about delivery sissy -- system reform. That maybe one strategy. It allows for the database to grow and the organic and have multiple forms of termination -- dissemination coming from that.

As I see what is happening with PCOR is there are two types of research going on. What they had been doing and then there is also a tendency where there is a trend toward funding some of these big make a studies -- mega studies. Let's say there is a study on something that will be the only study that is going to be out there for 5 or 10 years because it is a multicenter study that is expensive and it's going to take $10 million to $10 million-$20 million to do a no one is doing it anywhere else in the US or elsewhere. Internal you need to think about, one size may not fit all. Secondly, I don't know what your legislative requirements are in terms of, do you have to have something for everything or can you totally pick and choose?

Use of the legislative language -- you saw the legislative like which. That is it. [ Laughter ] There are no regulations. [ Laughter ] That will be the ultimate goal.

To me the issue is two things. One is, I think these things which the people around this table and people like us could do is the easy part. I want to pick up on what Mary said. I think this needs to be stakeholder driven. He stakeholders, the role of research is to make sure that your prioritizing -- you are prioritizing correctly within it. It is like, how many psychiatrists does it take to change a lightbulb. One, but he really wants -- has to want to change. The things that these people are interested in, it's what the physician groups are interested in on the patient's as well. Those are the things that I would encourage you guys to focus on, the ones where people are -- [ Pause ] -- It talks about changing this enchanting that. It's a lot easy if people are already interested and engaged. I would think about structuring some sort of formal process that has this final as well as stakeholders -- funnel as well as stakeholders that you could convene with via email etc. The other thing I would emphasizes this is an opportunity for AHRQ to do good at the same time and do well. What I mean by that is, adoption diffusion innovation communication is something that I know about. I knew that if I was there long enough someone -- a subject will come up that I know about. I know a lot about it but we don't know a lot about what to do. Built in from the beginning is a research design that when you do things -- if you go back two slides you had something about evaluating what you are doing -- I would encourage AHRQ to build in some experimental design to how these are done. That way over a course of implementation, you will have generated information about how to do this better. What works and doesn't worked -- work. In a sense, there is something to funding research as a study -- as we are studying these things. There is something to just actually doing it but it's really good to put the two together.

Kevin?

I wanted to endorse the thoughts that have been put out there. I think these are good ones. I want to push on victors suggestion -- Victor's suggestion about social media cowed sourcing etc. I love that idea. I think that is a question of something where we could push AHRQ more. You would have to start tweeting a lot more.. I really like the idea. If we have a network of folks that represent different sectors, the health system, consumers, patients, -- what do you perceive as information that is helping you to think differently. And then you can aggregate that. That would be something to pilot in a different way. I like that idea.

[ Laughter ] I think almost everything that you have been hearing is you don't want to be at the mercy of NIH and PCOR. The potential for a lot of valuable research coming from other areas. The key thing is you have to engage the range of stakeholders to find that and to help you in screening. I think that is the bottom line. -- I think this is an opportunity to begin to talk about patient and family and how patient and family should be part of that consideration because of the impact.

I am Howard Hall and I direct the office of communications at the agency. I think that we are looking for multiplicity of inputs around these issues. Increasingly as we think about how can users adopt the work that we are discussing right now, that has to also be with a view towards how in the end are the patient's going to be affected and what can they by way of their perspective, put into the decisions that we have to make about what ultimately is going to be disseminated. We want to think in some ways about where there may be particular populations of people who are either in most need of the information or for home or with whom we could potentially have the most impact. -- For home or with whom we could potentially have the most impact. Is there some [ Indiscernible ] with these groups that could lend themselves to this decision. These are issues we will be thinking about very carefully. I think that Sharon is going to be going on to talk little bit about feasibility for implementation. Of those will be part of the factors that will be discussed.

I was thinking the obvious. I am wondering why no one mentioned this. In terms of priority, aren't their priorities already and the national quality strategy has prioritized six priorities. Shouldn't the research findings that you are disseminating for implementation be supported of the national -- the different national initiative such as 1 million hearts, patients at [ Indiscernible ] and so forth. I was wondering.

I think that is a good point. Patty?

We want you to know that patients and families tweak -- Tweet. I am giving Rick some support. [ Laughter ]

I think Rick may have it right. [ Laughter ]

[ Indiscernible - low volume ]

I wanted to register that this is a very helpful conversation to us because we tend to get blocked in by this idea of funding research that is done by PCOR and NIH. We take those findings and the first thing we run into is the single study issue. We need to think about more creative ways of capturing this idea of incremental contribution and when something tips as far is one it is interesting and we put a lot of information -- effort into disseminating this.

This is Joanna Siegal and Stephanie Chang is next to her who runs the [ Background Noise ] that's probably not quite the right name.

Victor?

Your last point about implementation, receptor entity to implementation, I suspect many people in the room were thinking about how the -- if the healthcare system is ready and to what extent it is ready to taking this up. Most of the comparative effectiveness research does not identify winners. What it does is it identifies the features of the options, the magnitude of the benefits and harm, intravenous is -- inconveniences etc. Sometimes the cost as well. It doesn't identify winners. The issue is providing evidence for the decision-makers to decide which of them is better for this patient at this point in time. Those decision-makers tend to be the clinician on the front line or the patient and families. One of the challenges [ Indiscernible - heavy accent ] for implementation is you may made to look at the point of care. This may be in an ICU environment where you might have to worked with the hospital leaders. This might tend to be complicated when you are looking all the way to the patient level in the implementation problem -- process which may include things such as decision-making.

Thank you. We were also conceptualizing that once we had gone through this analysis and prioritization of what we were going to focus on, to us that also included the environmental scan, receptivity to implementation. Were their decision support tools? Where their specialty groups and providers willing to come to the table to do that? Could we gather people together to talk about what they would do? These kinds of dissemination issues and implementation activities we would undertake included some sort of dissemination press release, electronic newsletter, outreach to stakeholders, etc., for all of the nominated findings that were deemed to be important. I mentioned before, one of our requirements is web portal linked with online research databases to help both patients/families and researchers easily search for PCOR findings. We are going to support that and make sure that whatever findings are related to other things -- and then implementation support that would achieve meaningful measurable change and practice. -- Change in practice which would likely be a big bang theory, intensive effort. This would be not unlike the ABC initiative that Rick was in the -- alluding to that we are getting ready to announce. We could see doing one of these based upon findings that are the right -- or smaller versions of those focused on a specific subpopulation or setting. Those of the kinds of things that we were thinking of that might be appropriate for support. We could also think it would be appropriate to identify important research gaps to cycle back to funders and researchers. Where we may know that some thing is right but we don't necessarily know how to implement it, or we think there are pieces of the process that maybe aren't writing place and ready for full-scale implementation and we think that might be an important surface as well. I again would like to hear your thoughts about the different things that we are contemplating in terms of implementation and dissemination.

The work that I did with the AMA in the last couple of years, there is a parallel here. We had done research and identified a whole set of issues and potential solutions for things such as practice workflow, practice efficiency, the use of support personnel in practices, etc., etc. The question that was, how could we get this information out to practicing physicians, particularly in small practices. We did, with some outside help, we did a meta-analysis of about 10 large-scale quality improvement and other change processes that had been going on around the country in the last 10 years or so. Without summarizing that because it is complicated, one take away from their, which relates -- there, which relates to what you have in the slide, this is a broad statement. You should allocate two thirds of your money to implementation support and one third to the actual physical dissemination. Again, this is a broad brush. If 90% of the work is press releases, publications my newsletters, etc. and 10% to supporting those that actually have to make the change, you are way off. Something like one third Something like 1/3/2/3 has generally worked better.

-- 1/3-2/3 has generally worked better.

Two things, one is I think the money will be in the third and fourth bullets, particularly the third bullet. I think one of the things that AHRQ should do is think about what we need by implementation support. I think where we are is both three go and -- both 3 and 4, this is an interim cycle. There are things that we know and that we do for implementation report -- support but most of them have not done well. Giving payers, providers, patients, to find out what they think would work for their constituencies, and then building that intuiting going back and showing what you have an doing this across projects on a [ Indiscernible ] basis. The same thing for the research grants. I think the patients and providers are missing from that fourth bullet. I would add that. When I was on the task force, what I saw as a research gap may be different than what I saw the day before or the morning of and I was taking care of the patient with that problem.

You guys -- we are the only group that brings those things together. That is where AHRQ can make an incremental difference. The first two unnecessary. You are going to do them. But I think the exciting parts are 3 and 4 and thinking about how to build a learning system while we are doing it. What is done two years from now, while it may still be in a paper report, and it may look the same, it will be very different, or it might be different.

I think this is exciting. We live in the world of a selfie. I think everything has to be taken down to, what does it mean to me. [ Laughter ]

Do you have teenagers? [ Laughter ]

They're all gone. [ Laughter ] I think our assumption is that people read and that is probably wrong. They want anything that is quick information. That is what they want. If this is going to go to medical schools and medical students, we need to think, what does it mean for me, and patience and everyone else.

I think a great example of that recently was talking to a group of doctors who admitted that they googled looking for things rather than working their way through what they knew was a much better documented evidence-based item in our guidelines, because this was faster. They acknowledge this. These are people that are part of our leadership. They acknowledge and they were embarrassed to say this.

[ Indiscernible - low volume ] [ Laughter ]

It goes back to making the right thing easy to do. The thing is to say -- not to say that they are bad. It is to say what does that tell us about what we need to do differently, we need to make the tools at the point of care the easy thing to go to and the ones that we know we have vetted. That is the point.

I talked a little bit about putting evaluation into the process. We thought that was important to not only add to the DNI knowledge development but also identify promising strategies for implementation based on evaluation that could be disseminated war -- more widely and provide feedback on our investments to know that we were spending our money wisely. This is our high-level framework. We appreciate the feedback that you have provided so far and we look forward to additional feedback and ongoing feedback, as we refine this and continue to develop this framework into working -- a working program. Any further questions or comments or thoughts?

This is Cherie Davidson on the phone.

Go-ahead.

I wanted to add that I enjoyed working with [ Indiscernible ] on the evidence-based evidence -- committee where we have research presented to the committee and we discussed how that will be best implemented into employer health plans, and working with specialty organizations as well as large employers, such as low back pain, opioids, --. We did labs and CT scans and MRIs and we discussed autism and infertility, all based on research that we have gotten around to the room, both from AHRQ and some other organization such as Kaiser and so on. I was pleased to be working with Corey -- PCORI in the future. It is a good way to bring the employer/employee health plan side as well as being able to put it into our dissemination machine. I wanted to throw that out there. We would be thrilled to keep working on that and adding that to your list of dissemination possibilities.

That is great. Thank you.

I want to add my thanks to Sharon's appreciation for the feedback. The emphasis is on working with stakeholders to figure out about receptivity. Also perhaps the notion about public media and tweeting and getting much broader input than we have in the past. I am sure that will create a variety of challenges if we go in that direction but it is an interesting approach. Sandy, to your comment, I think the exciting part of the work was on the third bullet, figuring out how to do that in effective ways.

And how to identify how we focus our efforts best. What is that one or 2 studies over a couple of years? The resources are such that we are not going to be able to do very many of those. How do we find that gem that is going to have an impact?

Once we generated there, that will have a tremendous impact for us in NIH and other aspects of AHRQ. We might also think about some of the stuff that is going on -- was going on when I was on the task force recently about getting much more thought to communication and how to do that and build off of that. To me, this is the biggest thing we have in medicine. George Bernard Shaw once said the problem with communication is assuming that it has occurred. [ Laughter ] Doctors don't listen to patience and don't have the right questions. We don't give people the information that they need to make decisions. We have a lot more on the shelf that we know we could practice. Anything that we can make that will have leverage to the impact would be great.

Kevin?

Rick and Sharon, I want to follow up on something from last night you said Rick. In the charge to AHRQ it is implementation is not in the language. Did I miss that? Did I misinterpret that? It's around generating evidence and disseminating evidence. Do you want to speak a little bit more. I agree with what others have said, implementation is a real hangup, not just the knowledge dissemination. Where do you see or charge in this space? It is a lot more resource intensive than dissemination. What is your position on where implementation fits into your charge and activities?

I think it fits in very centrally. Our charge more generally is to do stuff that will have beneficial effects for patients. We want to have beneficial effects on practices. If disseminates it remains, put it on a press release and don't panic attention to whether Exley gets done, that doesn't make any sense. That is part -- whether it actually gets done, that doesn't make any sense. That is part of the charge.

There is a lot of tools and pragmatic things and networks. I don't know if that is a discussion for another day. How does one engage in implementation? How far do you have to push it to really drive change? As you said, the new grant around 1 million heart, 6000 practice, it's not 2000 practices but that is more info mentation -- implementation and direction. This is my first meeting.

I don't think there is -- I don't know what the legislative intent was. I have no indication that there was someone who was sitting down and deciding, should they right disseminate and implement and then they struck out the word implement. I treat them as a Carmen part of this -- common part of this.

I'm not being specific to the Affordable Care Act but I think the balance between info mentation and dissemination -- [ Indiscernible - multiple speakers ]

In general, given the resources that we have got, the most effective strategy, I think, is to develop evidence and then to work with various partners who have much more -- many more resources to try to make sure that that evidence is used/implemented. I think even in this -- I shouldn't say even -- in the third bullet on Sharon's previous slide that Sandy remarked on, a lot of the strategy would be sitting down with patients and providers and payers and others to say, here is this evidence. What needs to get done? What you think about the evidence? It looks like stuff that should the used to change practice. What is everyone else think? If there is agreement that the practice to change, what needs to happen, which may involve development of tools and some implementation on the side but not resource -- [ Indiscernible - low volume ] to do it ourselves.

I wanted to support what Kevin is saying. To me CMS seems to be a big your fermenter -- big implementer in all of the work that they do. Hopefully there is collaboration and coordination there. QIO is a large network that can help implement and support the implementation and dissemination.

I only want to say that we completely agree that figuring out how to get the information used is the key step. That there is attention because that is a pretty resource intensive activity. The more we are digging down and trying to get it used, the less hour span is. That is attention that we are aware of. Is gratifying to us to hear that you agree with us and are in training -- are encouraging us to go in the direction. We felt like we needed to check in with you.

If only we could solve the problem. [ Laughter ] [ Indiscernible - speaker too far from microphone ]

[ Indiscernible - multiple speakers ] [ Laughter ]

[ Indiscernible - speaker too far from microphone ]

Another observation I would make where I will have -- I think you will have an opportunity with this work is in the whole overused phrase of coproduction Rome. I think -- realm. I think if you engage patients and families in the development of the research question and in the process of doing research that you are going to get more usable findings at the back and. Similarly in some of the work that we are doing around dissemination and implementation in our own organization, we are bringing the decision-makers in at the front-end, helping say, are we even looking at the right Russians. And then we want to the -- that to be a part of the work is a goes for. What I have observed in projects like that is that there are two things. One, they keep the researchers in the bowling lanes, on the question that they originally asked. In a may have a chance as things develop to ask what about and get the secondary analysis done in a way that helps them think about the info mentation situation. -- Implementation situation. It would be useful to find out how the work was done as well as looking at the findings. That is a new area. People are making about that that will make a difference. It would be a nice time to start trying to capture that information.

Thank you.

I think Steve Colin is up next. Steve is a director of the Center for financing access and cost trends and he is going to introduce the speakers for our next segment.

Thank you Beth. I hope we have more than 20 minutes but I wanted to give a little bit of overview in terms of the centers focus in terms of -- center's focus in terms of access and affordability. We have a large group of very talented health economists doing working this area and aligning very closely with these two and aligning very closely with these 24 priority areas -- two core activities. You heard a bit of this before but I can't miss the opportunity to go deeper for one minute, in terms of capacity. As I mentioned, it is a household survey that allows analysts to reconsider -- reconfigure, health insurance, at health -- eligibility units, longitudinal in nature with the survey. One could oversample vulnerable populations. A lot of the emphasis is looking at the uninsured and what happens over time with its connectivity to the healthcare survey. We have the capacity to look at the long-term uninsured and expenditures, personages -- persistence of the expenditures and rich income data. We have quite a bit of analysis on burden issues in affordability. You have heard about the household survey and the medical provider survey. There is a third arm which is an establishment survey, a very rich component that is 40,000 businesses every year that give details on employer sponsored coverage, premiums, co-pays. Our analysts try to push as much of a return on investment in terms of doing the appropriate research to inform policy in these areas. I am really pleased to introduce three speakers today. The first talk is going to be given by Jessica [ Indiscernible ] a senior economist with any center. She is in our division of social and economic research. Joel Cohen directs the division and will be speaking about, this public insurance for children improve single mothers health care use. Am not going to go through the other introductions until Jessica is finished. We may take some questions right after her talk.

While Jessica is getting ready, I will take 40 seconds to do another health services research commercial. I think one of the underappreciated triumphs of health services research is that CBO in late 2009 and early 2010 estimated that 7 million people were going to enroll in the marketplace in 2014. More or less, 7 million people did enroll. My younger daughter works for a market research form -- firm. She does research for consumer product company's and they come to her and say, we want to sell this product. How many units are regard to sell? She has taught me that she would never make a forecast unless or client told her details about the marketing plan because sales are depending -- dependent on the marketing plan. When clients come to her with products that are similar to products on the shelf already, she can do an estimate on sales. When they come up with a new product, the firm is often not so close. I won't tell you what form -- firm she works for. They have no idea about the marketing plan. This is a new product. They get an incredibly close estimate in part by law. But also because of decades of investment in health services research. The commercial is the basis of the micro simulation models, any PS -- [ Indiscernible - Intermittent Audio ] that is part of the value. Sorry for the commercial. [ Laughter ]

The paper I will talk about today is public health insurance for children and does it improve mothers as pop health care use. The paper I am talking about is coal -- co-authored with Alan who was at the Rutgers University school of Public health and Jessica -- this paper builds on literature that focuses on the benefits to children based on expansions of healthcare insurance. That includes improved access to care for logging children -- low income children, reduce burden of medical spending, improved health outcomes. We are looking at [ Indiscernible ] public health insurance and whether that the expensing -- expansion of that impacts mothers health care. One of the main pathways we are looking at is to see help -- in order to see how this can affect the mothers use of services is a what they would experience if they signed their children off -- up for insurance. [ Indiscernible - low volume ] otherwise reductions and out-of-pocket spending for medical years -- use for children -- of our the data that we are using from the 2013 through 2008 maps -- NEPS we found a savings of 292 we found a savings of $292 when comparing single-family -- single-parent families where there was employed -- employee sponsored insurance. [ Indiscernible - low volume ] reduction of premiums -- of out-of-pocket medical care for children. There is a small literature that has looked at these types of savings with families with employee sponsored insurance. Both looked at the magnitude of the shavings and also -- savings but also how families would spend the savings.

We extend this literature because the other studies didn't look at health care use. They looked at other ways that families might spend money. In our paper we are asking the question, do the savings from enrolling children in public coverage and hands single mothers' use of medical care. We are looking at two different types of others. We are looking at mothers with employer-sponsored insurance and mothers who are uninsured. We are comparing the 1017 -- 1019 use with -- mothers' use [ Indiscernible - low volume ]

Looking on the left-hand side of the table, on the bottom, we look at models for mothers with employer-sponsored insurance. We are comparing families where the mothers have ESI to the ones where the mother is ESI in the children have public coverage. And then for the right side you see that we are comparing mother and children who are uninsured versus a mother who was uninsured and the children have public coverage. Do they mothers have for healthcare use when the children have public coverage? We are using data from the 2001 -- 2008 MEOS -- MEPS. We are looking at low income single mothers ages 25 through 54 in families with children ages zero through 18. Left the independent variables. First for the ESI insurance, we included a measure for out-of-pocket premiums for coverage. We would expect the amounts paid for families with all ESI to be paying more than the families with a mixture of public and private. When we evaluate these results we are looking at the main difference of those different groups.

The key independent variable in the uninsured model is different. That is a 01 indicator for whether the children are enrolled in public insurance or if they are uninsured as their mother. This variable, we can't interpret it as a marriage savings -- measure of savings. Public coverage might affect the mothers use of care -- mothers' use of care. They might have more contact with providers and they prevent -- providers could contact them to address health care needs. That is another possible way to interpret our results. There is not a bias, it's more an interpretation of how this may affect the use of services. We also controlled for a wide range on the right-hand side of the model such as soap you economic status -- such as socioeconomic status, health status etc. We look at a wide range of outcome variables also asked about in the any PS -- MEPS, the likelihood of any spending and the magnitude is spending if spending is positive. We look at the number of office space visits, total and physicians. We look at the preventive services last year for the services listed on this slide.

Turning to the results, single mothers with ESI, we don't find that much. That's not surprising. Since they have providers -- coverage already. There were a few positive results in the likelihood of receiving up happed test likelihood of expenditure, likelihood of any prescription drug use. This is not a conventional levels.

We found a broader range and stronger effects on the results for uninsured Sigel mothers -- single mothers. We found preventative services comment number of office-based visits, going to adopt -- to Doctor, any expenditure and office-based visit expenditure, any mental-health drug use and expenditures on prescription drugs and office-based visits. [ Indiscernible - low volume ] the magnitude of the effects range generally for the likelihood of the use such as preventive use in the something does seven point -- 7% range to 10% range. In conclusion, children's public coverage had impact on the uninsured single mothers' healthcare use. The results have applications -- have implications for possible rollbacks for -- of coverage for children. The bill that was just pass in the house yesterday, funding for two years was extended. -- Funding was extended for two years. ACA in 2019 coverage -- if they children's public coverage is rollback and it -- and they become uninsured, it may change coverage -- medical use for them and also for their mothers.

[ Indiscernible - low volume ] Medicaid expansions on single mother as popped medical care use should take into account -- mothers' medical care use should take into account whether their children have public coverage, in order to make an apples to apples comparison. Thank you.

Are there questions for Jessica? We can wait until all three presentations are complete. We will go forward. Jessica has clearly illustrated the strength of the capacity of using data for behavioral analysis. One of the other strengths of using MEPS is a lot of what if questions. What happens of coverage goes away? What are the changes in utilization of people pick up coverage? The next talk by Ed Miller, he is a senior economist in visualizing -- visualization. Comment not with us but this reflects a column -- a collaboration with several people, all at AHRQ and the urban Institute. Let me introduce add to will give that top. -- Ed who will give that talk.

Jessica's presentation examined potential savings for single mothers of enrolling their children in available public coverage. In this paper, we will example -- examine the implications in the opposite direction. When we think about the potential cost invocations for families -- implications for families if funding from chip is considered. This was posted online on the health affairs website on Wednesday. The motivation for this research is that federal funding extends only through the current fiscal year. If Congress does not extend the funding for CHIP then children are likely to lose eligibility for the separate CHIP coverage. The house has now passed the bill that would extend coverage for two years and now Senate has to pick that up. Our intent was to think about input temptations -- implications of the CHIP funding ending, when and if that occurred. In 2013 there were about 5 million children who were enrolled in separate CHIP policies at some point during the year. While some of those children may have transition to Medicaid, it is clear that there are still millions of people in 39 states at risk of using -- losing public coverage. Among children who are currently enrolled in separate CHIP , 56% have a parent with an ESI. In the overwhelming majority, that will be deemed to be deemed affordable and that also will include an option of dependent coverage. Those two things in combination mean that in the vast majority of cases, these children will not be eligible for subsidized marketplace coverage. Their parents offer of ESI will be the primary pathway likely, to health insurance, if CHIP funding ends. Our goal is to look at the financial implications for families of this potential shifting coverage from CHIP to ESI.

Our center research questions, what our employees as popped -- employees' out-of-pocket ESI premiums for covering their CHIP eligible children . [ Indiscernible - low volume ] marginal cost -- we also examined what availability there was for high deductible plans. This is important because high did that double plans would represent a significant increase in the cost sharing for CHIP recipients. We use the most recently available data for the MEPS insurance component. [ Indiscernible - low volume ] an important caveat is we do not have data specifically for CHIP eligible families. Instead what we do is look at the situation for all workers . In addition to looking at all workers, we also present results for workers in majority low-wage firms which you get us closer to our population of interest, which is low income families with CHIP eligibility -- eligible children. We look at a number of scenarios in this is the results from one scenario. In this scenario, we started with all workers who have single coverage. We examined were estimated what their marginal cost would be of adding two children, 2 or more children to the policy. 43.1% of these workers they have no option of dependent coverage. For these children they would have eligibility to get subsidized market place coverage. On the other hand, we see that 7.6% had access to dependent coverage at zero margin cost. Among those with positive marginal costs, we see a variation of that is to be sure. Those in the [ Indiscernible - low volume ] distribution would pay $8875, to add two dependence to their plan, which is about lifetimes is what it would be in the bottom half of this distribution. For all private-sector workers, we look at five different scenarios in the scenarios differ according to how many children are being covered and whether the worker took up the coverage prior to the scenario. In each of these cases, what you would find is that there is a distribution of cost similar to this but if you're going to cover one child instead of two, the cost shifts down because workers can cover their children under employee +1 rather than family coverage. On the other hand, if the worker did not previously take up the coverage come of the cost will go up because the marginal cost will include covering them cells as well as covering their children. -- Themselves as well as covering their children.

The same set of scenarios that we look at for all private-sector workers, we look at for other groups of workers. Overall we found that the marginal costs were somewhat lower in lower rage -- wage firms but they still can be fairly high especially compared to the family incomes of CHIP eligible families. We found in general that state and local governments are more likely to have access to dependent coverage with zero marginal cost and have a lower overall cost. The most noteworthy result in this light is that a large percentage of workers did not have access to non-deductible plans. 32.4% of the workers in the original scenario, all private-sector workers, did not have access to -- only had access to a high deductible plan. That is either because they didn't have access to dependent coverage for a small percentage. More it is because there were no nonhybrid is a will plans available at AirPlay support -- at their place of worked. This highlights that moving from chick -- CHIP to ESI would cause more increases in cost sharing in addition to the higher premiums.

The implication is that among families not eligible for marketplace of cities, a small fraction of access to dependent coverage at zero marginal cost. Others may have to pay family premiums that are high, relative to both the current CHIP premiums and relative to their incomes . We also find that many only have access to high deductible plans, as an option to cover their children. In summary, we find that rollbacks in public coverage could leave many families with a set of comparatively more costly options for covering their children them the CHIP coverage that they currently hold -- than the CHIP coverage of they currently hold.

Any questions were Ed ?

You posted this on health affairs yesterday. Did this work have any direct influence on what happened yesterday in terms of [ Indiscernible - low volume ]

I would note only that the vote happened after the article was posted. [ Laughter ]

I called my wife.

I am clearly new at this game.

The timing of that, I doubt it. As part of a larger project in collaboration with the urban Institute, Jessica and I also generated a lot of data for the urban Institute that fit into the micro simulation model, which then was part of the [ Indiscernible ] report, Medicaid and CHIP payment advisory commission report. That report goes to Congress and I would like to think that that did something.

This is Carol on the phone.

Go-ahead.

Thank you very much. [ Background Noise ] [ Inaudible - static ] what I took away from this presentation was, families may not be able to afford coverage on their own. [ Muffled Audio ] [ Indiscernible - papers rustling ] would they be penalized under the provisions of the Affordable Care Act for not having insurance? [ Indiscernible - low volume ]

The question is whether families would be penalized for not having coverage?

[ Indiscernible - low volume ] [ Inaudible - static ] [ Indiscernible - papers rustling ]

In many instances, if you look at these premiums for employer-sponsored insurance and you look at the incomes of CHIP eligible families, it does imply that for many of them , if there was their only option of coverage, it would be more than a percent of their income, which would then give them the hardship exemption from the mandate to purchase insurance. That in itself is not -- in addition to being difficult to afford the insurance, they also lack the incentive to get insurance. That leads to more uninsured children because of that.

Thank you.

You pulled to build power and the pride of presentation is actually focusing on trends, in particular looking at the growing gap between public and private payment rates for inpatient hospital care. This represents a collaboration between our center, you can see that our director is the -- a co-opted -- co-author. We were fortunate to have the senior economist at the office of the Director. to give the presentation.

Is great to have the opportunity to present this work. It is a worked in progress. This is a case where your comments and feedback to be hovel to us both in terms of this paper and potential directions for future research. I would like to of knowledge all of the co-others on this paper and in particular Thomas Seldon who is unexpectedly away who paid a -- played a key role.

Before I start, to give you an overview, the question at this paper is asking is, for an average hospital discharge, does the amount that private insurers, Medicare and Medicaid, that they would be expected to pay differ, across those. Types and overtime. That is what we are -- across those payor types and overtime. There is a preconception that private insurers are more generous than Medicare which is more generous and Medicaid. There is more recent evidence showing that gaps have been growing between Medicare and private care. Those glimmers of evidence which I will discuss the next slide are coming in the context of a number of changes to hospital markets in terms of, for example, consolidation, limits to Medicare payment updates as well as the fairly severe economic downturn of recent years.

I will go through existing evidence quickly. From Paul's worked across NSA -- MSAs over a point in time, you see 47% up to 110% difference between private and Medicare gap in 2000 and. The prices paid by private insurers have grown more rapidly than was 2012. These seem like potentially tantalizing more intriguing data points that could point to a change in terms of how hospital stays are being reimburse. The question is, each of them have different limitations. The two key issues are the ability to separate out payor types. Self-pay and/or Medicare advantage would be included with private which would bias the private payment rates downward. The other question is the mix of patients across payers. That is a challenging thing to address. In the context of that, we thought that ideally what would be needed would be nationally representative -- represented data that would represent all payers that could be distinguished. And [ Indiscernible - low volume ] differences in type and severity across payers. We use the component which you have now heard a fair amount about over the last few sessions. This is the data collected from billing offices, medical provider component that was discussed earlier. This also talks about Medicare managed payments as well as those included as well -- this determines what a Medicaid payment is a Medicare as well. This data gives us the ability to look at trends in payment rates. This is part of a larger study.

To give you a concrete example of what the outcome is that we are looking at, one key out come would be, as I have been referring to, relative payment rates between private insurers and Medicare. After accounting for all differences in patient type etc., if we were to find that private insurers for an average discharge paid about $15,000 and made -- Medicare paid about $10,000., We would be saying that that is a 50% payment gap. Again one of the challenges is the ability to make an apples to apples comparison. We took a couple of different approaches to try to standardize payments.

First we used a multi variant analysis to account for characteristics both of patients and the discharges, the bundle of services provided to save and -- patients across hospital stays. To further try to capture the types of services provided across states, we also relied on charges as a measure of resource use.

We can use the payment and charge information from the MEPS . We looked at other data sets, HCUP as well as Medicare cost report, to try to affirm that our approach of using chargers -- charges to further standardize payments, the charges mean the same across all pair types. That was one additional analysis that we took to confirm our approach. The other is that we carefully benchmark the MEPS payment data against a private database as well as Medicare claims. We actually found out that they were similar -- in terms of the results and apologies for the generality, this paper is under review. We think our analysis provide strong evidence that the public -- private to public payment gap has widened considerably. We are interested in your thoughts about what the implications of that might be. You can imagine potential applications in terms of private and health and -- private health insurance premium growth, people's ability to hold onto their private insurance, as well as differences in private and public spending. --, In terms of what is driving variations in growth.

In addition, we are interested in your thoughts about other potential directions for this work. For example, this was focusing on inpatient care but we could also look at payment gaps in outpatient care, which might not track, for a variety of reasons. Another potential avenue for research would be to look at what factors are actually driving this widening payment gap. In relatively urging the -- early stages, we are trying to look across MSAs to look at what variation might be across markets as well and to look at that as a way of looking at what factors might be driving the variations in this payment gap.

Lastly, going back to my example of the $15,000 payment person the $10,000 payment, if you believe the result that there really is a difference in terms of what's private is paying versus Medicare for a comparable hospital discharge, there is also the question about what implications are might be if that widening payment grab -- gap occurs for treatment. I will stop there and welcome your comments.

What I am seeing is the cost being directly passed on to the patient as well as to the private insurer. You have the additional cost, the fee when I hospital is affiliated with a private practice. You have the physician that is not in network that is now creating his own charges. What this study doesn't account for is all of the costs that patients are now paying for that they never have before. It also doesn't account for the billing of hospitals which might tend to want to charge the private insurer a little bit more because they know that that traffic will be more possible there where it wouldn't with Medicare. Is a possibility?

I think the evidence shows that that is certainly what is happening. David has -- he is gone. I don't know if you have any comments from the provider site. Patricia is being circumspect because it is not publish. We shouldn't have the data out there. It looks similar to the more aggregate data that VHA is put together and that MedPAC's has been publishing. There is a wider gap between Medicare and private payments now than it was 10 years ago. The question then is, why. Many hospitals will say, it's because Medicare is not paying enough and therefore we have to go to the private sector and extract more money. Most of the economic research on cost shifting is that it doesn't really look like that. It looks like they are able to -- they are getting more money because they can. The hypothesis would be greater consolidation in the industry. As Patricia said, we are working on to see if we can develop any evidence around that right now. The trend is quite eye-popping. Just in the last decade -- our data only goes through 2012. What we see though started in 2002, of varying market. -- Prices.

Maybe you said this but the private/public gap, is there a gap between me two public payers as well? What is the trend?

They are much closer. The two public ones are much closer. That's the short answer.

Except in California. [ Laughter ]

The data that we have from MEPS is not dense enough to do state or even regional level approaches. It is a much smaller gap. It doesn't seem to have been systematically changing over time. Although if anything, it has been getting smaller. These data, as Patricia said, come from hospital billing offices. We go to the hospital and say, how much did you get Sayed -- paid. For Medicaid those don't usually include a disproportionate share of hospital payments. We have done some research [ Indiscernible ] it looks like Medicaid may be paying even more than Medicare, which is not how many hospitals behave. As previously commented, that depend a lot by the state.

Did you look at hospital margins during the same time period? Were those maintained or did the gap change to maintain those margins? How did that play out?

That's a great question. This gets at some of these potential explanations in a way of what is driving this. If you think that hospitals -- costs are fixed for hospitals such that there is not a lot of ability to very the cost, based on -- vary the cost, based on [ Indiscernible ] on the other hand, the market power kind of interpretation of this finding, there's a lot more potential question about whether costs are inefficiently high.

Hospital margins, at least according to this data, have been increasing. The most recent data point was 2012 or 2013?

2012.

The total operating margin was 6.5%. That was higher than it had been. I am not sure about the number. But it has certainly been increasing.

You said that you wouldn't necessarily expect outpatient cost to be the same. Were you talking about outpatient hospital cost or total?

I was referring to outpatient hospital cost. It would be interesting to hear what folks think here in this room. There are mixed indications in existing literature. Medicare pays different we on the outpatient site. There are movements from inpatient to outpatient. There are a variety of reasons -- that was what was behind my comment that I wouldn't presume it to be the same.

I think the world that Paul did, he may want to comment, a few years ago, suggested that for outpatient hospital services there was a very large gap between private insurance and Medicare. For physician services, MedPAC has been doing worked for years. There, I think, the gap is much smaller and has been relatively constant over time at about 20%.

You have it right. [ Laughter ]

Any other questions?

I am thinking about the drivers again. Do you have data on capital expenditures from hospitals over this time period? Then you could look at what they were used for and whether it would be Medicare populations or commercial populations on which that technology would mostly be used and where they would be getting the cost act.

Do you mean in terms of types of treatments that would be more common among Medicare patients versus the private patients?

I thought the big investments were IT investments.

[ Indiscernible - speaker too far from microphone ]

I would be interested what Paul has to say. I think it would be important to get a view of the hospital as a whole. In answer to your question, are the apps widening? -- The payment gaps widening? If I was trying to run a hospital I would want to look at the whole picture. It would be like Rick going back to your sister.

I daughter.

It's the whole customer. You may have higher margins in some areas and lower margins in other areas but how are you doing overall? Especially given the dynamic shifts that are going on in hospitals with buying physician practices, differential payments, in the outpatient area, shifting of care to the outpatient area. I think it would be important to figure out the company has it [ Indiscernible ] and what the impact on hospitals this and leave it to people like Beth to figure out what that all means.

[ Laughter ]

The other thing I don't know, I am thinking of the whole hospital which shifts over time, the distribution by service line. The consultants that come in two hospitals -- into hospitals are off been pushing the most -- often pushing the most financially positive -- profitable service line. It would be interesting to see if any of that is happening in different markets differently. I don't know if that is something that you guys could do.

One of the potential applications is -- has to do with what is going on with exchanges in the Affordable Care Act and the affordability with closed networks and things like that. We don't want a situation where if there are hospitals that are disproportionately taking care of Medicaid patients it they would be a disadvantage in the private market because -- it's like what happened to safety net hospitals. The ones that are really doing God's worked and taking care of the underserved populations, have the lowest margins and they have the most capital investment. We punish them instead of rewarding them.

Not on this paper, but we are giving you a bit of a smorgasbord of some of the work that gets done in the agency. I would ask, not now because it's a long day -- on your way back home, if you have thoughts about how we might -- what we might do to focus the work as we are thinking about the portfolio moving forward. We will be working on generating evidence about the effects of the coverage expansion. That is basic health services research. We will be doing that both internally as well as encouraging researchers to submit grants on that as we have in the past. We will presume -- pursue some of this work on trying to understand pricing issues, which I think has not gotten as much attention as it should.

We are thinking about some work about how we might be able to use HCUP and MEPS to address the perennial questions of what is the role of new technology in healthcare utilization and spending. Any thoughts or input that you have on these questions, would be greatly appreciated.

I really regret having missed the presentation. It strikes me that using the MEPS survey as a way to monitor price trends and differentials for different payers is a really important thing, because the data have been so weak. This is going to be much more credible and reliable data. I am very enthusiastic.

Take this for what it is worth as an annoying comment.

[ Laughter ]

Thinking about the dialogue that we had that MedPAC over the years, I wonder if using the term gap in this instance is the best term. Generally another areas we tend to think of a gap is a bad paying and usually it needs to be filled out.

Kind of like the opioid presentation closing the gap, was not a good thing.

More like a differential.

Maybe we need another term.

Maybe differential would be better.

[ Indiscernible - multiple speakers ] [ Indiscernible - low volume ]

[ Laughter ]

That's as annoying as you get.

You're going to have to up your game to get annoying.

Thank you very much. I understand that we don't have any -- anyone who has registered formally for public comment. What if anyone in the room would like to provide a public comment, there someone behind me. Welcome. Introduce yourself and your affiliation.

I am Margot Evans from Academy health. Lisa Simpson can't be with us today because she is at an event at the White House with her daughter. She sensory guards and wanted me to tell you how much we rely our -- we count on our relationship with AHRQ and appreciated. We will also be working with Sharon and Jamie and others this fall in a couple of conferences that AHRQ is involved in. Back to Sharon's discussion with you all about dissemination and implementation, the Academy of health is working with 1015 with their first conference and PCORI -- and AHRQ are working together. We have had a lot of Richeson the field and we want to make sure that we have the right role to make sure each presentation is the best possible reflection of health services research. We stand by, we will be calling on some of you to help us. Maybe moderating panels were being on panels themselves. It has been it -- an interesting day and Academy health is always interested in serving. Thank you.

Anyone else? I think it is the wrap. I feel really at a disadvantage to provide any general wrap. We came, we heard, we chatted a lot. We heard that a lot of this is useful to you. Rick?

I would say that the two extended discussions, one on paying for value and the other on dissemination and implementation were both extremely helpful. I thank you. I thought we got great ideas from implementation and dissemination as well as validation and challenges as well. That was useful. I thank you for your time and energy.. -- Please provide feedback to us about how we can make better use of your time.

I was concerned about the agenda seeing lots of presentations but I applaud all of the people who presented. I thought they were all very pithy and concise and hit on really good content. Well done. Thank you to everybody.

For Kevin on others who are new, this is the first on that we have done presentations. We haven't done research results before.

It was well done. Congratulations.

Think everyone. Safe travels.

[ Event Concluded ]