Event ID: 2994006
Event Started: 7/22/2016 8:14:10 AM ET
Please stand by for realtime captions.
Good morning. We are going to start in a minute.
This is Jose, can you hear me?
This is Andy. We will start in a minute or two.
I think we are going to get started. I guess I will call the meeting to order. Unfortunately, I missed my first instruction. We will get into this in more detail but note it is the first meeting for the new AHRQ director . I want to welcome all the members, participants, and those on the webcast. We will do a couple of housekeeping notes. One is, if members need transportation after the meeting, you can sign up at the registration desk after the lunch break. And then, all numbers, at the beginning of the lunch break, we would like you to meet the Pavilion for a group photo which is why I called ahead to make sure the security threat -- badge matched my blouse. For those coming from the public, if you like to make a public, -- comment, there are a couple of opportunities. Please sign up at the registration desk. And I are discussing the rules associated with government meetings. Those with copy and water and food desires, cafeteria is open until 3 o'clock. You can do a self-service there. So, without further ado, let's go around the room and introduce ourselves. If people could use the microphone, thank you.
My name is Jamie Zimmerman and I am the designated management official for the.
[ Rollcall ]
On the phone, Sherry would you introduce yourself?
[ Rollcall ]
I will introduce this person in a more formal way and a minute. If you could look in the packet on your table, the draft minutes from the April 20 meeting and let us know if you have any changes or edit. If not, I will entertain a motion to accept those minutes.
So, it is my great pleasure to introduce you to to Andy who became the gravity of AHRQ on May 2, 2016. Actually, and he is a terrific example of health systems research. He has touched the field and the things AHRQ cares about in many different ways. He is a primary care physician and focused much of his work on studying issues related to primary care and to the ability for low income individuals to access and receive high-quality care. He has done work evaluating Medicaid programs and designing interventions for those with low income and chronic disease. Houston work to try to translate the kind of research into policy and practice. In addition to all of the academic and training work he has done, he is trained some terrific primary care physicians and researchers. He also spent quite a bit of time in this area trying to influence national policy.
We will recognize the us. Years in the health policy world in this country. He was a senior advisory for the secretary putting evaluation from September 2011 to June 2014. He work on establishing Medicare payment codes for transitional care and chronic care management. He was the senior advisor to CMS health launch the innovation accelerator program to stimulate transformation and state Medicaid programs. He is a researcher, physician, educator, and a policymaker. We are really tremendously lucky to have him as the head of AHRQ. Welcome . And, he will now give us the directors update if I stop talking.
Let me say one thing. Shortly after Andy came here, he and I had a chance to have a conversation. It was about the NHQ and some of the kinds of things the NHQ has been doing . What has worked well and what can be improved. One of the things we talked about always to approach the NAC. It is really figuring out, and the patient centered outcomes research world, the relationship between the work that -- does and the work that AHRQ does. We talked about the possibility of forming a subcommittee that might focus on this relationship issue so that we could, leverage the resources of this agency. We were talking about how incredibly effective AHRQ is. Keep in mind when he is talking about whether it seems worthwhile to you to form something like a subcommittee of the NAC on the core and what that might help the agency with. Now, I really will turn it over.
That was exciting. Thank you.
It really is exciting for me to be here. Thank you so much for the time you give. Thank you to the terrific staff to help me prepare for today. I think it will be an exciting agenda and meeting. I am excited about might inaugural NAC meeting. I hope you will be engage and excited as I am. Just to give you a feel of what we're going to try to do. We will start by giving you my directors update which is something you probably recognize from previous meetings. After that, you will deserve a break and we will do that. The directors update will be a chance to present a tutorial and I look forward to interact with you. After the break, we will have a couple major topic areas to delve into. One will be the work that AHRQ has been doing on the synthesis of evidence. I'll tell you what that work has been done historically. We will end up there and have an opportunity for our first set public comments. Then we will transition to lunch. After lunch, we will have our second in depth conversation which is the AHRQ role in quality measurement. . All of that, we will have our second opportunity for comments. We will transition to wrap up and gain any final NAC input. That is the layout for today.
In terms of the update, which is what we will get into, this is how I am participating going through that. I will update you on some of the things that happened since the last time you were here including what we know about the agencies budget at this point. I will focus on some of the activities we've been doing inpatient research. And then, organize some of what we've been doing into our three main categories of what we like to talk about. I will welcome you stopping me at any point to raise your questions. As best -- as said, one of the updates is me.
I just learned about this myself. One of the major updates was a new photo. That was very nice.. This is the first job change I had as a professional. My entire professional career was at UCSF. They have given me a lead to come here at AHRQ. A big change in my life. In the city an opportunity. I want to share about why I did it. I want to see how thankful I am. It has been a wonderful couple months for me.. AHRQ has been in the front of my mind as a health research researcher. My original connection on a personal level with AHRQ began with John Eisenberg. He was on I got to know during my time as a clinical scholar.
I was immediately captivated by his imagination and innovation. These were exciting and big ideas. I had an additional opportunity get mentoring by John. He was the head of the national advisory committee and benefited enormously. This was a very big part of how I was connected with AHRQ. After his untimely death at an early age, I developed a strong relationship with Carolyn Clancy who worked a decade or more here. Who was, extremely instrumental in my own career in terms of being very engaged in the work I was doing in helping me understand the connections between the work and the national agenda developing. Then, the most recent director, Rick chronic -- Kronik was someone I admire tremendously in terms of his ability to think about, in some ways, well, for many years I did health services research thinking, we do this work and we hope the window of opportunity will open and that you will get to go in with your work and hopefully make an impact. I think Rick taught me a lot about really big strategic about identifying when that window might be scheduled to be open and how to plan your work to be ready to have an influence. So, the three previous directors have been very judgmental in my own career helped shape and how we learn about this organization. Being director has reinforced a lot of the wonderful feelings I had about AHRQ.
The people that work at AHRQ, from a culture point of view, are very familiar. They feel like a lot of the people I spent years with. Mission driven and understand complex issues.
Like a primary care division, underappreciated but still very driven to always give their best for the overall goal.. I have a wonderful staff do those to help me understand the issues and prepare me for talks like today. I am grateful for that. I am privileged to serve as the director where I hopefully can take advantage of the input. We use the limited resource to have a large impact on healthcare in the United States.
One of the things I referenced to is, I had an opportunity to talk with Beth about this. This is the first time I have are -- time I ever had a NAC in my life . It is a nice thing to have. But, Beth made a very good suggestion. Maybe it is a good opportunity to reach out and talk to as many of the NAC members about what's your experience has been in the NAC, how you conceptualize it, how we can best take advantage of what you have to offer and how AHRQ can use that information going forward.
I had an opportunity to speak with many of you individually. I appreciate the time and feelings you expressed. I wanted to share back to some of the things I heard. These are some of the things that I think are important. One of the things is, these meetings come up sporadically during the year. We have a few times that we get together face-to-face. So important issues get raised and sometimes they feel like they drop off after the you leave the meeting and we start with new ones. I am interested in try to foster opportunities for you to feel like you continue to engage and talk about those types of issues. Think about them between meetings. How we can foster your reflecting on these things. One thing that Beth referenced is the role to do things that she might find helpful to work with you in terms of things like a subcommittee. That might be an example of how to do that. But also, of course, to remind you, I think many of you do this, to make use of the other staff and myself if you want to be able to follow up on any issues that arise in these meetings so we can continue to work on them together.
Secondly, I heard a lot of positive feedback on meetings. There was a lot of interaction around the input. Our goal is to try to do that. We will try to identify information that will allow you to feel engage. Want to take advantage of the fact that you are here and can offer us input. We want to make these as interactive as possible. Third, the topics we select, we are just driving them from within AHRQ, but we have the opportunity to hear from you about what you think are afforded. We have our perspectives, but the benefits of having a NAC is that we want to hear from you. What is bubbling up as an important issue that we should attend to. Please feel you have an opportunity to help shape the agenda. Today is an example of that were some of the themes we get into are things we have heard from you that you think are important for us to discuss. Finally, I think I heard from any of you that, when you learn about the topics we will discuss and you attempt to go to the website, you get just as lost as we do. On to acknowledge that the website is something we have heard a lot about. It is something that is a high-priority issue to make sure there are ways of organizing the information that allow it to be searched more easily by certain types of key content areas. I want to reassure you that this is a high-priority issue that I identified with feedback from all of you and others here on the staff. We are beginning to take constructive steps toward that.
We want you to feel comfortable so you can find the information you need to help communicate effectively about our program. We want everybody to be able to do that. I want to tell you that the website is something that we are aware of. We need to organize it in different types of ways.
In terms of some of the things that did come up at the last meeting, I did not have the pleasure of being here but was briefed on it. Some of the things that have come up from the last meeting that I think there were issues raised about, I wanted to have more feedback about a line in funding grants. I want to tell you that I will have some materials today that will share some of that information for you. I have been out attending a variety of meetings right tried to share that information. I'm trying to share information about this but I will talk to you more about that here today. As I referenced, we did hear that you wanted to have the ability to give input to us about the topic. That is the case for today.. There was a request for documents to help reinforce the summary information that was presented at the last meeting. I know those materials were prepared and I hope to hear that and new views that want those will receive those. We will continue to do that as part of these meetings when we present you with information so you have the ability to know what you need.
We want you to have materials to refer to afterwards.
So, Beth started to do a little bit of this but I wanted to update you on things that have gone on at the AHRQ -- NAC member level.
--
We are pleased to have an experienced and accomplish person join the NAC. Thank you for your service and becoming part of the NAC. Thank you Mary . Mary was in a very striking red dress on the podium. Congratulations. A very well deserved honor. Want to congratulate our share.
Thank you. This is an accomplished group we have here. One sad note for us, Paul Ginsburg as many of you know, informed us that he was recently appointed to MedPac. Because he is concerned about his time, he has resigned from NAC
We will replace Paul Ginsburg on the NAC One of the things we've done on the website, a new feature, but trying to make people more aware of the connection in different health services research is. This is a picture from our website but some of the individuals that we have begun to profile. You'll notice that Jennifer is one of those individuals there. These are very accomplished researchers. The speaks to the role that is simply to develop career and help them to become accomplished leaders in the field. We can't anticipate continuing develop many of these sorts of profiles. ~History of AHRQ and what is this -- tell the history of AHRQ and what it has done..
I would be remiss to point out, that Chris is beneficiary of --.
These are various points --.
I would also mention --.
Thank you for highlighting that.
That is terrific to know.
I also want to acknowledge that we had a wonderful new recruitment. C Ted here. Ted is a former faculty member at the University of San Diego. Along and distinguish career primary care and help services research. We are extremely pleased to have him come and direct our national Center of excellence in primary care. In that role Ted will that the point percent for the wide range of activities that we do in primary care. I think he will be a fantastic resource and a master for these activities. I hope there will be an opportunity for you to interact with him. He has been here for three weeks. He made me feel extremely experienced and has hit the ground running and had an opportunity to be on a panel with some of our grantees related to primary care-based research networks. He did a terrific job. This wonderful relationships with that group.
Spent something else I want to update you about is that AHRQ is well into his planning for a research summit. Just to give the context for this, there's a process now where we are on an every other year, slightly different weight we are doing a research conference. Every other year will do a very broad-based research conference. On the in between years, we will do a more focused research summit. The research summit is, we identified the topic of this year. It will be on improving diagnosis in healthcare.. This is also on the heels of the national Academy on diagnostic error. Our goal is to highlight the unique role of NAC in his role of patient safety and to try to explore the state of the science in healthcare and discuss ways that AHRQ and key stakeholders can contribute in a clever way to build and improve diagnostic performance. Our goal is to identify research and evidence, tools and training needed,. Patient safety people will clearly hope to set the stage if you will. It is meant to see the connections to parallel the work we do in these three areas of research and evidence. Focus on diagnosis in healthcare. The meeting will help us bring some people into the building and to reach the broader audience, we will be webcasting portions of this summit so we can reach that audience..
Let us transition to talk about the state of the budget. We are coming into the final fiscal year of 2016 budget. The budget was $334 million from Congress which was a $29.7 million decrease. You probably had an opportunity to reflect on that a little bit prior meetings. But to the impact of this, because of the way we get our budget, the way the cut manifests itself focuses on part of our budget whether --
The cuts resulted in real-life consequences. As a result, certain programs had to be eliminated. This was to support states to use some of these measures and a healthcare exchange program. These are programs that these decisions were made prior to me coming here.
Also, decisions were made around trying to find efficiencies to continue some of the work here and to do it in a more efficient way. That included ways we produce the report. We are doing that as a more integrated activity to seek greater efficiency for that. We had to not have any new evidence reviews. The only one we have been able to add is one on opiate research which has been a high-profile issue across the country. We have had no new implementation and rapid cycle research projects.
We had anticipated, had we received our full budget to have the opportunity to do new research related to multiple chronic conditions. That had to be put on hold because of the budget. This again is a high area of focus in the country because of the wreckage that individuals with health and chronic conditions have a complexity that is challenging or our healthcare system to address. These are individuals that are high cost. They may provide opportunities to provide care and the research could be helpful to guide how to do that. Then we also had a reduction of $1 million in new grant funding to reduce the abuse of opioid drugs. We were able to find three grants in this area, but because of the reduced funding, it had an impact on the number of grants we could support.
I want to give you an update on where we are on funding within the budget we did receive. This is where we are to date. We have 21 grants to improve health care quality by accelerate implementation of patient centered outcomes research. If 21 new grants, three of which are hundred through our trust fund. This is the allocation of funds that we received as part of our budget authority. We also receive some mandatory funds. Last call is the amount we have awarded so far fiscal year 2016. We still have decisions to make related to grant applications.
To make healthcare safer we had 53 new grant in that area, some of which are the PCOR trust fund. We've had 11 new grants in the area in increasing accessibility bite evaluating expansions in insurance coverage. 42 new grants in the area of improving healthcare affordability, efficiency, and cost transparency which are funded through the PCOR trust fund of $6.8 million. That gives you feel for the grants we were able to give out this year and we still more to do.
For 2017, there is no budget the past by the Congress and signed by the president. We had put in a request for a total of $363.7 million. That would be $280 million in budget authority and $83 million in what is called PHS evaluation funds. That it's a Magnuson where there is an opportunity for a small percentage of the funds that HHS gets as a whole to distribute among is different agencies. This would be the mechanisms that AHRQ would be requesting. In addition, there is a mandatory allocation to AHRQ related to the PCOR trust funds for 2017. We were anticipating that it would be $106 million which, when combined with the discretionary request with the made the AHRQ budget for 2070, $469.7 million which would have been an increase of $41.2 million from our fiscal year 2016. This is our request. What has happened so far in terms of congressional activities related to the budget request on July 14, the House appropriations committee recommended to upgrade to -- recommended $282 million. That is less than what was allocated to us. However, I think there is a significant silver lining to think about this. This is the first time in four appropriations titles that the house did not terminate or defined -- remove funding from AHRQ. I don't know what the percent increase is from zero. However, also in the House Appropriations Committee, there's language that would resend $150 million from the PCOR trust fund which would also prohibit discretionary funding --. This would also identify --. I mentioned that to point out that AHRQ has not been the only focus of the house activity related to funding. On the Senate side, on June 7, the Senate subcommittee on labor Health and Human Services recommended $324 million for AHRQ which was a $10 million decrease from our 2016 budget. So, more generous than the house but still not at the level that the president had sought.
I want to say that I had an opportunity since becoming tractor to meet informally with many members of Congress and their staff in both chambers to educate them and share with them about the work underway at AHRQ. I do think the feedback I personally felt the and heard from others is growing over time. I also want to share with you that my experience within the department, beginning with the secretary and down has been positive. I have heard in numerous meetings, both face-to-face with the secretary as well as other senior leadership meetings, very strong support for AHRQ. Her articulation of the support in a variety of meetings as well. This is a positive development for troubling to have a highly visible public official communicating effectively about the special will that AHRQ place.
As I'm trying to highlight, we still have a challenge ahead of us in terms of the opportunities for work that we can do at AHRQ that is very important. I want to make sure that you are aware of where we are. It is still alive process. As you are aware, Congress is on a recess for a significant part of the summer. Their ability to work on ultimately passing a budget will work.
There are scenarios on whether there will be a continuing resolution or if they will pass something. We are on standby.
If there are not any questions about the budget, I want to transition to talk a little bit about some of the work we have been doing to update you on patient centered outcomes.
What can we do to help you?
Kind question. It is important for everyone who is concerned about AHRQ to make sure that you are aware of the programs AHRQ has, what the impact is, and to feel confident about being able to communicate and educate others about the AHRQ work an impact. One of the things that I have experienced in talking to members of Congress and staff on the hill, in many cases, they either do not have a full understanding of what the work is, well, so it is important for those that have special knowledge need to articulate that.
One of my goals is to help formulate those messages but what you think is the impact and importance of those programs so we can all speak about it. That is an important part of the process, educating others. The work that is done is very important, but perhaps is it redundant with other work done by other agencies. I think it is important to think about the role AHRQ place. These are some of the things that need to be addressed in some of these communications and education opportunities.
Congress is excited about our persistent -- Chris's and and -- Congress is excited about a precision initiative. I'm wondering, to what extent AHRQ can build, and not in its entirety, but some momentum, to try to pick -- paint a picture of care process. Not just the content of care and try to help so that story. It seems like storytelling is a big challenge. Try to show that you are nice redundant that complement Terry.
-- Redundant but complement Terry -- redundant but complimentary.
I am having face-to-face meetings with doctors and the FDA. I had the opportunity to speak with leadership and CMS. I want to reassure you that I share your views and I think it is important for AHRQ to communicate and plan and away -- plan with sister agencies to identify the unique and special role. At the you articulated very nicely one way AHRQ could contribute to that activity. It is a process. These are very large initiatives. It is important to be on the ground floor to do that. I appreciate your suggestion on that. It is something I thought a little bit about. If you have additional device or other members on how to accomplish that, I am open to that feedback. I agree with you, working along and aligning with other agencies is an important way that AHRQ could have impact. Our special role is communicate and work with healthcare providers to elevate the quality of care to make sure that they are implemented the Safeway to maximize their benefit.
I had an opportunity to talk with Rick before he left. Of course, he tall -- told me about the budget cut an impact they had with support from some of our duties to support you all in the quality and disparities report. I told him that we will continue to support you no matter what. So, I think this gives us an opportunity to work closer together. The other thing I was thinking about, as you know, we do health the United States for the secretary to give to the Congress. Maybe we need to look at a special topic that would be useful in this area for you. Finally, I want to thank you and apologize for stealing him away from your organization. We are very happy to have him and we hope it will help us work closer with you. Thank you.
I learned about Ernie's departure after I signed the deal. I should have read the small print. Ernie is a traffic person. This is experienced with the report. It is wonderful to have a friend and colleague. Thank you so much. I appreciate the sentiments as well. This has been consistent with what I have experienced. A real sense of teamwork and camaraderie.
So, thank you for reinforcing the communication you had. I very much appreciate the help so we can continue have an important report continue.
We haven't had a chance to have a conversation about what to do..
Some of the things I worry about is, you probably have substantive major things that are worrying you at these levels. Sharing those with us would allow us to really, in real-time, at least commiserate and maybe offer some help and support. I would welcome the opportunity to hear what is on your mind that way. The second thing, those have been produced or eliminated in the budget. Perhaps you could elaborate more on action. Where will this it if not here. It would be nice to know what your thoughts are with how this fits.
A couple of things and I will turn to my deputy around some of this. Want to thank you for the group therapy. I welcome that. In terms of my worries, if you will, the topics we are going to delve into in greater detail today, work on the evidence synthesis and quality measurement, well, I've a couple examples. These are areas where I think there are some strategic decisions for us ahead. I think there is some complexity that needs to be addressed. I welcome the chance to delve into that. My thought about how to structure these meetings is to give you a little bit of this higher-level context but to use these topics as ways of trying to go exactly what you are suggesting. In terms of the action network, I don't have all the numbers on the top of my head but I do know that we have continued to use that. We share your view. That is a powerful mechanism for us. It will help us to rapid work to identify changes underway in the field to do this implementation and evaluating delivery system activities. You'll see some examples of where we have done that. That is it useful tool for us to continue to do our work. I don't know Sharon, if you have any words of that.
The budget cuts reflected our appropriate budget but we see a strong role for the action network in particular in our mandate to disseminate PCOR information into practice. So, our appropriate it dollars maybe sniffly reduced for the action network, we still see that as an important vehicle and we anticipate that the trust fund dollars devoted to that will increase. Looking at other agencies to pitch the use that resource. We recognize it is an important resource. Unfortunately, in the short term, our appropriated dollars need to be limited.
Let's talk about some of the updated information on patient centered outcomes. -- Is mentioned and the legislation in the PCOR trust fund. The PCOR trust fund is the mechanism through which any entity was formed. It also is the basis of funds that go to the Department of Health and Human Services and specifically AHRQ receives a majority of funds. We are, in a sense, a twin. Although admittedly we are not identical twins.
The focus a really designated for work related to dissemination of PCOR evidence. Evidence will be developed from research that goes on. That's NAC will disseminate evidence. --. One of the things that we have done since the last time is that we set up a formal mechanism that invites researchers and other stakeholders to nominate findings that have been shown to improve health outcomes and need widespread investment to accomplish adoption. So we announced this at the Academy meeting. Is not information on our website about this. We are really looking for what is high-quality evidence that needs this push. We anticipate that there will be the development of more of this type of evidence. But we are ready to be able to be in a position to evaluate that. We will assess those nominations in terms of the quality of the evidence and the potential impact on health outcomes. We're really looking forward nominations of great ideas. We hope this will be useful in thinking about this and alerting others to this opportunity. As I say, based on our assessment, we will then consider them for dissemination and implementation activities.
-- Our wait for us to prioritize the nominations that come in. Obviously, we are assuming that all nominations that come in will identify worthy projects for investment. Maybe we will have the resources to do this. We can identify what we do and how we do that. We will look to those categories.
Can you give an example? This is a little vague.
A hypothetical example of something that falls under this.
If there has been a direct comparison of different strategies or treatments to take care of an issue. Someone in the agency mentioned the other day that there was a society supported that looked at evaluating pain for kidney stones in the emergency department in which there was a comparison study of using ultrasound -- scan. This has not been scaled sincerely to health systems to think about the practical realities on how to do that. What are the steps taken to follow through on that evidence. It is a matter of thinking about this evidence he ascertained in clinical trials. AHRQ is anticipating trying to use its available mechanism grants and contracts to think about how to scale up these activities.
That is correct. We see this is not, we will write it on the website, but think about that. We're quite interested in learning about the process. We anticipate that frankly there are things how to effectively do dissemination and implementation so that there is a set of methods and skills that we think need to be further developed. Fairly, we have a long way to go in this country to become more efficient in doing that.
Another area we have been focused on is in the area of health IT and patient reported outcomes. These are some of the have taken so far to try to do that. Basically, we think it is important opportunity. This captures how AHRQ doesn't work -- how AHRQ does its work. Trying to find catalyst opportunities. How do we come the added value. One of the areas that we try to think about is, what is the role of patient reported outcomes? How did they get into electronic health records? How can we catalyze the process if there are valuable lessons to that. We've done a few activities so far. We used our action network to take advantage to do a landscape analysis of what has been going on in the area of measuring patient centered outcomes in ambulatory settings and using those in a laconic health records or other kinds of ways. Try to identify usefulness of this affirmation in different types of settings. We did a quick assessment of the landscape to understand if this is an important concept. We also convened a meeting in which we brought together payers, providers, developers of electronic health records to talk about their perspectives on the value of this information. The challenges of collecting it and making it actionable and so forth. We've also put on -- out a special emphasis notice on which we have calls of investigators to think about this is something we would like to see grants developed on to try to see what is the use of patient reported outcomes and patient contextual data to improve quality and comes in the ambulatory care setting again. And also, we have developed a concept -- -- place a function within Health and Human Services to try to build data infrastructure. There is an opportunity for different agencies to come together to identify what is important and billing data infrastructure. We have proposed to work with -- on this work and the internal assessment of the project. It is very highly rated. So, we do think there is exciting work developing in terms of how to align the work related to patient centered outcomes to capture the information to make it actionable through electronic health records. Another area that we are active in the area of clinical decision support. We have developed a cooperative agreement. It uses clinical support as a tool to incorporate patients had heard that comes into clinical practice. What is important for patients to be able to engage in decisions about their own health care. In addition to working with RTI, we developed a couple funding decisions. One is to support the implementation of patient centered outcomes. We are really trying to develop the research understanding of this important area. How can we make it possible and a natural part of healthcare delivery to support patients to make decisions about their own health care so that we're really trying to develop research to make this more of a reality in healthcare delivery.
I want to come back to something that was said in the beginning. We think this area of investments related to patient centered outcomes research is an important new frontier. The question has been raised about whether it might be an area that requires ongoing attention from members of the NAC and there may be opportunities for members of the NAC to engage in these discussions with some of the key partners who work with us in this area. So, I wanted to come back to this issue because I think it is something that would be helpful for us to hear, whether it be a good direction to think about -- to contribute to thinking while we develop these important areas work on in this area. I welcome a chance to hear about that.
It seems like a good idea. One thing that went to my mind when I can through your discussion, the focus on patient centered outcomes. And insights that was patient contextual variables. The big push for the patient contextual variables that are essentially genetic. There are at least 35 different categories of which genetics --. It is beyond social behavioral determinants. If you look at our ability to understand the processes of care and measure them. This would be a potential opportunity -- that are resident that will help determine the patient centered outcomes to help explain the variation for targeted opportunities for improvement.
When Beth made the original suggestion it sounded there's a subcommittee of this. What I think I heard you say, perhaps it would be a subcommittee of this plus people from the advisory board. That joint committee could be --.
If such a thing is conceivable.
That would be discussion of the chair but it is a mechanism that is available to the chair in this committee to decide whether you like -- and those other entities that are reference.'s back for my perspective, anything that facilitates coordination, --. There are a number of people around this table who live at the sharp end of the stick at the implementation and of things. I know our own experience and what we have learned, we're trying to embed patient centered outcomes. This goes to trying to make it useful for patients and clinicians. There is a lot of wisdom to be shared. Or areas where --. The opportunity to talk to some of the federal partners is a great opportunity.
Earlier and he was talking with some of the challenges. I think this subcommittee could help with what is -- is doing in the space. I think this could be very powerful to help clarify the positions.
I wondered whether that confusion you .2 is part of the budget battles that are ongoing.
[ Indiscernible - low volume ]
I do healthcare disparities research, particularly with diabetes. I am delighted to be here.
I've also been thinking along the lines of having subcommittee that involves other sister agencies. I only hesitation is the concern for bureaucratic redundancy. On to harness the power of the good intellectual debate and minimize unnecessary bureaucratic stuff. Try to find that appropriate balance.
Then also thinking about these other contextual factors. I also think a little more broadly about some of the determinants that I think a lot of us that are interested in don't have a lot of work in. That might be another area to focus on.
Have another comment. Many of you know that after John came to AHRQ, he gave an elevator speech. Like --. He couldn't clearly articulate what AHRQ is about. It comes back to John's leadership.
Just, procedurally, what I would like to do is, make a general proposal and get a vote for me about the creation of the subcommittee. We can take it off-line to add value and try to work on what the charge to the group might be. But let me just get a show of hands of those supportive of the concept of creating this.
I am also supportive.
I also wanted to check in and see if this subcommittee would include -- once we rolled off.
Yes, good point.
While I have you, one of the things I've been thinking about has yet been talking is that, I will be spending more of my time trying to coordinate between the national is this group and the commercial employer world and the public private world. I think there is an opportunity that we should absolutely be trying to take advantage of to use all of the great work AHRQ is doing to figure out how to best do that, you know, with the commercial site. I am happy to try to figure out how to do that best because there's so much work that you are doing that isn't being disseminated on that side of things.
Now that I've been reminded that people are looking for the phone, check to see if there's anything you want to add?
I sent you an email. I wholeheartedly concur. I'd like to volunteer for the other perspective that is urgent is having the patient's perspective about what is important from their perspective.
Actually, I will say for once my life, I was paying attention and not multitasking. If I forget about those on the phone again, Jamie is monitoring email. So, please send her an email and she will prod me if I forget you again.
Yes, I agree with everything that has been set.
Great. Thanks.
I think I need to move into the lightning round. You provided terrific comments. The good news is all the things are in the slides. But I will highlight a few things here in terms of updates related to our work and research and evidence, tools, training, data, and methods. This again are the ways we think about our work here. In terms of a couple of announcements, just to make you aware of them, we have a special -- special emphasis notice. Just so you are clear, doesn't mean there is guaranteed money related but there is a way to signal to the research community about things we are interested in seeing. We are interested in seeing grants on hand -- those with bring multiple chronic conditions but there is a funny opportunity which means there are funds for it to seek cooperative agreement applications to support dissemination and implementation of a subset of new child healthcare developed by the pediatric quality measures program centers of excellence. This is about moving forward and work that has already been done about identifying pediatric measures making them able to be implemented.
I also want to highlight, --, there was a why the noted set of grants we were able to give $9 million in new grants. Over a three-year period to support primary care practices in rural communities delivering medication assisted treatment to patients with opioid addiction.
I've listed here the three grants. Because of time, I cannot go into great detail about them but I want to highlight something that I think is gratifying to all of us here. We had a terrific set of applications that we got in response to this call. Because of the funding limitations you are only able to support three of these, but this fills a niche in terms of supporting primary care in rural areas. We have worked with sister agencies around identifying gaps in some of their work and how we could partner with them on this. Something that was exciting to us in the grants that rose to the top aside from the impact we think they will have, all of them have as part of their backbone, a telehealth strategy that many of you may know by the name of the project ECHO. You may or may not know, AHRQ was essentially the first under of project ECHO to evaluate their impact of the program to support rural primary care providers in the care of patients with hepatitis C. It is now become a model and many other health conditions and frankly is a model used in many countries around the globe. But I think it really underscores how AHRQ has this incredible impact because of the work it did with project ECHO and the beginning. This is an exciting and important story to understand about how AHRQ takes its limited resources and impacts.
If you would like to know more about any of these we are happy to provide you additional information. Because of the shortness of time, I think I need to move ahead. I do want to also make you aware of, we've had some updates since the last meeting in terms of some final recommendations and draft recommendations from our US services and prevention services task force. There have been final recommendations -- and for screening for colorectal cancer. You can see the topics that of a draft recommendations and what some of the upcoming draft regulations for upcoming public comment our. Personally, I have the opportunity for the first time to meet the US prevention services task force. They were sitting pretty much where you are now earlier this week. This is a remarkable process to see the level of science and consideration given this process. We're really proud to be associated with this work. We've also done a lot of work on how we help and support them to communicate related to the important work that they do.
This does these are some of the updates. These are some of the reviews. And method reports that they have done. I think we will have a chance to go deeper into the evidence-based practice centers as part of our late morning session. Again, I will limit discussions but this will give you a sense of some of the topics. I want to highlight the most recent one on telehealth that I would recommend to you. Is a mapping of the evidence for patient outcomes from systematic reviews. It focuses on telehealth and I think had some important findings about what we know currently from the literature and what the gaps in literature our. Research should start to shift from assessment on how to promote telehealth to broader implementation to address the barriers. That is one of the main conclusions of this report. It is a traffic report and I think it is something that you will find valuable. In terms of our tools and training, I will give you some of the quick highlights. We have some new toolkits. I asked, for my own education around the work in the area of toolkits, for your general knowledge, -- supports 36 toolkits in different areas of healthcare delivery. Really, are growing -- of these toolkits which are really meant to support the translation of our research and evidence into practice change for frontline providers and health systems. -- Is about helping health systems and health providers communicate openly with patients and families one harm occurs in the delivery of healthcare. This has been rolled out in the last few weeks. We also have had the ability to evaluate and look at the opportunity to work in our -- programs, the comprehensive unit-based safety programs. It's related to the work of catheters. That is the source of the most common type of hospital acquired infections. Through artwork and the implementation of evidence developed, we have identified a 32% reduction in non-ICU catheter related UTI. This builds upon the work that we have had with previous great success. We also have a new tool that is helping to use electronic health record data. A tool to identify adverse events in pediatric patients. This tool is meant to help staff to identify report adverse events and to make it easier for them to track them over time. This is also work that has recently rolled out. Then we are also doing work in the area of integrating behavioral health and primary care. This is a new playbook related to this important topic area. It has had increased attention.
Those are some of the tools. Again, I welcome you to take a look at that on the slides that were provided for you if you have any questions, you are welcome to bring that up at any time.
Relates to our training activities, I want to point out that we have conducted an independent program evaluation of our award programs. We did some comparison of those that received awards from us as compared to applicants who were not successful with us. We also used a comparison of awardees at the NIH. Some of the highlights of this evaluation to see the effect of what our awards have been. Higher publication rates. The respondents --. Obviously, we compare individuals were funded further awards, there are some selection difference is going on. I think that nonetheless, we did not harm them. Truthfully, it is nice to have an ability to reflect on this program.
Did you assess what percentage is still and research overtime?
I wanted to know what our success rate is.
Yes you did. We looked at career trajectory and research as well as other services. We did tease out a fair number of persons.
I don't know if there is similar information, but, other awards, federal or otherwise.
Thank you very much. We're doing two things. One is, we present a qualitative assessment of our program. We're in a process of planning and evaluation of our investment.
The reason I say that is, I work with a lot of specialties. Spend the broader question I have, -- I think one of these was evaluation. I wondered, I understand why it might happen in an acute sense. But, I think from the long-term success of any program, is important to know what is working and what is not working. I would encourage you to try to figure out how to reinstate that for major initiatives and things like this.
Thank you. Excellent comments. I think it is useful to have this type of evaluation when we are forced to make hard decisions. It helps guide us to make general decisions about impacts we're having. It is very useful for the agency to have this kind of evaluation. It is consistent with our mission which is evidence-based. Thank you.
And to be able to go back to Congress which has responsibly been able to show them how they are funding and how it is productive.
Just as a foreshadowing, I thought about the topic at a future meeting that would be of value to me. It would be to gain your was related to a robust ways of thinking about the impact of programs. I think, as many of you know, we are good at counting publications and not so good at all the things after that. So, developing more robust ways of evaluating impact, particularly when we think about areas of dissemination and implementation. What is a measurement of impact. I also think this has potential way of influencing academic institutions to have other ways of teaching promotion and other ways they could evaluate. It has been something prominent on my mind. Not today, but we welcome future meeting in but.
Want to congratulate you. This presentation is the most balance and best I have seen. It is just the range of activities and impact, better than I've seen before. Secondly, in terms of the IT, there are other opportunities to partner. -- Is sitting to try to figure out how it's healthy planet app can bring in data to bring improvement. Those are the kinds of partnerships that would be very important. Can who is another trainee sitting with his platform to use --, which is really underrepresented. 4% of the reuse --. That is much more than getting into the traditional EMR. This is the future how we get out and measure health and well-being functional status, condition specific or not. There is an area where think AHRQ can play a leading role . Finally, in terms of evaluation, I have to agree that the current way in which HHS is evaluating million dollars project is not functional are very help will most of the time. I remember, well, it may have been the national meeting that NORC was talking about the real-time evaluation of projects. And talking to the lead investigator, I said I don't see real-time here. I see six-month interventions. This is not helpful to the implementers and it is not real-time and so forth. And evaluation of the evaluation is a good role that could be played. We came back from a global seminar on this topic over in Austria.
There are various ways in which we could be influential in that field and sobbing what I think is the trifling a way of vast sums of money.
Thank you very much. Very helpful.
Quickly I want to point out that we have of funding announcement of for a -- career development award which is the parallel for the 08 which we have for the clinical scientist. Targeting clinical opting for people and research. And then, just to talk a little bit about our work in data and measures. I wanted to draw your attention back to something that I think you might've heard a little bit about before. I think it is an exciting development in one of the projects I have an opportunity to get in on. The foundation was built before I got here. That is for to do comparative health system performance -- performance. What is it about what we're seeing? Tremendous change in the organization of healthcare. Consolidation of a POs. New organization units. We have some ideas, but why these organizational units meet offer benefits in terms of quality or value and so forth. We don't have a systematic way of understanding the taxonomy of these organizations and what is ultimately making a difference. Think we are all aware that there are some decisions that started off being integrated delivery systems from the ground up and others that are trying to do it after the fact. To those and upping the same thing or do they function differently. We are excited. We are three major partners as part of a cooperative agreement. --
In addition we have a coordinating center. -- Is supporting us. Really, what we're trying to do is develop a way to help inform the health service community about ways of thinking about describing these organizations and to know what all of the health systems are in the United States and to ultimately be in a position to then measure important outcomes related to those organizations so that we can again look at the relationship between organizations and important outcomes. You have a slight that describes some of the particular focus of some of the different partners that are involved with us. The thing that I am very excited about is that AHRQ will put together a compendium on comparative health system performance that will identify all the health systems and their physicians. This is something we anticipate being able to update overtime and to then link for combined evidence and data that our partners are providing to us. We well, -- we will be doing some other data supported projects. Similar to others where we develop the data resource and develop an Jamail Graham to help develop tools are ways of thinking about using that resource. Make it publicly available so that researchers can leverage the resources and help us think about all the ways it might become even more useful to give us insights about the health system. I think this has the potential to be a major contribution in the field of health services in terms of really giving us a much more systematic way of looking at the healthcare landscape. And, one of the questions I wanted to get input on from you is, what are the most important research questions we need to ask about health systems and the kinds of data elements that we should include from the data compendium. I want to hold that for a minute. We're running low on time.
Finally, since our last tragic, we put together our national healthcare quality and disparities report. That is available. The highlights are noted on the slide. Again, you have these selected findings. I think I have not exhausted the slides I have. I hope I did not exhausted all of you. It has been fabulous hearing your comments and questions but again it would be really help will to me if we have additional time as we go through our data get more evident about what we are doing in the area of health systems compendium. We are in an early stage of that. We have good ideas that we will welcome some of the expertise around the table in these areas. Thank you very much.
So, here is what we will try to do with time. Your break has five minutes left. So, enjoy it. Go wild. If you could try to get back and about 10 minutes, that will help us catch up a little bit on time. In those 10 minutes, you can spend five of those taking a break and five giving Andy questions. Or, write them down an email them. I'm sorry we didn't get to it. There are a lot of people around the table that could give excellent contributions to the list of things that might be possible. So, 10 minute break. Comeback. We will talk about evidence summaries.
[ Meeting on break for 10 minutes and will resume at 10:35 AM ET. Captioner on stand by. ]
[ Captioners transitioning. ]
We are going to try to get started if everyone would please take their seats.
Before we move to the next topic, I just want to thank Sandy for once again filling in the gaps of the feds refusing to feed us. [ Laughter ]
Next meeting I am going to fund to grant their [ Laughter ]
This is what we are looking for from the NAC. Leverage.
Back again by popular demand is Arlene Bierman, M.D., M.S.. I told her she should stop doing such a good job of leading us through discussion areas because we will just keep asking her back. So I suggest she do a lousy presentation so we find someone else next time.
As you recall she is the director of NAC Center for evidence and practice improvement. Which has five divisions. I will not read them to you in the interest of time. Does -- I think everyone remembers Arlene, and if not, I think you have information about her fantastic accomplishment and how lucky we are that we have her at NAC -- AHRQ. And you are going to talk to us about evidence and synthesis.
I am really looking for the feedback from NAC next month I will have been at AHRQ for one year. I just wanted to give you feedback about how helpful feedback from the NAC has been for us. Andy told you a little about the work we are doing in patient reported outcomes. A lot of that emerge from our first discussion on each IT -- HIT are at tutus in that area are growing. Activity -- and IA and NCI are involved with our next project and we are excited and love that came from advice from the NAC as well as our primary care work . It is really moving forward and a lot of the information I got last time has been incredibly helpful to us.
I want to acknowledge Stephanie Tanga -- - Chang she is a co-author of an article in the MJ on updates.
I will start by giving you a little history on the EPC program. It was established in 1997. In 2003 under the MMA it was expanded to the effective healthcare program in 2009 there was a large institution of funds of evidence synthesis under ERA. Where we are now, we actually have two streams of support, appear for the funding. It is fun for the PCOR, evidence is the first of and getting evidence into practice. And increasingly, we actually get a lot of requests to do systematic review by other parts of the federal government. For example, we get quite a few requests from NIH, CDC, DNS, and network is growing. What is missing from there is we actually had a funding stream with appropriate funds and one and he says we are not doing that anymore, that is what was cut this year. I think that was a huge cut for a couple of reasons. We used those funds -- we had a budget if we got a request for Congress that's how the telehealth report was funded losing that capacity, that funding was used to support a lot of message work. As well as -- evaluation, we were hoping this year to do an evaluation of the EPC to get more information on our impact and what we could do better. We have lost that capacity to do so. Even though it was not a large amount appropriated funding, the loss of that has been really challenging for the program.
What happened to the flights? Okay. [ Laughter ]
The principles of the EPC program is that it is stakeholder driven so we get topic nominations widely from the public. Office from professional societies. Scientifically rigorous, I think we are really cutting edge around methods run evidence and I will talk about that later, all our products are peer-reviewed. In terms of evidence syntheses, we really work hard to be independent and unbiased and 11 topics of interest in the reviews.
In terms of the continuum, there is evidence generation and as I mentioned before, evidence synthesis is really the pipeline for getting high quality credible evidence, then to stakeholders for implementation and dissemination. It's all part of the continuum of work we do here with a goal of better health care decisions and ultimately better healthcare outcomes.
I will try to be brief so we have time for discussion. This is a snapshot of our work and this is what we have done since 2009 in the programs I think our products are full systematic reviews, technical briefs which is Mike rapid review or evidence, it looks more perfectly at the literature. We do a lot of methods work. We do technology assessments and reports for CMS and you can see we have had a total since 2009 of 335 products. We welcome your input on how to make those products better and how to disseminate them more widely.
We really partnered with stakeholders in terms of what's with his -- once we synthesize the evidence, and we know evidence is enough to make any decision and it has to be influenced by body's preferences and resources. We see AHRQ role as globalizing evidence and our partners are localizing evidence in terms of implementing it within the context.
We have a large number of stakeholders and this is an area where we would like your input is our audience? Who should we be targeting? I don't think I knew we -- I don't think I need to read the list to you. It is a huge diverse audience that we aim to target. Should we be targeting all of these? Who should we prioritize?
And the stakeholders have multiple points of engagement in the process. The topic generation development on our website, people can nominate topics and we get them from really diverse groups. Topic refinement, we get a nomination of a topic for sample from a professional society, we work at them to refine a topic and make sure we are asking the question in a way that is useful to them. People per my comments in terms of review, peer review, dissemination, and getting out to stakeholders. There is a large group of stakeholders for the EPC program. This is just an example of some of our current nonfederal partners. Actually have a handout -- we actually have a handout that will give you more details on reports and who are partners are. But for sample the American Academy of Child and adolescent psychiatry, we have done reviews and [ Indiscernible ] you can see the list, we work with ACP, we work with a wide range of professional societies we partner with for the reviews. We are having a growing number of federal partners. The values and our process and the rigor of our process for example, I think one good example is an IA -- NIA -- this is a partnership on dementia. We are doing a systematic review on what we know about preventing dementia. They have it oh I am committed to a report on that were doing the background reviews for that we are doing the telehealth report for Congress and were doing a set of reviews here for NIH and upcoming guidelines for CDC, there is -- we have increasing activity in this area and if you look here, you can see the number of interagency agreements that we have with other federal agencies has been growing fairly rapidly. In terms of evaluation, the use and value of our work, we have something called an impact database where we try to capture -- when we do traditional metrics we cite the number publications, number of guidelines that have developed from it, but it is really hard to capture the impact. Dislike does give you an example of the range of uses for our -- this flight does give you an example of the rage for our work is to clinical guidelines by Kaiser. The US preventive services task force, that EPC does the work to perform the liberation since 2000 955 clinical guidelines have been based on these reports. 13 government conferences that we supported. MEDCAC Meetings, government policies, the national business group on health is used to report that we are trying to look for ways to disseminate it and when Medicare recently released its dashboard, I looked at it and said we have evidence reviews on a lot of the drugs. We put together a table of links so that people who wanted to get more information on the effectiveness of the drugs, we had the EPC report ready.
In addition to doing the reviews we do a lot of work on the message. We have a method site for comparative effectiveness reviews and some of the current chapters that we have done and work includes integrating nonrandomized studies, integrating systematic reviews, updating reviews, and we have also done a lot of work around how do we synthesize evidence for complex interventions and hosted a series of meetings. We have worked with international -- PRISMAthere is a extension and there will be a supplement in the Journal of clinical epidemiology with a series of papers on issues related to synthesizing complex interventions.
We have a systematic data review, and this is a resource you may want to be aware of, it is available on our website. It has all of the tables from all of our reviews. So if someone wanted to go in and update a review or the papers or get was -- get what was extracted, it is all there and publicly available.
I am going to go back.
And the last thing, we lost with funding -- we were able to do special things with appropriated dollars, we funded a lot of method grants this year because those were cut we actually put into the pool with all of the investor -- investigator initiated research. And without a special emphasis notice we are in the process of renewing that. We got outstanding applications, and basically, over this year we will be able to fund about seven with scores under 17. These were very high-scoring grants and the cover everything from dealing with heterogeneity and network meta-analysis and I think important work. These are grants with a small investment that really can advance the field.
The other thing we do is we are thinking about ways to him that's innovate, we have a pilot -- with pilot of updating reviews. We picked three topics, Venus Tom reliabilism prophylaxis, and we're doing this with a regular update. The updates will be done about every year and a half or so. They are in the process now and we will see what we learn about updating, this is something we should be doing more of. It is a question of how do we do that?
One thing I wanted to focus on, that is the background and now I want to switch to going forward, where we should be heading. We know it is a changing environment both in terms of the need for evidence as well as other people -- we are not the only players to systematic reviews there are evolving needs and evolving questions and through earlier in -- conversation around them limitation it is traditionally looking at RCTs and looking at discrete information. Does this drug work? Does this procedure were? Now we are looking at interventions and complex models of care, multicomponent in -- information. For the question is a lot different than it is not does it work, but what works for whom and under what circumstances. And more and more this is the space AHRQ wants to play and see how it works. How do we get that evidence? The traditional systematic reviews, basically often when you look at complex interventions you find out they are heterogeneous, and there is insufficient evidence, but meta-analysis is used for homogeneity, but I think [ Indiscernible ] what methods do we need in the future to get better evidence of these types of intervention
Okay. [ Laughter ]
We know there are different types of research designs that are needed to basically study complex interventions, improvement implementations, and what is the implications of different trial designs and methods and more and more mixed methods on doing systematic reviews.
Basically, this is an summary of what we feel is overall currently of the EPC role. It is to synthesize evidence for decisions that were trying to increase our work for other federal partners. Increase uptake and use of evidence. How do we get these reports to the more useful? Develop methods and tools for evidence synthesis, which is a big role of what we do? And also more and more, how do we improve future evidence for healthcare decisions? With the US preventive services task force we now have actually a quarterly meeting with NIH where we present all of our effort -- recommendations and that has led to some funding in some areas such as chronic kidney disease the EPC program is now meeting with people across NIH to say what we come up with insufficient evidence can we develop a similar process to handle these areas that need to be considered for further research.
The questions for discussion today and given the environment or more more people are doing systematic reviews, what should we be doing as AHRQ , how can we prioritize shrieking, and how to maximize the impact of these reports to our partners.
Thank you, Arlington I want to provide a little more, a small amount of additional context just to make it really tangible for you to think about. What I think is so special in what Arlene just highlighted, AHRQ kind of invented the methodologies here and has really become the gold standard for how to do this. I think it is a recognized product line, if you will, with agency that has been very important for its identity. As Arlene referenced, it is becoming a more crowded space as others are speaking about this. I want to share with you, I have engaged with Doctor Selby at the quarry several times in my role here, he has shared with me that they feel a need at the quarry for evidence-based reviews to support their work. He has raised the possibility of doing that work via AHRQ but it I has also been raised as a possibility that it could be done in parallel to AHRQ or as a separate activity. I think one of the things that brings up is a question of strategic long-term strategies given some of the other concerns I race for you earlier around aligning our work with others in the field, and identifying unique niches. How do we think about navigating where we are in this work and aligning in a successful way that allows the very important work to continue but also is in the best interest of the agency given some of the concerns that are raised about potential for being labeled as redundant or an efficient because of how work is aligned. I want to give you that additional context, I certainly can address other questions related to that. But I think that is part of why we are particularly interested in gathering some of you on this, because we are feeling some sense of our all those being pushed a little more against our body. It is great that people want evidence reviews. It's an important part we think of how work moves forward. But what impact does that have about the agency and unique role it plays in the space.
I had a comment and a question. I suppose my initial comment just thinking about your proposition of AHRQ the -- being faced with systematic reviews for the quarry to me that seems like a win-win. It allows us to show we are collaborating, reducing redundancy, but we're still making a meaningful contribution I can imagine there are some downsides to that in funding and infrastructure that accompanies that arrangement and I think that would have to be thought through. But my initial response is that it seems like a good idea. One thing I wanted to have unpacked for me a little bit if possible is on what appeared to be 2 different narratives in the presentation, on one hand we have lots of ways in which we have looked at evidence over the years. Showed a little grass -- you showed a little grass -- graph and it showed the number of agencies for AHRQ to be able graphto part of the at the same time on the first --
These have been declining.
I should have explained that it was sort of -- we had that infusion of funds from ERA. Basically we have been steady and then we were able to do a large number more with ERA funding that's why those years are higher.& It.'s open nevermind. [ Laughter ]
Two thoughts popped into my head. On your slide on stakeholder engagement, federal and nonfederal, I don't see anything about how EPC reports are the process might engage developers and entrepreneurs and innovators and medical device or drug for unmet needs. When I consult and talk to a particular medical device companies, they seem explicitly unaware of the role of evidence and evidence work that has been done to date, yet they are the ones who are trying to influence of care delivery also have an impact on cost. I am wondering -- there are mechanisms, do you have any connections with Pharma or the actual innovators themselves?
We actually do. Elyse [ Name Indiscernible ] from our center works very closely with the FDA and other federal agencies in those areas, that is part of our technology assessment work. You are absolutely right, what the huge challenges is the quality of the studies we can never get the evidence. We are very interested, I could have gone in a lot of different directions, but what is our role in improving the evidence that is up front? We have had discussions with FDA that we would be happy early on to talk to device manufacturers as they design their studies providing them some typical assistance to tell them what kind of evidence actually be acceptable for making decisions that is a whole other conversation but I welcome the question of do have a role there?
On your current federal partners like, you do not actually list the FDA
Those are for systematic reviews they have.commission a systematic review from a but yes we do partner with many other federal agencies. I should say there is the EPC programs, and then there is the EPC division which is actually much larger and has a lot of other moving pieces I narrowed my discussion just to the EPC evidence program, which is the systematic reviews itself.
So I like the notion of collaboration as you have, the demand you said is going up and it seems with the evolution of the healthcare system and the attention to the things that they have to care about and it will be increasing demand. The centers that you have built are a bit fragile because of the funding being dried up and it would be a loss to lose them. In thinking about the heart of what AHRQ should retain , certainly you have built the teams. You have established methods. You continue to advance methods. And you can teach and bring others to the space so it would seem like you could carve out for a modest price which is the price that you can probably afford a key role for how AHRQ can continue to lead in the space while drawing funds from other partners to do targeted content oriented work. I guess that's where I would see this, while being careful not to lose the teams that you have built up who have those skills and capabilities.
Mary.
Thank you this is terrific. Just a couple things. All of which has been set, but just reframing it. AHRQ is the gold standard I cannot imagine any meaning that I have been to that has not referred, used evidence-based reviews, etc. I guess the question of being a gold standard is how do you build on it? And what are the railways in with you stay ahead of the curve? I think your focus on complex interventions and how you synthesize evidence related to complex interventions is critically important future direction. Been thinking about how you might use some of the test cases, the data you have available and the comparative health systems is collecting data around complex interventions, Dartmouth and ran and all of those places using that as test cases to show the rest of the world how to do this. I think constantly thinking about the notion of integrating nonrandomized clinical trials data into your reviews, or thinking about how data or findings from one systematic review connects to an other even the issue of updating. People think that is extraordinarily simple. It is unbelievably challenging. ~Methodology that you can show, Timmy it is the real challenge is I had building an agenda related to that. I also think the way that you have the data available to everybody, now you have it in tables. The question is can you make it accessible in different ways that will allow people to really build other research intensive environments to really build on -- to do the kind of updates from your actual data. I don't know if that is possible or not.
When you say, make the tables more accessible, are you say treatment in a relational database? [ Indiscernible - multiple speakers ]
Making this organic so that you have a real opportunity, not just to be able to put tables, which are quite meaningful, but literally to build on and extends them.
I think that is a great point I should say, separately our center, we are thinking about integrated digital strategy for the center and if we could get resources would love to link all of our resources which are very separate. For example, where the national guideline clearinghouse in a national quality measures clearinghouse, could you imagine if you click on a guideline, get the measure report, actually also have something that has the E measure standards in it, you could go back to the EPC report and get tools we are thinking about ways to electronically integrate all of our resources and also make them more accessible to users. If you have thoughts of what we could do to actually make that helpful, being that is a fabulous idea.
Christina.
I have actually a more basic question when I look at the continuous updates for a five-year cycle, and I look at the topics, I guess I think of what is relevant or is what -- what is a critical need from the population health perspective. And to build off of Mary's comment, we look at the data do ever take a step -- I know resources are limited -- do take a step to focus on updating something that may not be within that five-year cycle that may need to be updated just because more of the evidence has come up that is conflicting in literature or may need to be addressed because it is more of a population health issue I am curious about that.
Stephanie might want to adherence and she just wrote the letter. There are a lot of questions. What do update? When do you updated? How do you update it? Stephanie, why don't you jump in.
On the graph, Arlene talks about the top part that identifies links that we continuously update. Whensoever process to identify reports that we continue under surveillance. We assess -- there are other updates through the process. [ Techical issue with audio ]
I have been trying to figure out how to articulate all of this. I will give you a problem. [ Indiscernible ] has been ended by commissions to put together a playbook, I have written a satirical essay on what is a playbook it has become popular it is supposed to help in version 1 leaders of ACO's, other APM organizations can't deal with patients who have, healthcare needs and have to use a lot of resources and cost a lot. And figuring out what is the evidence that will help us there, it is very difficult this morning was talking about great criteria and conquering criteria which do not begin to cover the kinds of evidence narratives noted as bright spots that exist. And how to include them and heterogeneity is only part of the problem, the evaluations themselves are so generally so pathetic even from a qualitative or next methods of you that you don't know how to categorize them. That's the problem of usability of the kind of evidence we need to afford I was thinking what with a patient what? What with a provider what? What we really want? Because ultimately the playbook is supposed to go down to all stakeholders not just to the leaders. Beth said something this morning about disparity subcommittees that was spot on. What is the data visualization that you really need? Systematic reviews including the really spur produce here do not even begin to use modern data visualization techniques to compel any of those people to pay attention. One thing you can do, I recommend this exercise, take the Obama JAMA piece what could be more important than the evaluation of the ACA. Is actually good for data displays, if there's not a lot of room, but even so almost under those it could be we explained to people care without big legends and context there is a whole area, AHRQ is really good at translating the toolkits, I am constantly impressed but I would take that to the next level and think it through the eyes of the patient, provider, community or whatever. Of course your hands are tied behind your back, because if you are in England you would have an ICD -- you would have NICE.
There are two things I want to say to that. I would love your ideas about how we could tackle letter I actually had a paper, I was an academic and I had a paper from old work that was published this month in medical care research and review which was actually a scoping study of interventions in the community to improve heart failure care. And basically all of the problems you say, for so long with vaginas for 20 years and have all these positive trials, the needle has not budged in the general community as far as care for the patients. But when you look at the literature it is the wrong population. None of them dealt with transition to end-of-life, audible morbidity, ovulation stratification's were problems with the study design use of quality of the study, reporting of the studies. There are new challenges and how you synthesize the evidence in a meaningful way. I think that is a whole area where we can do a lot of good I would welcome -- I don't know where we take it -- but I think we could really lead in that area. I do not necessarily have the answers, but I agree with you here it is a huge challenge.
In terms of the reports, I will be honest, if you have insomnia you could read our EPC reports. [ Laughter ] They are great and rigorous and they are incredibly dense difficult to read. And we did a healthcare report in response to a request from the Senate and Congress are interested in telehealth. The draft was released got tons of criticism. I read it, and the information they wanted was there. But unless you're going to spend hours digging for it, it could not find it. Its presentation, we went back and did a lot of work on it to make the message clear, the language more clear this there is a disagreement among the EPC , we put in a box like the NJ, what is the bottom line and what did we find, then when it was released people loved it and we cannot change any of the findings, it was just presentation. Self thoughts of -- should we do things like that? How can we present reports that are more accessible to users?
In response to that, the single best thing, that JAMA has done, is put in a text box that explains the significance of the study in language anyone can understand. And that doesn't necessarily -- or it should not dumb down the content. I could never explain to my mother when it epidemiologist is. I have to be able to tell you what that is in three minutes and now I can't, but I won't. [ Laughter ] But that is a brilliant way to engage people
I have Kevin in Sandy and then I will go to the folks on the phone, is to let you know I am not leaving you out Kevin.
I agree with what has been said. I think it is just trying to fit into the package. Arlene, I have heard presentation needs to be improved how to make this user-friendly. Second because the marketing and the communication strategies, which I think like you do with the CMS, I do that's the other challenge for AHRQ and it goes back to what you are saying, Andy. How do you get the stuff so that it is their right in front of people when they needed. I see them here for -- what is the latest stuff on the medications. Are the new it's better than the old ones? You would want that link for anything. Is like Everett there's a commercial on television, you want to have a little dashed or on social media, you would want every time one of these is mentioned. You want it to be right there when someone is sitting and thinking what is the right treatment for something as basic as -- [ Techical issue with audio ].
The final area I heard in this discussion was that Mary was talking about, traditional evidence-based stuff, it was about internal validity. It was the drug trial, classic wife, tidy intervention, very discreet outcome. It's the world of complex behavioral intervention, help system intervention, care management transitions. It is messy. And they never meet the internal validity standards. To me that is not what I need to take action on. I think that's probably the methodology. How do you then actually think about what are the criteria for actionable evidence particularly with things that are not necessarily concern about harm like beings with drugs and things like that. Should have a different way that we set the bar on this? What is actionable information? Right spots? I think that's the methodology I think it is probably presentation, how to market and get linked and out there. And where can you make and roads where there is a need for some route to come out and say these are the new methods became about evidence and synthesizing evidence that is not just the old discreet clinical trial type of stuff.
Just from AHRQ, our evidence now, we are basically testing practice facilitation and studying capacity to change with an independent evaluation and a mixed method evaluation. Hopefully some of the work we do on the research side can support what message you need to actually get that kind of evidence.
Sandy.
I have four thoughts. On the second to the last flight has the multipronged approach, help me organize my thoughts. I start at the bottom. You have healthcare decision information, I think that is the right thing and I cannot think of anything better when AHRQ finds issues that need to be addressed, to talk on a regular basis with funding agencies. The only thing that may be expanded there is to include, to the degree there is any funding from other sources like AHA or Johnson foundation or whatever beyond the federal government, the other things sort of -- looking at the tools from the bottom. I think there are two problems we face where Tremont can make a difference. I've had the opportunity to be on a preventative task force and then on the EPC. I appreciate the quality of work two things strike me. Number one is we need to learn better how to deal with not experimental data. For some of the things he talk about, you'll never have really rigorous randomized trials. And even when we have randomized trials, it always struck me on a task force that we had a big methodology manual and we talked about how you see all the data. But if it's randomized trial because of time and money and effort we ignored everything else. And in terms of assessing efficacy in the class extension and safety, or at least efficacy, that is okay. When it comes to safety it is never okay. When we came out with what I always refer to as the infamous breast cancer recommendation, where that ill-fated -- even then to try to get some assessment of risk and safety had to use registries and observational techniques because random test are two broad.
AHRQ should be the group to figure out -- and learn more about what it can be used with the caveats. I think that has a lot to do with the generous uptake. Regarding uptake I think it's important to sit down when she -- shareholders. I think the communication issues, going back to what Andy discussed earlier, there are some things us dealt with on an agencywide level. Communication is one of them. It is really important to figure out how to communicate what is done more effectively in everything that we do. And it should be consistent. And I don't mean rigid consistent, but if you are used to reading one type of AHRQ report, it would help to look at the other. I think it would help AHRQ to figure out the dissemination communication issue more effectively by looking across its products and figuring out how is that all it needs to be individualized or modified. You doing that in the research group doing that, your remarks at the end alluded to the fact -- that's one thing I would look at at a broad agency level. And with that, think about why things have not been adopted more and then sit down with the appropriate stakeholders and ask them why. I do not know what the answers will be, I have ideas about something's. But whenever I have done that whether it is sitting down and talking to patients about how they look at diseases, talking to other physicians or nurses, or talking to healthcare system leaders, I always learn a lot of things, some of which I should have known and others I would never have guessed. You need to find impact and decide on decision-makers and find out -- push them to tell you what would make a difference to them. I really like the idea of making it -- not the report readable. At Hughes came up with a standards that most clinical trials use. It is for basic questions. Is it new? Is it true? How does it affect policy and/or per -- practice.
I was talking to others and they say they still use that when they look at it. Anyone was ascendant Article 2 JAMA, [ Laughter ], I try explicitly not to pick I start out with what's new about this article what's important about this article, this is the impact on policy and practice. I think if we can answer those questions then maybe target -- slightly targeted to what the CEO of health system needs to know about what to patients need to know, what do payers need to know. Tie it with the website. Don't be a huge undertaking but I think it would take the dividends.
Let me check in and see if anyone on the phone from NAC wishes to weigh in on this topic.
I have a couple comments.
Go-ahead José.
It seems to me that one way to look at what I could do -- some of my comments may have been mentioned already -- to transition from a producer of these evidence-based reviews to an organization that can influence the way they are done and ensure they are done well. I think that includes at a minimum of three components. Number one would be to continue to work hard on developing methods and people have talked already about several areas where the development continues to be improvement one is the use of observational data, increasingly how do you assess complex intervention or system set interventions? Behavior interventions. Things like that that are increasingly complicated as opposed to a particular operation or treatment for particular condition. The second, this would be to grow its role and develop it, I'm not sure in what way. But to Groll the role and disseminate and become the place people look to for how to do these things. So other people are doing them, they understand what the standards are. Many years ago, 10 or 15 years ago, her efforts to disseminate standards for cost effectiveness. How do you deal determine what will work and what will not work. How do you do all of these things? There was a great debate in the economic community about how to do that. I think there are some standard guidelines that most economists accept now on how to do this. Event that becomes a playbook or a set of standards. Something like that that does not exist at this point would be a wonderful goal for AHRQ to have people to look for it and do well. The third and I don't know if this is feasible for AHRQ to do , but to build a team of people that deal with the technical issues. Finally to begin to study how to disseminate anymore effective way. Dissemination of himself needs to be studied for the development of guidelines on how to do that. I guess the big picture is transitioning from the producer of these to organization that has symmetry and includes the work of all of the other people who are doing it.
You can go back to the evidence reports that supports it. Would that be a useful type of dissemination?
I assume any form of dissemination is good. I used to work dissemination, I guess what I meant dissemination that leads to pick up. I don't think anyone has done that very well. There is dissemination that is produced [ Techical issue with audio ] that is a different science in its self there -- in itself. Working on that is probably what will work well.
Lucy, Jen, Sherry got any comments?
This is Lucy, I would concur with ever think that has been set. In particular the comments that Don may. The area were going and and and despicable mission accomplished intervention is so incredibly challenging. We look forward to the evidence report coming out on new methods. I also think that the notion of how and where we solicit ideas, if there was some way we could cherry pick one or two professional societies that were grappling with something and were able to produce something useful to them. Then they become champions for AHRQ. And a vital role that AHRQ laced and helping them move the field forward. We create partners. I think that would be very useful.
This is Sherry. I wanted to add, we have been using the evidence reports in our evidence-based benefit design committee was also has a lot of the specialty groups represented and some researchers or research institutions as well as employers. If there is a topic that is valuable to employers, we have been distilling down the research to what possibly could be an appropriate plan design and coverage under self-insured employer programs and why. Autism and [ Indiscernible ] were the two most recent. But also lab and radiology expenses. That has been extremely valuable in the folks around the table can help us set it. Then we disseminate that out to all of our members including health plans and health plan -- the strategic partners that could then consider it for inclusion in their programs as well. It has been valuable and I would love to see us continue to do more and more of that. And we are hoping that we can actually help you get back that data so you can disseminate it the way you want as well.
Thank you Sherry.
Jen, I'm not sure if you are on or have any comments.
Let me make two quick comments and then Andy wants to make a couple comments. I do think the pass off to implementation -- I think of dissemination in the way we talk about, I think the NIH thinks of this, dissemination is the transom, implementation is working through the heart on the ground stuff but it takes it to do something. We have a program that I would love -- both because of the work more because of the acronym which is havoc which is health and value creation which is taking a look at things that look like a great idea and not sort of taken off in our system. In one of the pieces of work that I think has really been quite interesting has to do with cardiac monitoring devices. The interesting thing that that team learned was all the things that have nothing to do with the evidence that got in the way of the uptick, it had to do with lack of connectivity in member homes. This led to new forms of contracting that allowed us to build it into the price of the device as opposed to have to put that burden on the member. It had to do with the workforce being afraid that their jobs would be eliminated if we found ways to keep people from coming into the clinic for monitoring. Those are just two examples where it is not about the knowledge, it is not about whether or not it is cost-effective practice, all the reasons, it's not is it better for patients. I think sometimes that takes a really hard work to do a diagnostic on where the implementation failures are. There may be a whole area of study that helps begin, and this is the way that the teams at Kaiser have been working on it. This is nothing to the look for as opposed to in-depth studies of each and everyone. But to think about it in terms of rollout strategies.
The second item is, I want to sign on with everyone who has endorsed AHRQ important role in pushing methods forward in particular trying to bring different kinds of data into systematic reviews. Because I really think we need more and more technology randomized trials have lots of limits in terms of what do they need for real people in the real world. We absolutely have to get better both at producing real-world evidence and using it in these reviews. I just think it is time to get over the RTC fantasy and move on to how we think about these things on a continual.
The third thing I have been pushing on copper interventions within our own system, because we do lots of small pilots that seem to be tough to add a to say what works and for hair and for whom. The notion is and when it -- meta-analytic framework. At the end of the day what do we want to know? Is there a way for us to frame it so even a small studies have a much higher chance of being brought together for broader learning? We are working on some of those frameworks in places to care patients with multiple chronic conditions. There are just some areas where we know the answer will be in the space of complex interventions. And that we are going to be dealing with messy evidence but there are things like a we use a few of the same measures when we are looking at those programs? Could we identify who got the programs and when? There are some things that would make use of that evidence a lot more likely and possible I think some of that work to be done upfront. And I loved what you got out of the RO three, the example of ways to small investigator initiated work but directed to help the puzzle pieces together in a more effective way. Is where my comments. Andy?
I want to thank everyone, really valuable comments that we have received in this discussion. I want to first key of a little this afternoon a parallel to the discussion there particularly in the light of some comments I was a mate if you minutes ago. I think one of the things we're trying to think about our work here at AHRQ are the different phases of production in a sense. I think José referenced the fact, is it time for AHRQ to think about pivoting from doing the evidence-based reviews to kind of making sure they are used or applied. As an example, I think we'll talk about quality measurement this afternoon, you'll see some parallels about whether or not we are developing quality measures, are we maintaining them in some way, we will show you the different steps of production. Again, I want you to think about that in terms of giving us some strategic direction about prioritization. Because I think you have generated some really valuable ideas of new things that we can to. But what I did not horse upon you, but what is obvious from the first discussion, we have constrained resources and we have to think about, therefore, where is our best place to make those kinds of investments? Want to at least get out on the table some parameters that I think about with regard to that, but also welcome your input around that. And that is, should I be thinking about, should AHRQ be thinking about, should we doing it? Are there others that could be? Therefore we don't have to do it anymore. Is that one criteria we should use? Another idea, I think this has become really palpable for me, and the things we choose to do, is there going to be a way we can demonstrate impact in a way that allows us to get the kind of credit or recognition that may be important for sustaining the effort overtime? That is another one. I think you have all the really good at identifying questions that come up for the things that we need to get answered. I fully agree with that I think part of the challenge is for us to think about given the constraints we will have and the ability to measure impact in some ways, does that influence how you would guide us in a long-term way toward which places to put more of our investment in? People have noticed that AHRQ is an unbelievable infrastructure organization of sorts. It is hard to celebrate infrastructure sometimes. That does not mean it is not important and we should not be doing it, I frankly value infrastructure tremendously but we need more best thinking about how to think about the right parts of the production for us to really prioritize and to make sure that you at least help us think about if we are going to be doing it, and were providing the public good, is what we are here for, how can we expect try to measure that effect in some ways? Developing the specific evidence reviews like the ones Arlene showed here, they are very tangible you can save there is an answer about what to do with prostate hypertrophy, you can say AHRQ made that they. On the other have some on the other methods which may have ultimately a greater impact if we could measure it, might be a better thing to do. But that's where I particularly would value about. And that will come up again this afternoon to talk about the quality measurement. Ideally, of course we would do it all we love all the aspects of it. We are quite capable at them. But we are going to be constrained in some ways. I think that is a big part of it for us. Thinking about others can do it and how that also should shape our decision-making is part of what I am trying to think about as well.
So Arlene, you did such a nice job again that you may be asked back again. [ Laughter ]
Here is the game plan. Photo, food, financing. Photos first, everybody gather outside of the atrium to get your picture taken. Immediately to grab food, bring it back here and Joel will talk to us about -- I had to use an alliteration -- boa talk about the 2015 insurance component financing. We need to be back here by 12:15 PM.
[ The event is on lunch recess. The session will reconvene at 12:15pm Eastern Standard Time. Captioner on stand by ]
So we are going to get started again with the lunchtime entertainment. I am pleased to introduce and actually for many of you read introduce you to Joel Cohen, Ph.D. director of the Center for financing access and cost trends. His unit both overseas some of the extramural research, but also his group does quite a bit of intramural research related to the use of the MEPS. he is going to give us a latebreaking update on the private sector national table. Without further ado, take it away.
Thank you. I think this is the second time I have done the lunchtime presentation here. I guess MEPS pairs well with a turkey sandwich or sushi. I like to eat myself. What can you do. Hopefully, this will not disturb your digestion at all. It is actually 30 good news.
As Beth said, we recently released data from the MEPS insurance component. The system actually several different surveys which -- one is a household survey. This is the component that is an employer survey. Is an annual, private and state and local government. It is done by the Census Bureau. The data are Census Bureau confidential, which means that the micro data are very hard to get a hold of. So what we do is copy's early, put out a set of tables on our website. These data are used by a lot of people run the government. They are used in the computation of the gross to -- Gross Domestic Product. They are used by the national health account people when they are putting the information together. It is really the largest and best indicator of what is going on with health insurance in the employer sponsored market. The employer sponsored market is really important because in fact most people in the United States at their health insurance through their employers. People are interested in what is going on there. Recently it has been of even more interest because with health care reforms that have come along recently, or is a lack of concern about what the impact of those reforms would be on the employer sponsored market. For example, with the institution of the marketplaces, does that mean employers. Offering insurance -- does that mean employers will not offer insurance and send people to the market. There were mandates --
[ Indiscernible speaker away from audio source ]
Sorry. So anyway there is a lot of interest in what is going on. What we released, or what the data I am going to present right now, we released both national and state and Metro areas. I am going to talk about national estimates. Disarray really collects information as it says here on premiums, contributions, eligibility, etc. You can see whether and players are still offering insurance, whether they are changing eligibility requirements for their employees, how much the premiums are for the insurance that they offer, how much the employer pays, how much the employee pays, etc. Then we collect a lot of characteristics of the employer's so you can to cross cuts along different dimensions of the employer sponsored market to see what is going on.
Last week, we released our data on the website and concurrently we did a statistical brief. I would like to acknowledge the authors of that brief, who are not here today, Jessica [ Name Indiscernible ], Ed Miller and partition Keenan wrote the step brief and all the figures I will show you in a minute are from that staff brief. We are looking at national trends and we are looking at trends in policy relevant firm size categories but basically what I will show you is firm size categories but we actually have a lot of different information about the firm's, the type of industry, wages within the firm etc. I and this would look at firms size and a policy relevant categories because they are geared to both concerns that policymakers have and what the mandates are in the affordable care act for employers to offer insurance. There is a penalty if they do not offer insurance. Does were delayed for one year but the ones for the 100+ employees kicked in last year. This year it will be extended to 50 and greater. Those are the cuts we will look at.
Later in August will actually put out an insurance chart book we did this Lester as well. We had an initial release of a stat reef -- brief
To get into the data, this is the enrollment rate. Some of these estimates are going to be conditional estimates, for example, if someone is enrolled they can only be enrolled if there firm offers insurance. Enrollment would be looked at conditional on offering. This is an overall rate. Where it is conditional, at the bottom it says the nominator within each category, all employees in the establishment. If it is a conditional estimates, it will be noted at the bottom. Here we are looking at the enrollment rate across the United States. It shows the different size categories. Basically to paraphrase chicken Little, the sky is not falling. [ Laughter ] There is basically very little change in the trend in enrollment for all private sector employees. Basically, you can see in 2008, if you look at the US figure it is about 54% and by 2015 is down to 48%. But that has been a trend that has been occurring across time and there is nothing different there that is happening. In fact the change between money 14 and 2015 nationally and or the large employer's and the medium-sized employers is actually not significant.
Small employers there actually is a significant decline it went from 28.3% to 27.1%, but again that is a trend that has been going on for a long time it is just an extension of that trend. The other thing to note, in terms of just the percentage, about half of employees are enrolled in health insurance across the United States.'s
[ Captioners Transitioning ]They had an increase in both --
This is enrollments. Divided by the number of employees. If both members of a couple both are eligible and enrolled in insurance they will be counted twice.
[Indiscernible - low volume] This is for the employer's point of view.
The survey is of employers. I think what you're getting at. Let's say I am the employee. I turned down my employees offer of coverage because I am enrolled in my spouses plan. I would show up is not enrolled in my employers.
That is right. It might actually slightly undercount the percent of employed people in the United States.
Absolutely.
We're seeing it declined because people are electing their spouses coverage.
If they both purchase a policy -- that has been happening. That is occurring over time. That could account for some of this trend.
Who they did you have a question?
I only wanted to say I cannot hear the questions being asked from the group.
Yes because they were not using their microphone.
I will be good and hit by button.
As you said most of those are trends increases. I think given 2008 being when the affordable care act was passed it would be useful -- 2010. Okay -- never mind. You can go back further.
Usually I like shorter periods because they are easier to look at. In this case given the political context it would be helpful to go back a couple of years and maybe even draw a line to shape the area where the affordable care act is being phased in. Reduce the chance the data could be misused or misinterpreted.
There is one wrinkle their -- we actually switched how we did the survey. In 2007 we went from a retrospective to a currents. There is no 2007 data. But we can go back to 1996 --
It just makes it clear that that is why you're using -- the question I have here is a is there any measure of a structure of the plan anywhere in MEPS? In order to enhance the -- what we're learning from this, as we see no error networks it will help us interpret what is going on and other spending. Obviously that is extremely hard to get anyhow. But I wondered if that is in their or if there might be a way to try to incorporate that in future years. I think we will be seeing continuing narrowing.
There are some characteristics of the plan that we collect. The network is not one of them. That is very difficult to figure out. We have -- one of the issues here is a plan is not necessary a plan. There are different generosity of plans etc. When you're looking at cost, one might be more generous or less generous plan. That is hard to get at. We tried to get some actuarial values. Employers are not very good -- even though they are supposed to report them they are not good at telling what the actuarial values of the plan is. The actuarial value won't tell you about the narrow network either because it is based on what is covered under the plan. But you are absolutely right those are really important issues. I think as we move forward we need to try to figure out ways to measure those. Incorporate those.
Hello this is Cherie on the phone. I can add though that large employers know exactly how many members they have on a family plan. And they definitely measure actuarial value. And they do a benchmarking. If there is data that we could help with the because that is right in our -- large employers for sure. We certainly have plenty of information on that parts.
That is greats. We will be contacting you. We just added this actuary value question to the survey we're getting the data back now and trying to figure out if it makes sense. And how to edit it etc. That would be extremely useful -- thank you.
Large employers don't have insurance -- itself insurance so they have to come with COBRA rates and actuary values for everything. That they can be compliant on the outside.
That is great -- I appreciate that. The next table is the number of private sector employees enrolled. Again, this has been -- there is some drift down and back up -- this has been relatively stable over time. I think the thing that might be interesting thing in 2015, there are 57 million employees enrolled. Most of those are actually from large employers. Even though large employers are a smaller number of employers -- the actually cover most of the people. Most people work for larger employers even though most employers are not large employers. In terms of offer rates, actually here, if you look in 2015 you will see that for large employers they are really close to 100% offer rates. Actually even though it is close to 100% it did increase from 2014 two 2015. 290 a point a. That is statistically significant. The large employers are the ones who are subject to the mandate that year. So there is an increase in there is potentially some impact. That is actually a good sign. We will see what happens with the medium-sized employers next year with the mandate kicks and. For small employers again the offer rates have been declining and they continue to decline. Over time. Between 2014 and 2015. In terms of eligibility rates, again this is conditional. Within the employer, what proportion of their employees are eligible for insurance. They offer it. There is not much going on here. For the large employers, there was actually some declined between 2013 and 2014. And that has been maintained. So there is some decrease in eligibility but it is relatively consistent over time. It is about 76% of employees.
Take up rates are on average 75%. There have been relatively consistent -- they are higher at large employers than small employers. But again, there was some small bump up in the large employers between 2013 and 2014 and it came back down to the 2013 level in 2015. That is really pretty stable over time. This looks at premiums and the percentage change. So growth and the cost of the premiums over time. We are looking at single employee +1 and family plans. Employee +1 plans have become more popular over time. Including them -- the available of them has some impact on the family premiums which we will see evidence of later. Basically if you look at the growth and premium costs over time you will see between 2009 and 2011, they were much higher than they have been since then. Between 2014 and 2015 there is a decline from four points 7% to two point to 2.2%. Even though the number of people enrolled is maintaining and increasing slightly, the cost of insurance has not been growing. As fast as it has in the past. Continues at a moderately level of growth.
This looks at single premiums. You can see between 2008 and 2015, there is gradual growth in the cost of those premiums. So 2015 on average for the US about $6000. There is not much difference between small and large employers in the cost of those premiums.
This was employee +1 -- again, the cost of the premium here is about double what the cost of the single premium is. It is about $12,000. The single was about $6000. And again, a small and large employers are very consistent here -- the medium are slightly lower. There is growth in these premiums, but it is pretty consistent. This is the family premium and here, there is some difference between the small and large employers. The family premium is about three times the single premium -- about $18,000 a year. As I said, the existence of the employee +1 might account for some of what is going on here. The small employers are much less likely to offer employee is one plans. If you do offer an employee +1 plan, that will take out some of the smaller families from the family plan. It will increase the premium for what is left. That might be what is going on here, since the smaller ones are less likely to offer that. Their average premiums are lower for their family plans to the larger employees.
These are the employee contributions to those plans. This is for the single premium -- again we are growing over time. They are about $1300 in 2015. They tend to be slightly over 20% of the total premium for the single plan. Again, the small employers have a little bit lower contribution they are. And that might be related to the fact that smaller employers -- if you have a small pool and you want to get everyone in that you can. You don't want to discourage people from signing up for your plan. You might want to keep your contribution a little bit lower. This is for the employee +1 coverage. These tend to be a little over 25% of the total premium. Again, there is some growth over time. Not much difference between the different kinds of employers in these premiums and the growth over time. This is the family premium, same story here. Again, about a little over 25% of the total premium and about $4700 on average. And again they are pretty closely coordinated across -- tightly distributed across the different types of employers.
This is the percentage of private sector plans that have a deductible. If you are trying to contain costs one way to do that is to try to shift some of those cost to the employee so that they will be more judicious in the kind of -- amount of health care they get. One way to do that is if you don't have a deductible you can add one to your plan. If you look in 2008, you will see that in fact all of the different sizes of employers have about 70% had a plan with a deductible. If you go over to 2015 you will see it is up to 85%. There has been a fair amount of growth in the proportion of plans with a detectable. Even though they were all very similar in 2008, the small employers are little more in terms of the deductible. Compared with the larger employers. That might be related to trying to encourage their employees to sign up for their plan a cause of the smaller pools. So basically I guess the bottom line is that, these data are just out. Is really pretty good news. There isn't increased enrollment and some increases and operates. The proportion of people with employer-sponsored insurance is pretty consistent. The growth rates and premiums in recent years has remained very low.
Thank you.
First of all, I think MEPS is a great example of what AHRQ does well. I have my thoughts. One is branding -- getting back to discussions. We don't get enough credit for the types of things that are done here. I work with a lot of people who use MEPS. Recorders don't know it is a AHRQ product. If I were at a faculty meeting now we will be calling this AHRQ MEPS. Or something. We would have the name right up front. The only argument there would be the president's office wanting [Indiscernible] is to -- in front of Wharton. When it is cited even, if the center for financing access and cost HR Q. I'm thinking maybe as an agency thing, AHRQ should always come first. You have an NIH card and it always says NIH first. Get the name out there. In terms of some of the things -- if there is a way to get at some structural [Indiscernible] given the long lead time you need -- thinking about talking to employers to see about it there is some measure of network. That would be helpful. The other thing I think would be helpful, would be if with this information, there could be a measure also of what is happening to the average employee income or family income. Because, 4%, this is one of the things I'm always focused on -- we have had it decrease in health care costs but the economy has slowed down and incomes have stagnated. Even though on the one hand to economists it might look good, to individuals it may or may not be as good for families. I think the more we can get -- sometimes just putting those on the bottom of the curve can help put into context.
The last thing -- if you can comment on things that you have learned in using MEPS that relate to what Andy was talking about earlier in the initiative about the healthcare -- what was it -- the outcomes measures and systems. Because if MEPS can be considered a model, and he referred to it before -- what can the agency learned from all the knowledge that you and your colleagues have developed in MEPS? Maybe think about how to facilitate the transition of this new program.
In terms of the income -- we do have information on wages and the chart book has a number of charts that look at low-wage, high wage firms. It does get at some of that information. In terms of what we have done with MEPS, we have been around for a long time. The survey started in 1977. With Gail Wolinsky -- it predated the agency. I was here before the agency became an agency. It is been going on a long time. We have fostered a lot of relationships with policymakers. We have a lot of input and we talked to -- most of our input to the policymakers is not necessarily with a congressman but with their staff are with the organizations that actually provide information to the policymaker. The people who do the work -- CBO, CRF, treasury -- Mac pack -- other organizations. So that might provide some kind of a model for doing some of this stuff that Andy was talking about. I think we do try to disseminate as much as possible. Putting the stuff like we do on the web, you can almost everything we do you can find on the web. We spend a lot of time developing analytic files. We have a cadre of intramural researchers who know what you need to do in analysis. Then we design our analytic products to contain the information. We think about who our stakeholders are and what they would need to know in terms of the information we disseminate as well. And like these tables even though we cannot release the micro data publicly, I think there is something like 268 tables in this set that we put out there. We do make available -- you have to go through a process with the Census Bureau to get into a data center, but we help people with that process. And we do encourage them to do that. For the data that does not have to go through there we have our own data center. We're in the process of doing a joint one with NCHS in this building to try to create more access. To federal databases -- confidential data.
I think particularly in this political climate that becomes something that is bipartisan. Access to government permission and things like that. NCHS has already -- always done that well. I just want to congratulate the agency. I think this is a prototypical program that has huge policy and research a systemwide impact.
Thank you
Any questions from the phone?
No thank you
I just would like to go back to our morning comments about roles -- this is a good example. Some of the questions that were asked of Joel really can be addressed by the other portion of MEPS which is interview survey. And by the way, that interview survey is based on another survey which is the HRIS. And then there are other surveys that collect insurance information is a covariates. One could say why don't you just do 1 thing? Well, what he just provided something that we cannot really talk about in the HRIS. What we are able to do is look at health insurance coverage just very important. And also what the health status is of the individuals in that family that we're looking at. If you are really wanting to look at the -- at what is being provided in that health insurance coverage you need to look at it a different way. Frankly, that is the competition of government. First of all our society is extremely complex. And thus the way that we have to measure and work together in government is becoming more complex. I think it is naïve to take the point that another agency is doing this so why do we need AHRQ in this area? I think that is a red herring. You need to have a way to put that down. I think this is a good example of where -- you can't get at one thing with one agency. You need to cooperate and work together and then provide a richer view of a very complex situation with health insurance.
Thank you Joel.
You might have observed we are having some technical difficulties. So you earned yourself an extra -- you know what we could do? We have a little extra time while we're working on trying to get the slides working again. Otherwise I will have to do interpretive dance for my portion. We did not really have a chance to start hearing from people about ideas for research related comparative systems. I don't know if you were mobbed at lunch for all these ideas or there were think people thought about. There is a chance for us to talk about that while we are working on our friend technology.
Mary?
First I thought we were going to get a break. [laughter]
We can all run around and try to warm up.
I am not sure the extent to which the work that is being done is really looking at health more broadly. It is looking at health systems but the extent to which there is attention to the increasing recognition of the partners in health care in the Trinity and the integration of health and social enterprises to meet health needs of people. I was wondering, I think that a really depressing question and wondering the extent to which the existing work is going to help illuminate.
That is a great question. And points. I will say -- this is an extremely complex area as you can imagine. One of the things this might just to give you a sense of the parts we are trying to navigate as we build out to what you are talking about. Even defining a health system is occupied a fair bit of our time. These can be ownership versus rental arrangements if you well. You can have contracts to set up arrangements. Versus when there is actual ownership. There has been a lot of discussion around that aspect of things. Right now the focus is much more on the health system part of it. Having said that, we have also had some preliminary discussion about how do we discussed the markets in which those health systems are placed. That is not exactly what you are saying but it is a way of trying to build out from that to understand some the context in which those health systems work. How many insurance for example are there? Some of the other aspects. I think what you are raising is terrifically valuable and it would be fantastic to go into that direction over time. I think our thought was to try to make sure we have this helps is something right because of the attention on consolidation and those kinds of issues. Really our focus because it is a PCOR funded activity -- how do those systems work with regard to having an awareness of evidence, using evidence, and that impacting important outcomes like quality and value and things like that? I love the idea of what you're talking about, my guess is it is probably in the next iteration. It could be built upon if we do this right with a good foundation. Probably a little too much as a first step at this point.
Just to pick up on that, I think that there is a question about linkages. For the community health needs assessments that were done -- a lot of those draw on data from all sorts of places. To try and set the context for the community in which typically, not-for-profit hospital is operating. My team has been reviewing some of the state that our own system. One of the things I think we lose when we build a new kind of tool, a new piece of infrastructure is the ability to dock up with those things that are out there. Not so much you need to create was then that particular framework, the ability to do that. But really thinking about how what we set this whole system in the context of the community or communities in which it is operating? Will we have the ability to put pieces together in any sensible way to say, something about its footprints inside of a way of describing the footprints of the community and its residents or his economic base or whatever? I think that is where the lost opportunity comes..Ability to make it relatively easy -- even something, I'm not saying this is the solution -- could you link the footprints of the system to other data that are collected at that level? And so like I was talking, and the quality report this meeting about all the work we've been doing with data visualization including a lot of the GIS work. You can use a lot of that from a place perspective to connect the dots across lots of different data sets. As long as you have that -- that becomes the unique identifier that lets you connect the pieces together.
Are we operational?
We're going to try.
Okay -- here is a special guest. You don't recognize her because she changed seats. We were really pleased as we try to think about some of what we thought would be an important topic to raise. We have the benefits of having a chair who is really an expert in quality issues which was a great benefit to us. One of the things I noticed earlier on when I got to AHRQ, what does the Q stand for? I had the opportunity -- AHRQ sponsored a session on quality measurement at the annual meeting of the Academy meeting. That was a part of the panel. He did a terrific job of laying out some of the questions from the field with regard to quality. We can all benefit them hearing from that judgment about that. She is going to do that. Following the presentation we will ask Sharon Arnold to come up and try to give some -- drill down to AHRQ to talk about how we have tried to relate our work and how it does or does not relate to the big picture. That death has been asking about in general. Thank you best for offering to be a presenter at the NAC is at the stage for this next discussion.
Thank you Andy. Is all of you know any chance to repurpose talk is a good opportunity..
I think it is important to start by setting some context. I will say that in the time I've been doing work in this field, going back 30 years ago which really is not that long ago. People did not think measuring quality was important and necessary. I remember as a young researcher trying to convince people the fun work in this area and being told that is not a problem. Why would we spend any money there?
30 years ago that you have set 30 years was not that long?
I thought I would be dead by now. And then 15 years ago people were quite surprised to discover that there were large gaps between the things we're doing. Now we're at a point we are beginning to ask whether the measures that we are using really matter. Are we measuring the right things? Providing the right kind of information? Etiquette it is important to reflect on that. There has been a lot of progress on that. It doesn't mean we cannot do things better. So I will just start by saying this is my definition of what do I mean by measures that matter. They are the following -- there are things that cause the health center -- system to pay attention to performance in key areas. Measurement has served the person this of saying heritage to this. There also measures that are useful to someone for making some type of decision. Those decisions might change with someone originally attended to do. I was a unfortunately a lot of the research in this area focus just on this. The only declare success people do something different. Is also the case somebody might have greater confidence in the decision a were leaning towards were already made. The information from the measure needs to be relevant, the individual who has to make the decision -- it needs to be able to be understood and integrated into an existing decision framework. I'm going to come back to this is something I think we have paid remarkably little attention to. Just the notion that people come to the task with a framework within which they make decisions. The thing we are trying to do is a small part of that framework. I think it is important to acknowledge upfront there is considerable heterogeneity in the target audiences. There are a wide array of decisions for which measurement -- might be useful. This is challenging work actually at the Academy of health. If it were easy we would have to go home and find new careers.
When I thought about the kinds of decisions that we have talked about people using measures for, they range right Whiteley. From selecting and help insurance plan which is probably the easiest thing. Choosing a primary care physician, selecting a specialist either for ongoing care or a single episode -- choosing a hospital or any other kind of facility, nursing home, etc. Either for a planned admission or an urgent admission. Making a decision about where you are going to deliver a baby is different than what is going to happen when you have a heart attack or stroke. Choosing a treatment course. Those can include one time -- now redo possible -- a lot of surgeries. One time where the future choices might not be limited -- I can try this and try something else. Was something that is an ongoing decision where the most important component might be the relationship component.
And then, if that were not enough we're talking about quite different kinds of measures. I have spent most of my time with the standardized clinical measures to talk about insurance guidelines or patient safety clinical outcomes. We talked this morning about the use of patient reported outcomes. Which can vary from general to condition or procedure specific. There is the patient experienced information. Which not only can include the standardized surveys -- we talked about the contributions in this area. But increasingly, yelp reviews -- the unstructured narrative comments. And complaints and grievances. Some of that information particularly in the nursing home area is collected. There is cost of care and assorted other information. Lots of different ways in which performance can be characterized.
This is one of those talks where I agreed to do it but then I looked back at the title and thought -- is there really a difference between patients and the public? I want to argue there is a difference. For the public we're talking about what I will call good housekeeping seal of approval. What we're trying to do is ensure that the best performing providers in the system are recognized and with any luck are paid appropriately. The Nirvana from a public measurement of quality perspective -- at least this is what I have talked about for a long time -- to eliminate variation and technical care. So that it doesn't matter where you go. You don't have to make a choice in order to get the right here at the right time no matter what. That is what I think the public emphasis has meant. But for patients I think it is a bit more like match.com. Match made in heaven for those of you who can't read. It is really finding the doctor hospital nursing home that is right for me. And right for me is a place or a person who is going to make sure that my values, my preferences are respected in my planning. It should be easy for me to find that option. Just like a good mate in this day and age. That is a different kind of requirements. It does recognize that what works for me might not be what works for someone else. There is variation there that is legitimate. It is about trying to put those two pieces together. I'm not going to walk you through this. I think when I went through and talk -- thought about how that drilled down to the kinds of measures that we use, largely the public facing measures, the public accountability measures are about trying to improve the whole system. Were asked for patients, these measures, what does this mean for me and what can I expect? That is the essence of what people are trying to understand. Whether it is standardized clinical measure or a patient reported outcomes, or any of the other kind of measures. As you look at the details of this, there are differences in terms of the expectations of those different kinds of measures. Those differences then extend to other considerations. Like the unit of analysis, or the nature of the metric. The way the presentation goes. We talked about the presentation of evidence-based reports. We know that for people to be able to really understand a lot of the quality measurement information, we need to do a lot more work on the presentation piece. The time is different, people's expectations around the scientific strength of the approach are different.
So I actually think that we focus a lot of attention and our research and a lot of the work that is going on on the public side. For the remainder of my remarks I want to talk a little bit more about what patients need out of the quality measurement. And I think it is because it is more difficult to address. It is a lot more challenging. In this way it is a reasonable area in which we can think about additional research and investments. In thinking about this is my I hope Arlene is not in the room. I did a brief systematic review of the literature trying to get ready for this. When I was going into this talk at Academy health, I thought I have not looked at this particular training longtime. They're going to be a lot of people who probably know more than I do. I should actually at least try to not be completely stupid. Here we go. We know that consumers are interested in information about health and health care. And how do we know this? They do lots of web searching on symptoms and other kinds of things. That is where -- online search behaviors is the basis for that evidence. We think that translates into all this other stuff we care about. Relatively few consumers are aware of report cards and other public information. A lot of the literature suggests that different studies have been done -- 11 or 70% of people have ever seen a report card on the hospital or health plan or whatever. Among those who are aware, only about 1% or 7% depending on the study actually use the information for their own decision-making. So that is fairly sobering. The available reports themselves very quite widely in terms of whether they produce ratings, rankings, assessments. They tend to use different measure and analytic methods and presentations and purposes. The real -- the result of that is that the same institution or individual can be rated quite differently depending on the system that you happen to look at. And that in itself makes one wonder -- what do I believe? Which one do I trust? On top of that we have known for a long time that in making decisions, humans can only hold in their brain about 4 or 6 factors. If you give -- there have been experience where if they give people more information, and they are actually happier for more information that their decision quality due to read speak -- deteriorates. This is about the right amount. We also know the reports themselves are rarely contextualized within a specific decisions framework. People have actually -- we go in with our hammer and we think people are looking to pound nails. But what we don't really think about is whether the information context -- content making the decision being made. We don't often recognize the fact that this is multi-attribute decision-making. At its finest. The bid we would like people to pay attention to is just that -- is a bit. We often don't make it easy for them to put it into all of the other things that they are thinking about making a particular decision.
We do know that the method of presenting information affects interpretability. People actually really get it wrong if it is presented poorly. And then, we rarely pay attention to issues related to literacy, and motivation. All of those things affects people ability even interpret the results. Just a few other observations from my non-systematic review. The existing literature is incredibly sideload. There is very little that brings a different bits together. In fact as I was reading through a lot of the publications, it is almost like they barely reference or acknowledge each other. Is lots of micro bodies of work. The research tends to focus on trying to make our hammers, measures work better. And rarely entertained the possibility we need different tools for different audiences. There is a lot of one side thinking in terms of even the way that the studies are set up. What is being learned in research does not seem to be translated into what is happening in the world. This is the general challenge the event talking about. It is important to recognize that there are some studies that I see, when I went to rush up on the literature I discovered we were still don't some of the same studies learning some of the same things we learned years ago. I guess it is still fresh because we have not used those lessons. I think it is important to recognize no matter how well we do the research, we're not going to stamp out the entrepreneurial spirit of repeating reports. There is a belief that if we just tell people this is the right method to produce a report card, all of the system in the world -- America's top 100 hospitals, leapfrog.Consumer Reports -- they will all do it the same way. That is not going to happen. I also don't think we're going to inspire people to spend more time and energy decisions than they do today. That is our budget constraint. People are going to -- I don't think as fantastic as we make it unless we make it Pokémon go we will get people to do it differently. I think it is important to think about what is the kind of work we can do that would make the greatest contributions to the world and not the literature in this area? What can we do to move the ball forward? Here are some things that are think are worth learning about. I don't think we actually have spent -- I did some work in 2002 trying to understand decision-making frameworks. Having an empirically validated conceptual framework about how patients make all those different kinds of decisions. I don't think it is a single unified -- maybe there is a unified field. At a minimum I will start with how people make those distant -- different kinds of decisions and understand what goes into it. And that we account for the different types of the consequences of those decisions. Those things are important. That we integrate the known dimensions of patient Harold Brunetti including the literacy motivation. And that we are open to learning about others. And that we are maybe at least going to some of the research about the role of standardized measures. It is clear from the research that these ring that draws people in our the narrative commentaries. Either we need to find ways to translate standardized measures into commentary or we need to find a way to more effectively incorporate narratives.
We need to ask ourselves what information do people need to make decisions. I think we have to fully engage patients to get there. I don't think frankly we as researchers -- I'm a patient to. I just think there is a believe that once you have been trained you lose your card-carrying ability to be a normal person. I do think it is important for us to engage with people and find out what it is they think they need to make information. A number of years ago I did an interview with my father about what information he was looking for in making a decision about choosing and oncologist. It was an interesting decision he was having to make at the time. He raised things that would not have occurred to me. I think we don't do enough of that asking people how they are thinking about it -- what is important to them and how they produce the information to help to make a decision right for them. This leads me -- I think the reality is no matter what we have a presentation budget we have to think of. Not only what people need and how do we get them the information for the decision-making. I think it means we will need very different types of measures for different decisions. That means we need more not fewer measures in order to capture the trend. There is a big push these days, the measurement reduction act of 2016 -- and while I think that that sends a important signal about the burden people feel. I don't think we should translate that into needing to measure less. We just need to find ways to measure more effectively input the information together that people need.
Then I think we have to look at how one of the most useful to get the information. And to be more creative. Websites Navigators, videos -- we're using a lot of videos in our system is ways to reach people. In a very different way. And actually we do that for our doctors and members. I think we just have this way that we would like to put information out and we just don't look at what is going on in the world around us and try to incorporate that. Here is just a -- some other considerations for the field. Having looked at at least a couple decades of work trying to get people to pay attention to report cards, it made me ask the question what if this is all we're going to get? What if 20% is where we topped off in terms of patients using systematic information and decision-making? Should we stop all public reporting? I would argue no -- but in some ways but keep using that as a reason to discount the value of public reporting that people are not really using it. I think it is a good question to ask. We have not moved the dial on that.
I think one thing to look at is how having this information on performance at different levels in the health system changes perceptions about the quality of the health care system. Whether that is good or bad I think people are a little bit more aware today that they can't completely trust that everything is going to be done perfectly. That means more and more people fell an obligation to get engaged in making sure their own care is delivered effectively.
I think making this information public has increased people's willingness and the delivery system side to improve performance. If you show people how well they are doing -- if you don't show people how well they're doing they won't fix it. I have found in talking to people, thereby introducing the possibility that their doctor is not perfect, it makes it actually safer to have a concert -- conversation about options. This idea that we are finding ways to try to even the playing field in terms of having a conversation about what else could it be -- we know that is important for diagnostic excellence. What else can I do -- we know that is important for matching options to values and preferences. The ability to open the dialogue I think has been a key role for public reporting. There might be other things. I think, we also should think about the biggest improvement to make to our research designs. Having being involved for a little while with the development of PCOR and watching the people -- patient are doing when we engage them and trying to answer questions, I am struck at how rich a resource that is that we're not tap into. A lot of the quality measurements and the health services research in general. We're just in the early phases of figuring out how to do that. But I think in terms of making this work really useful for patients, I think we will have to do a lot more of that engagement work.
With that I will turn it to Sharon.
I have one comment -- I think that it is very important that we think of those alternative methods to reach our patients. I had a patient recently who came to see me, not because of anything she saw on the standardized report on patient satisfaction -- but around yelp. I would mention is a great provider in my practice. I think that people are looking for real-time information. They are looking to alternative methods that we don't use in healthcare. We need to be savvy in collecting data sometimes an alternative forms.
I think some of the points you made speak for why AHRQ needs to find this work. You said what if this is all the time we get to make decisions. Those of us who have been in the process of making decisions for our family -- there is plenty of time to make decisions. You just don't have the information in the form. You can make much better decisions in less time the information was presented. As part of a research agenda. The second is, although we might consciously only be able to think about 46 pieces of information, there is a lot of information in a picture. A lot of information in a portrait. In a second Lafond a lot. We can tap into other parts of the brain is speaks of research agenda that can be put forward. A number of us have drunk the Kool-Aid around the 10 simple rules -- it is transparency is necessary. One of the things that we had is a goal this year was to post at the doctoral level all of the survey comments that we had for each position on our Internet. So that people can go and look into the doctoral level. Part of the motivation was to help people have access to that. Our main motivation was really to change Dr. behavior. The thing about public reporting for me, it is really about changing the behavior of our positions or other caregivers and other leaders.
Picking up on that -- since I use many physicians at my age, I have been not once have I been asked by a doctor about an important journal article or a publication from my organization has come out. But I am asked all of the time about something that was put into the press about something that we put out -- or on the Internet or discussion on television that we put out. So don't think that your public measures are not read and utilized by practitioners. In one way or another. I don't think it is completely one or the other.
I will just say my mother used to go into all appointments with a new doctor with my 2003 article in her hand. I tried to explain to her that was not a good way to get on the good foot with a new doctor. That is a way to get journal articles to doctors offices.
What we need to understand -- and what everybody does -- I feel the need to say this. I trained consumers all the time on how to be on their healthcare team. They do not know these things exist. We tell them -- go to compare..gov. When they get there they don't really understand how it is written. So a narrative will be wonderful. I also think that we can change the behaviors of our doctors if they could make that part of it. Of saying, you can go here -- if it was written in a way they would understand -- and read about this. In order to help you get to your decision. And possibly doing nothing at all. The big part is we have to educate our consumers.
Pose a -- you had a comment?
I think this is a follow-up or related to the comments a couple of comments ago. I think it is important to know -- of course we think [Indiscernible] and we want many people to use them. They as individuals perhaps have a better match with their provider and so forth. With regard to any system affects -- I think it was expressed as changing Dr. behavior. If we think that none of the potential of reporting quality is the quality improves, you don't need everybody to use -- only a few people tend to shop. It brings prices and competitive situations. You don't need for everybody to be choosing their provider based on quality her bedside manner in order for those reports to affect -- to meet institutions and providers to [Indiscernible]. There are [Indiscernible] that can happen. On the system. Are important to keep in mind when you're talking to people who say only 20% or 10% or whatever the petition is of patient use these things so why should we do some? Part of the important documentation or research agendas -- is to try to figure out to what extent these [Indiscernible] matter. And can affect the way providers [Indiscernible] etc. improve their care. My guess is that they are important.
I cannot agree more.
I am impressed that -- my very own founder of the organization has called for having the number of measures and so forth. The part I agree with is, at least having some consistency in how various people who put out measures defined and use them so we don't have to deal with 15 different definitions of the simple measure. Beyond that, in a way I agree that if I have a simple question I want to ask of the health system and I have done a lot of over -- online experimenting. It is hard to get relevant data for that decision. Let's say for example I need a hernia repair, I can't find any information that gets at that simple procedure. Even if you put aside the relational aspects of the position and all of that. I can't find it. And so for the vast majority of people, whether or not diabetes is well controlled in a clinic does not represent the issue they are likely to be showing up with. That is one issue. The second is a thirst for aggregate measures. To simplify I'm always being asked what is the one measure the hospital infection risk? Is to compensate it to have one for [captioners transitioning][Indiscernible].
So this is one issue. The thirst for aggregate issues, to simplify, I am always been asked, what is the one measure for the hospital risk?
[Captioners Transitioning, Please hang up the phone line so that the next Captioner can call in. Thank you.]
I use restaurants because in acute care, the counter, it is not like comparing it I spent a lot of time looking at the restaurants and how they function how to make a good egg and how to produce a nice loin of beef is actually very complicated. What would we learn from that? These restaurants really I rely on how many I thought the place was terrible ratings, so I'm not sure what can be learned from that restaurants are highly reliable to this risk. They can be closed down in an instant. I remember being in a Tuscan gap -- Grill. They only have three chefs, this rotisserie guy, they were famous for their chicken, he put it on the rotisserie, thoroughly washed his hands and dried them with a paper towel.
I seen the orders popping up, -- I said how do you remember to wash her hands? I thought that was pretty good. I'm not advocating for this but it would be a good and interesting thing to study and learn from.
I want to make sure that we have time for Sharon to tell me what is going on at Trenton, -- AHRQ pick
This goes back to Medicare stars, or [Indiscernible] which are public, it is what matters to patients, the food care. Medicare stars, they are much more public, I don't care about them. I will take the example, I am really proud as an organization that we have high immunization rates, as a patient, I do not give a damn about that, I do not want you to spend my money. You know -- So that I don't care about community. [Laughter] -- Immunity.
I like your concept. -- The stars are really geared towards patient measures rather than the public.
Okay.
Hello Beth, this is Jen on the phone, I have a comment when ready.
Go ahead.
It's a question I apologize I am having difficulty hearing part of the conversation and I I am also -- And I also am driving. For the future and current role in enhancing the science, how we adjust for vulnerabilities and differences and communities in the patient population, doing these quality measures we are doing across different health systems, they are not all homogeneous in the patient population that they serve, where they located. I am so glad you asked. We are going to turn to Sharon, who is going to give us a current role, they should say AHRQ roles. Sharon. I want to start by providing Academy health speaks to when we put them together we are developing an agenda for quality measurements, multiple perspectives really to try and identify what the open questions are in quality measurement. For the benefit of the researchers and Academy health, they will develop great proposal to submit to AHRQ, there were other presenters, and it was really lively discussion not just similar to the discussion we have had here. Clearly it is an important area. What we did was take that information, we have limited resources, and there is a lot of work to be done. How can we focus our attention in this area, it was important to turn to NAC, on how we should turn the investment in this area. For the agency and healthcare research equality, quality improvement, etc., quality measurement is clearly an important area, we want to be in the space. How do we want to structure the investment in the space?
Thank you to [Indiscernible] , the different type of roles that we play in quality measurement, these are the roles we have played historically, and our investment in each of these roles have been changing over time., We want to do a level set and where we want to focus our time. We have to identify quality measurement gaps around new methodologies, where we have been in the quality measurement development space, we are developing the role, and we have done a lot of data structure, we are also in the space in measure development identifying high-priority topics. Working with stakeholders working with patients, to identify the measures, testing reliability and validity, and re-finding test, a new roll over the last few roles, is measure stewardship over time, we have thought about that as the measure development work maintenance, there is really a whole new area of measure stewardship that has popped up in the last decade. Refining based on stakeholder feedback, maintaining the roof -- Consensus endorsement.
We have spent a lot of time up dissemination we do a lot of work to disseminate measures to provide toolkits. To provide toolkits for patients and we help and support users through technicals assistance, and we do a lot of reporting. We have had investments in each area, over time, I think it would be fair to say the amount of resources we are spending in each column has changed. I would probably say the majority of the investment in data development over time, our measure development and particularly stewardship has been increasing, we have asked ourselves is that appropriate. Should we be going in that direction. That is one of the things we hope to get your advice about. We have a number of initiatives and activities, we lay them out, so that we can see how they are categorized according to the four roles, we have a project looking at claims data. Looking that research and data development, with the hope measures can develop this data. We have been working on the development of care coordination and quality measures for quality care.
Our history in CAPS, this has been a huge investment for the agency. We have developed the national quality and disparity report, and we talk about the clearing counts these are the things we have done overtime to support the use of measures, and to cut back investments over time, the quality indicators, this is another area that cuts across all of the different categories, I didn't mean to skip over the pediatric program we are doing that in conjunction with CMS again, we have been involved in research and data development, the measure, and increasingly returning to the need of support for stewardship. Lots of different activities where we are doing activities in each bucket, the need to try to figure out, given eight crowded field, where should AHRQ be focusing our attention?
I want to give a little more detail about our flagship programs. The first one AHRQ quality measurement program, which is making use readily available data, this is an offshoot of our database known and well loved.
There are four modules, these are currently endorsed by NQF, used by CMS, in a variety of forms as well pick we have spent a lot of time researching and dated development we are increasingly spending a lot of time on maintenance and endorsement of that data, the question is -- With limited resources, and the more time we are spending effort, the less we have to do in other activities, these may not be in the context of the field, and how should we be devoting the time?
SCAP -- CAP Surveys. There are a number of different SCAP -- CAP Surveys, these are resource intensive. We continue to work on resource to measure development what is the value of our time in each category, how should we be spending our time for limited resources?
Can I just ask a quick question, when you say maintenance, you are just talking about instrument maintenance, not the research, or the results. Like with CAPs, you are just saying the survey instrument, the survey modifications?
Yes. Maintenance of responding to the questions, etc., absolutely. I talk about the pediatric quality measures program, we have done a lot of the research development, the measure development, now we are moving into the area of maintenance. The question is, what are the resources we should see spending in this area.
The questions are, with the limited resources, as a part of the portfolio and work, we have had a good discussion based upon baths presentation -- Beth's Presentation.
-- Given the response in the roles, with respect to quality measurement. How should AHRQ be spending time and resources? I will scroll back to the slide that defined each of these roles. I will try to get a discussion going and input as to how we should be focusing our scarce resources.
I didn't know if you had anything to add?
I was going to ask if anybody had completed NQF process? It is about three times better than ACA, it is an extensive filing, a significant grant proposal. When we are picturing the energy associated with that, related, part of that response that Sharon was trying to elicit from you, thinking also are some of these functions in some ways have natural transitions, handoffs, how do we interact? Do we have to do it all? What are the parts critical for AHRQ to retain? Those type of issues we are looking for garden -- Guidance on.
I think Jen, you had an opinion going in, I just wanted to see if you wanted to start the conversation with some of your thoughts on areas in which AHRQ can make a real contribution?
Yes, I think movement has been over the last few years, but I do think there is a significant need for science and evidence about the impact on disparities, and the hedges and 80 -- Hedging the Navy --
The disparities and how we compare apples to apples, at the same time there is validity and how we adjust for the patient population to understand the context in which the health systems provided are working in. -- Health systems, how they are providing and are working in. There can be better understanding on how to do that more can be done.
Thank you Jen, the miles in giving this grant for someone to develop it, this makes me think about bundles, when it came to maintain him it was okay.
This idea if you are going to develop that you were going to maintain it, they do not want to maintain something, that is a dilemma, I and then the measures for accountability, a dialogue in the NQF, on how they should look at measurements around quality, we were not going to measure with measures that were from improvement, you know how to do that, they are focusing on measures for accountability, that is probably okay for a federal agency to tilt towards that.
Having done that, the comment of research risk adjustment, and safety hospitals, this is really important. For researchers funded by AHRQ, there is another piece, which I don't think NQF have done, AHRQ is not doing it well either.
What are these measures to be deployed in healthcare systems, how can we learn from testing the prototype more thoroughly, one of the things has been done, and why people complain, measures have been throwing out, and measures have not really been understood fully, the measure, what it is like in their own hospital, they are stuck. We can probably do that her job in protecting measures that are about to be deployed.
I just want to say the majority of the measures involved have been used in accountability and payment, we have done a fair work across the board, in safety, for improvement within the organization. We have also done and are in the process of developing measures that look at patient engagement. Not just accountability. But more in the spirit of improvement. Within the health system practice whatever.
That is important. I would categorize your work more clearly. The willingness that they Congress, the funded whatever, it may depend on what you are doing, and the way. Now that you phrase it that way. Any measurement that you are doing to help us develop better quality improvement measures, is like saying if we are going to test new models, have measures that we can use across the models, that is helpful, but the clarity as to what you are doing.
I would add, even to the extent that we have developed have been used as accountability or payment, I'm not sure that they are working out that way. The research and the development were not for that purpose, but they were picked up for that purpose, and have been used for that purpose as well is another purpose, as well to our dismay. It is a fair amount of work to adapt.
If you think proactively. About testing these measures and how they will be used, you will probably predict which ones. I don't anticipate returns to the ICU, they probably will not be ending up in accountability measure, but you never know.
Yes, you never know.
Yes, I'm not sure that is a big distinction, who is using it, all of these measures after joint [Indiscernible] -- Replacements, is it accountability, who seems to have best outcomes? I'm beginning to appreciate there is this something that keeps coming up. This somatic issue. How do you optimize the assets you have? Versus how much do you push advancing science, and that is part of it there has been something there that has not been fully utilized by users. Really maintaining good stuff, you do not want to see just go sale. -- Go stale.
What is the new advanced quality measures, or synthesizing evidence from drug trials or complex real-world interventions. That is a choice, I would put a vote in trying to take what you have an optimize that how are people using this in there is so much good stuff here, I would say the same with this there would be a shame, to say -- You kind of have to be the stewards of something that is so well established, -- That to me would be a real loss.
Another example, Andy, you are working on this one. -- I'm working on hospitals and community groups, what are metrics that we will focus on in the city, one of them merged, preventable hospital hospitalizations, Andy and I did earlier research on this, someone from the health department says, here's the metrics, -- What is your opinion Andy, which one is the right one to use, they immediately went to AHRQ , which are the diagnostic codes?
I said Andy did not like the acute -- Do not do the whole preventable one. I said that is a really good service. There is someone in our local health department, the first-place she went to, was AHRQ . To me that is really important, and huge number of people need a place to go. Could they have gotten that at NQF? Or someplace else where they could have easily found those things? There are five other places, and they happen to pick AHRQ, you could've gotten for other codes, but you are the go to place, it's really important to maintain. I would look at things to hang onto, and things you can be the Stewart four, adopted implementation, whatever you can do to do that.
What are the new novel areas that you can help bring the whole community forward to? The pressing need now, we're overwhelmed with disease in the specific measures, we have not really figured out the measure of good diabetes care, or the care in the context of 12 more morbidities that the patient has. How do you advance thinking around that, primary care measures, I think this whole thinking, this primary care, you can take these 42 measures of cancer screening, I don't think that is really saying what we are doing, and getting value from those services, whether that is working with the community, doing strategic conferences, what is the methodology we should be working on in evidence synthesis? What are the whole care measures in patient care quality, that are not just patient care ulcers, how do you sympathize, where can you put the finger on the pulse, working with the community to say these are the cutting edge issues, where we can pull people together, and put into different thinking. That is a bigger speech than I intended, I'm sorry.
This is Lucy, Can I comment please?
Sure Lucy.
I agree with everything said, but one opportunity to advance around patient reported measures, CC JR demonstration, there are requirements where you include quality-of-life. For many of the patients and reported measures that are emerging in the field and getting widespread use, we don't know how to interpret the clinical significance of changes in the measure I would suggest that looking at that, and trying to interpret, what is the therapeutic response, how is it indicated, how can we use this measure, in many cases we are being used to collect?
Mary?
I think I will be repetitive and Kevin's comment, I do think optimizing what is known as AHRQ stuff, CAPs -- all of the work that has been done, that was you and are important to say, going forward, I would really encourage you to put whatever remains in evidence development. I sat on NQS board, there is not a lot to help us understand the linkage to what we are mentoring -- Measuring and what we know.
This systematic review, really it focuses on complexity, then I think, it might align well to think about building evidence around measurements of complexity, to the extent that you can have that kind of alignment going forward, and that would really track what happens. Meaning you build the evidence, all those measure developers out there, did it result in measures that ultimately become endorsed? You use in work around measures?
So I would invest in measures to track, it is all out there, but I'm not sure we are systematic leasing this work would not have happened accidentally, this I would say optimize what you have, invest your limited resources, the new ones in research and data development, especially around complexity measurement. Then track what happened.
Bob?
It is probably the engineer in me, but I would construct a four dimensional matrix.
I can show you how to do that.
[Laughter]
It is doable.
This would be one access. Another access would be problems you are trying to solve, the third access who are the other players that might be involved in how you can involve, and the fourth would be specific metrics, the framework in what you can do to optimize what you have, where there are gaps and rules, and how to play that out.
Charlie?
I guess I am going to say, and put it in another realm. For those of you who don't know. There is a set of objectives that are supposedly driving the department, called, healthy people. Healthy people objectives have grown to the extent, they have passed the percentage of growth, between ICD-9 and ICD-10- --
It has now been recommended by several of us, the department concentrate on what it wants. What is important to it right now? Then let it be one, and others, to concentrate what they do best, that is something that AHRQ should be doing relative to this, I completely agree with Kevin. He needs to look at what you do best. That in which you cannot maintain any longer, you can put your label on it, give a total partner, let them maintain it, and then look at areas that would give you greater us -- Those measures that would help you in the future. We will have some ideas about limited resources.
When we look at the cost of treatment going up quite dramatically even for diseases that have been around a long time, cost has tripled tenfold, a patient with multiple sclerosis cannot make good judgments on their peak, and which it is likely to help them, they are influenced by other stakeholders and ecology.
If you look at where the money, and where it will be going in treatments, and whether quality measures exist for quality measures, you are going to find there are a lot of gaps, they may have some measures but they may not have time for prime time public ability, and I will address the Paul -- A small portion.
-- Think about spending.
I had a very similar side. It seems where the cost is, and where the current bundles are going from CMS, take CJ are as an example -- Take CJR, not that we have looked at the vulnerability with joint implant, within the hospital as the surgeon started to work on this the amount of available best practice, this was fairly minimal, identifying nationally, if we can hook to that, to start to look at what evidence can support the important reduction in costs, there would be a lot of acidity with that as we get bundles working, the multidisciplinary group, which has a role in as well, is the land group, they are producing various papers around the high priority areas. Specialized care and cost, where that has escalated over many years. Those areas when you think about where AHRQ can have a lot of influence and impact, on health and impact on costs, but also on the political. Get the political recognition for the work, this me the helpful -- May be helpful.
Go ahead.
Great comments, I want to throw out a scenario to have you reflect on it, I am hearing useful ideas here. One thing we have had to reflect on a little bit, as we develop a measure, going through some of this process which becomes used by groups. Or in moves into the stewardship. Endorsed by NQF and so forth. Our capacity to brand that changes a bit, I would say we have been in meetings in which our measure has been talked about, we have gotten that from NQF. It is like saying you bought your Apple at Safeway. When Safeway of course doesn't own any trees, they do not grow any apples. That they buy them from farmers, I get that. I only raise that because not because -- Again this is an important part of the production process, but I think you all are reflecting back something I have been sensitive to, being visible in the political place where there is value seen. It gives me some pause to some extent, in a sense to have our resources used in this way. It is not quite as obvious the return on the investment and how it's in there being generated, you all get the stories as I'm telling them to you. As a day to day use, it is a challenge. Does that give you any pause, in regards to whether our best long-term strategy, is continuing to develop and have stewardship over these things? Or are there other kinds of ways in which we are able to develop RAND and -- Develop Brand and input. These are not easy questions.
We are happy in the development, you have touched on this Kevin, the work on PQRS -- PQI, Standardizing the approach, branding around that, and maintaining and developing all of these steps and so forth much of these quality measurements have become a consensus process labeling through NQF, not purely driven by the signs, which we are experts in does that runs risks? Some of the challenges we are talking about here today, in regards to helping convince people who would be able to provide ongoing funding for us.
I just wanted to restate that, at the end of the day we need some guidance on trying to thread that needle just a little bit.
I have a couple of comments, and questions as well. If you think about quality very broadly, you might say patient safety culture survey, and its associated measures, this is a part of the whole space, I have heard it is moving over to CMS, if you can comment on that? Sort of the second comment, we all appreciate a lot of these measures are moving into the electronic health record . On the one hand we have recognized the vendors are doing a poor job in that arena. There is this whole private sector activity for example, past, metrics, the second part in the electronic health record, do you have the core expertise? Amongst your staff? This far right area, it is relatively unique area.
So with the patient safety culture measure. This is part of the question. We have developed it, and you have used it, are we developing and maintaining this? Do we work with CMS to implement? We do the maintenance similar to the PQI, or the caps, they are using them for different purposes, we are the developers, we are working behind the scenes with CMS to maintain it, and to support them in the use of the measure. This is the question for us, how much do we continue to do, to protect the measure, and how much do we think the partners that want to use it, here you go -- Here is your responsibility. That is the first question.
Now I am forgetting the next question -- The electronic health records, we have some expertise internally, but we utilize the expertise of the research community. We contract with research entities, whether or not we have the staff right now that has expertise, we can develop that expertise if it is an area we need to go to. We feel we have the research community to expand our staff expertise. When we talk about AHRQ resources, we're talking about staff time. Contract money, grant money. Even political capital, our reputation. And the brand. We are talking about all of those things in terms of resources.
Yes?
Thank you great conversation, I actually love this conversation. For complex situations, our approach is to acknowledge the complexity. I think Andy, what you have said about the political perception is very important. The answer then, is not going to be all one way, or the other, it is going to be a mixture, once I got that far, and the thinking earlier, the electronic measures,, I've seen some -- The electronic measures, I've seen some that are generated automatically. There is a future dynamic evolution happening right now, you have to make budgetary decisions right now, one thought I have had coming from my experience in our rural, the partnership for patients, I tend to have a bias towards action MindView of measure development, certainly with patience, we are not going to touch measure development with resources we have, we want the resources to go into intervention, but if I were to be on your side of the equation, I would be selective with the measures I got into, I would use one of the criteria to decide which I got involved in, the measure in which you are involved is actively being used to drive improvement. The one measure I want to get a plug in for, we developed and partnerships for patients, the scorecard measure for patient safety, the estimation of the harm right. -- Harm rate.
Our baseline years, 145 harms per thousand just charged. That is an actively measure being used, it is being continued, I would argue strongly, that is the kind of measure you should be involved in. As you start to collect more measures, that are not so active or national, that is when you can become more selective. That is one approach that seems practical to me.
Don?
Yes follow up the -- Had why say this -- How do I say this.
And adjusting measures, they need to do this with NQS, I think this is a sorry state. It is nobody's fault, it is just the way it is. I was called by Australia, and can you explain the US system? I could not explain it. I would eat sophisticated. I should know enough to where I can explain it. This is a real opportunity, this provides HR Q, the same way we would explain to a fella, what is this agency, what are we uniquely good at? We talk about some of the research Orient, and what I get paid to do? Outside of HR Q, -- AHRQ , this requires something essential. Quick comment. Doing improvement measures, if this is important, all measures should be used to drive improvement. This is also an accountability measure. Thinking about the pathway from some ideas and different outcomes, including content very. We know certain antibiotics, we can reduce the rate of infection. Those things are good to measure, they are approximate to the outcome, if you can get the data, great, you can hold people accountable. Implementation theory, one of the activities, one of the things you can do to get evidence-based practice reliably out there, so that the patients can benefit. Most are around implementation theory. You Woodmont -- You would not want to hold them accountable for the context, and pay them for that, you would want to constantly reiterate on the methodology, and what spreads, and what seems to work, they are not meant to be durable, they are in a short time improvable, when I say that functional status, that should drive improvement.
If black people are not getting status, or not getting them at all, compared to white people, how you do that, and costs, some other method, these are different.
I agree with everything he said, with one thing, all measures should be, but the fact that the state of the world, all measures are not used equally to drive improvement at the same scale, this is something that Anna was saying, you can impact the world with scale of use, this helps you in these situations.
This is Lucy me a comment please?
Sure.
I agree with this discussion, I am thinking it is vitally important, I have encountered a lot of issues, that you are talking about. When we think about the measures and the action ability, some measures are constructed with severe lags. You may have data that does not reflect the current practice, as we consider the issue of action ability, to the extent in which two things, the data is closer to real time, and secondarily, when you actually put average improvement in place, the measure will reflect that. The one IM thinking about is CAUTTI -- You can do the improvement but it looks like your rate has gone up.
I just want to agree with Don, I agree with both of the things you said. Where there are some differences, the other thing you said Don, which is critical, agency harmonize nation, which is elusive in the federal government you can spit it -- Spin it a different way. You can switch it around. Now I will sell my Kiam contradicting myself, -- Now I will sound like I am contradicting myself. -- But I'm sure if you ask researchers, this is what they would like to see HR Q doing, they would like to see the funds go out to promote the scientific community rather to build and disseminate. There is this tension, someone has to be stewardship over this, to say we are the driver of development, science and defilement -- Refinement.
You are the biggest user of CAPs , if there needs to be some adaptation, and some measures for the people in mistake, it could -- Measures for the people who have in stake, it could -- I do want to put you on the spot -- But to do this and then handed off Turco
Someone has authorize Asian.
[Laughter]
Theoretically it does make sense, -- Someone has authorize Asian pick --
I don't know but to me it makes sense CMS is the headquarters -- There is a partnership partnerships cannot do anything at all measurable without AHRQ . We didn't have the ability, the personnel, or the access to the data. There is an analogy this logically rests within CMS, this costs money to do this, it works out well, and I think it is a great model.
I just want to pick up on Kamman -- Kevins last comment.
-- I would like to pick up on Kevin, and his last comment.
I am thinking as a pivot point, where we are needing new approaches, not only the progress that we have made, and frankly acknowledge the fact that we have very different ways of gathering information today, then most of what the quality measurement complex is based on. I feel we will miss the opportunity as a field, and we will be standing on the sidelines, watching what is getting developed, I hope there is a net investment and how we think about this, in a whole person truly patient centered way, how do we think about the advances that are necessary, how do we take advantage of quite desperate, novel data sources, that can give us a very different perspective on health and healthcare it is hard to support that kind of work, but I think it is an interesting question on how that helps, or doesn't help the brand, that I'm not sure if that is the branding strategy, you have to be: -- Kind of clear on your goals and what they are. In some ways I find the stewardship thing unfortunate. It is a dimension of resources used. In some ways it feels -- Actually I feel this is how most measurement development feels, you do not have to sign up to maintain the thing, anyway -- I would put a plug-in on having some amount of money to help youth move the field forward. Without the investments made in CAPs, I wonder where we would have gotten today, and what we think about important in healthcare, this is been an important investment. I think we are on the brink of needing the next leap forward. This is something the agency ought to lean on. This is my plug. For thinking about that.
You have all done a great job bringing to the surface of what I have wondered about even though I have said what it was, there was a letter before it, I was struggling should we be AHTQ? -- A stewardship and clearly it is part of something we do, but our part, does the research part related to quality? I think we are at the fork of the road, we have to take it as Yogi bear with say, because of the constraints and the resources that Sharon resource she emphasized, is there handoff of these functions to have them been maintained, perhaps the stewardship are, where if we didn't do the research part, is there anyone else available to do that?
I do think just to reflect a little bit. Not to pick on CMS in any way. I will just use them by example. In some ways when you are in the measurement development business, it is sort of like having a product you have on the shelf. If you are a payer like CMS who comes along, you are going to look around and say, what is a good product, for the thing I have? You pay attention to the company and what company made and whatever, but the balance point, it is not 100% clear, that there is a necessarily a requirement that CMS would say we will only buy products from other HHS partners, that would make sense that we can buy them from other places. It make sense what Donna saying, I am thrilled to answer your question Paul, -- It was a valuable partner, because it has been a partnership we have valued it, we have learned about our measure at the same time you have been playing it. Less fulfilling as an agency, we should think about it more if we purely supply products, and not continuing the development or learning from that in some ways. I have conceptualized a little bit how we prioritize in this area. One of the challenges we still have, and we will have heading towards conclusion today, bring about to the group, I want you to think about when you are in the plane, on the car -- On the plane, in the car, how can we have a systematized system, going on. We do not have a mechanism for that. I started to have preliminary conversations with folks, does the electronic health record open new ways? Spirit I think that --
I think that is an enormous challenge. This is a part of the production line, and really we have not created the business model, and how the receipt comes back in where we can show it in the systematic where you have heard anecdotes, these are little challenging, on the real impact, and what has been the impact of the work. IM feeling --
I am filling -- Feeling more of the sense of the case, it is hardly to full tell that story, to put AHRQ in the best light, -- That is a challenge, -- I want you to reflect on. May be something we can pick up on in the next meeting. This has been helpful to hear it is also hard for you to think about the same thing we are struggling with. We are happy that you can reflect on this with us. And think that it has been a valuable way to engage in, we have appreciated that very much.
I will turn it over to you back, Beth, to close it out.
I just want to think everyone -- Thank Everyone for coming out. I would love to hear from everyone, having a chance to hear from people, this is a very good way to structure the meeting, thank you I appreciate that it has been taken to heart, I appreciate at the beginning, the follow-up from the last meeting. I know personally, folks from the last meeting, we were in touch about a couple of things it reinforces, we know you are listening, but a kind of tracks the impact of your work. Sometimes you do not really see how that goes forward. I think it was very terrific -- Really terrific.
We are losing people faster so I will just say, we would love to hear from folks, about any ideas they have in terms of the next topics we focus on, is there an area that you would like to see us discuss, get in touch with any of us, we will start building the agenda for the next meeting around some of those ideas. Expect to hear back from us on the PCOR idea. So, that is a wrap.
Thank you so much everybody.
[Applause]
[Event Concluded]