Event ID: 2915865
Event Started: 4/20/2016 8:21:56 AM ET


Please stand by for realtime captions.

[ Captioner has not been admitted to audio event; if there has been a change, please contact Caption Colorado at 800-590-4197 or cc@captionedtext.com ]

They also provide support to the US preventative services task force. The task force is very busy and our staff is very busy supporting them. We had a number of final recommendations and draft recommendations that have been released in particular there was the breast cancer screening recommendation that was released a month or so ago.

To remind you that the task force found that the benefit of pornography -- mammography screening increases with age. Women ages 60 to 74 benefit the most. There was the recommendation that those women get treated every two years and for women under 50 the recommendation was that women should talk to their doctor about the general mix of [ Indiscernible ] and harm about screening. That recommendation was similar to their last recommendation and there was some controversy over that recommendation that you may be aware of. Interestingly, some of the other groups that developed guidelines seem to be moving closer to the task force recommendation so we see that as a relief. Smack the question out of ignorance, what is the actual formal relationship between the task force and [ Indiscernible ]?

The task force is independent -- is an independent entity. We nominate members to the task force. The director nominates members to the task force. The task force delivers on its own and they make their own recommendation they don't hear anything for us. But we have staff that provide support to them to work under direction of [ Indiscernible ]. We have the evidence reviews to support the task force.

Do you contribute to a choice and priorities of what the task force -- No, they do that independently. We respect their decision.

As a former member of the task force as far as support -- support is huge. When a draft recommendation is made, anybody in the country for something like breast cancer it seems, does comment, but every comment has to be catalogs, reviewed and responded to and that primarily falls to the staff which then -- the way they work is a set their own candidate . There are three people who were receive any certain projects but it is really different and on -- depended on the work being done. Support is great in task force couldn't do the job without -- I was prompted to ask that because of the constraint on evidence-based reviews that you mentioned earlier and trying to think through where in fact evidence-based reviews are done and how are they prioritized at a national level because we don't have the same organized way of doing that some other countries do it as a prioritization of what is required in this review and who doesn't need [ Indiscernible ].

The support that we get for the task force would provide funding for any evidence reviews of the tasks -- that the task force needs . That will not be cut. What was cut was evidence reviews that were not related to support the task force, so other issues that come to national attention [ Indiscernible ] or others I want to develop guidelines. We had to skill way back on those, but we are not required to scale back on support for the task force including evidence review to support that work. Thank you. So, more draft recommendation for out for public comment. We have call for nominations out for new task force members. Again, the director of ARC nominates new members but it is based upon nominations submitted. If you know of any individuals you think would be appropriate to serve on the task force, please let us know. There is a link there that provides more detail about the call for nominations. Denominations would need to be received by May 15 to be considered for this appointment, but we're always looking for new members like the [ Indiscernible- Name ] serve for a set term and they roll over every year. Please feel free to nominate new members or pass along this request others. So, that evidence-based practice Center has also been busy in spite of the fact that we are going to be limiting new ETCs that are not funded by the task force and not related to the [ Indiscernible ] trust fund discrimination implementation. As you can see the EPCs has been very busy and this is just the first part of the report that has been finalized in the last four months or so. There is a lot of really good work that gets generated out of the EPC that I think is important and critical work to help spoke understand the evidence in a very -- helps people understand the evidence in a very scientific way . I know that Rick has kind of talked about this initiative quite a bit. I just wanted to let you know that the papers that we sponsored around physician team incentive were released in a special issue of health services research in December. If you recall, this project was predicated on the fact that health care is highly complex and dependent on the specific needs of patients which may make it somewhat difficult to come up with useful measures that are aggregated across patients to use and value-based purchasing. So, any attempts to measure and reward some aspects of performance -- also run the risk of diverting attention away from other areas that may be just as important. So we wanted to fund research to look into this a little more deeply and help us understand how to identify ways to move forward and help us work with our partner agency particularly CMS indicting that and moving forward. We sponsored my papers which one was a conceptual framework with use of incentives the second was key implications of the evidence to date, innovations in the use of measures of patient experience, alternative to the use of financial expenses and finally a research agenda. We're also doing some follow-on work from this now. We are funding a project that is looking at the measurement of performance improvement insufficient practices, trying to understand how physician practices take measures to improve quality and how those measures relate to or don't relate to external quality improvement measure stuff. We're also doing some further exploration of patient narrative. That is the work today, but we're interested in continuing this work.

Do you know of the journal has made those open access? When I try to look at some of those I don't think I was able to see the whole paper.

We have paid for open access slam hoping that they have made it open access the we will look into that and make sure. I know it was really hard to find initially, but we did reach out to them a couple of time to remind them that we did pay for open access. I think I mentioned before that that we had a special emphasis notice on multiple chronic conditions. We also have other special emphasis notice is out there. These are notices out to research -- researchers to say we are interested in these topics , submit proposals through our open call for proposals for [ Indiscernible ] initiated research. Help IT safety, shared decision-making and primary care, three additional areas where we are particularly looking for proposals. Have a couple of questions to pose but don't limit yourself to just these. One thing that we have noticed is that we don't think that we have the attention a researchers in the research community especially with our funding trials the slasher we noticed a bit of a drop-off in the submissive -- submission of investigated initiated proposal. We have money set aside for investigative initiated work. How can we communicate to the research community that we are opening -- that we are open for business looking for proposals and how to work with the investigators to submit strong proposals?

Do you know what the typical pay lines are for investigators initiative proposals?

I think it is about 30%.

Really?

Yes, I think that is what it is. It is pretty close to that. We are funding a lot of proposals -- That is better -- there was a point several years ago -- I think there was a poll that when in two a lot of these investigators meeting because it was like six percentiles or something. Someone in my department had less than a 10% that was squirting got funded, so that is the kind of thing that people say why apply at that point. But if it is truly 30% -- I'm not sure -- I don't know how others feel but 30 percentile sounds pretty good I better go home and make the proposal now.

Is a big variation across different categories.

So I'm not the right one to answer this, but I think there is some but not a lot -- not all proposals like it submitted get scored , but those that get scored have a pretty high -- and we don't always go in order of pay line . Sometimes CSM we reach out if we think your proposal is particularly relevant and we think we can work with the investigator to meet the concerns of the study section. But we think that we have pretty good track record and we have been working very hard to look at our study sections and make sure that our study sections reflect the kind of work we are interested in doing. That is another area where we would welcome your input. As you look at our study sessions -- sections, do they represent the type of skill set you think needs to be reflected for the kind of work we are interested in receiving?

I think the best way to do it is to just send a notice out to the various groups research universities past applicants [ Indiscernible ]. I would look at the funding great as the percentage of [ Indiscernible ] that was sent it whether it was scored. [ Indiscernible ] whatever it is it will be wrong, but say that it is very competitive are very comparable to what is [ Indiscernible ] or NIH because we always respond to where the opportunities are. If it's not easy for me to get funded with the Corey or NIH that is where I go to put a lot of these things together. If I have a chance of getting funded I'm going to go there but if comparable that I will think about a different. I think just getting the word out within the context of the budget for HR Q budget funded research which increased. I won't go into bloody details of the others but everyone knows in the community -- that everybody but a lot of people are aware of the budgetary challenges that everyone has to face.

Is a but also the [ Indiscernible ] meeting in DC they also be a place to be a presence.

That is a great suggestion and we are also planning to have a contingent at the Society for health economics as well to try to generate the proposals. We were very surprised when NIH put out their statement clarifying what they do and don't fund to help economics. We got a call from people saying would you from this work and we said we have always been open and we have always said that we would find this and we welcome proposals in this area. It seems that people were not aware of that. That was a significant concern to us. We're trying to publicize that a little bit more.

[ Indiscernible - low volume ]

I was going to echo Sandys, which is just one of trying to think about the cognitive and psychological barriers people have around funding and pay lines and misperceptions and have very condensed targeted announcements that are very user-friendly that goes out to all the organizations and societies [ Indiscernible ] and then the word will get out. People will be very excited.

Yep here? Hope you all will help us spread the word.

I wonder if anybody has any ideas because it are nonobvious groups that have been reacting. To me it seems like the people who know about it know about it but that's because I'm sitting here, but are there other groups -- to the people you work with , are they aware of these things?

My organization wouldn't do this, but something like the PCPCC, we just to the paper with them to explain what employees are doing in that space. I wonder if they on the Primary Care specifically -- they do a lot of work in that area and they have a lot of that information that could be funneled, so I will definitely look at some of the groups there on the private side as well

As a training program director I'm thinking that the most fertile soil in the hundreds and hundreds of qualified health services researchers that you have helped train and the perception for years had been that while we have been trained and now we can't get funded from the industry that trained us -- agency that trained us which is a really sorry situation. So the fact that this is now an improved situation I think it will be treated with great enthusiasm. The question I have is is there right -- an ideal distribution of types of grant within the portfolio grants are ones versus other types, juniors versus seniors because a lot of the people merging from fellowship programs are ready for a full-scale -- [ Indiscernible ] but they have very good ideas.

I don't know we have said our targets in each one of those categories but we obviously like that you see a mix of different categories. We are welcoming proposals from former trainees as well. We have also been thinking about what is the mix across the types of work that we fund. We see a number of proposals and patient safety in particular and comparative [ Indiscernible ] from the days we did a lot of comparative effectiveness, but we have seen many fewer proposals around equity and cost and accessibility. Those areas we are particularly interested in. I think it would be worth it really emphasizing that because there's a little bit of disconnect between the publicity about [ Indiscernible ] successful funding and the [ Indiscernible ] and patient safety in the breath of work being done. Some of the most prescient it -- pressing issues we have right now -- there are other issues that are articulated in these emphasis statements so that would be really important to emphasize it so that people don't have a mental model of what works best.

Okay.

It seems like we have giving you a lot of input on how to get more but not so much input on how to get better, so could you comment a little bit on your sense of what the key deficits are or what the weaknesses that you are seeing in the proposals that you are getting because we might be able to get a little more guidance on how to speak to that piece of. You certainly don't want more crappy proposals.

I think that we are looking for good strong proposals, but we are also looking for work that meets our mission on how to improve the health system. We would like work that we can take to the next up for the investigators can take to the next up, how do you think about how the work will be used to improve health care quality safety, affordability, accessibility etc. and so we are looking for investigators that will look beyond the generation of the specific knowledge in the research. I would say that that is one of the most important things they're looking for.

I think often the way to do that is through the program announcements that are sort of very clearly written to say this is what we are looking for because of the time cost of submitting proposals investigators are going to get to pick a direction where there is Monday and the funder is sending a signal about in particular we're looking for things like this. I think some well-written proposals that really direct attention toward the areas that you feel like you're missing and I would just say I do think this area of how we go from evidence to practice, how we can spread those innovations in care delivery and the uptake of that evidence is ripe and mature. I think there are a lot of people out there who are interested in that but these some signaling about that being an area of interest. I think we will talk about this in the afternoon session a bit, but I think not only signaling the content areas but the kinds of designs that will be particularly value would be helpful.

I think this is really an opportunity for the agency through a program announcement or a more formal announcement to really say what you just said a few minutes ago because we are all adults and we are all used to being rejected frequently. It's the nature of almost everything. What is frustrating is when you do something and then you are passed over because of your score because of priority you didn't know existed. I think you where you can communicate, people will take directions pretty well from that. I think that is a really good way to go. The other thing that might be helpful depending on what you mean by strong and this might take a little bit of work but [ Indiscernible ] used to have an NIH -- and NIH still has a link you can go to about what are the most frequent uses for grants that get funded. People sitting around the table here -- the senior people have learned this sort of stuff , but I think one of the problems -- I have a lot of people -- I know a lot of people around the table have an you have that the senior people are eating the young . We need to make sure the next generation is being supported and funded along the way. I think to sort of real look at it -- maybe to put it in a formal announcement that will help early young investigators and brought in your pool so you don't keep funding the same general groups of people.

That is a great suggestion. We are working on program announcements so I think we will take these common start. Okay, now going to talk a little bit about some of the tools and training -- in training has released a toolkit for hospital reduce [ Indiscernible ] urinary tract infections. We took this based on implementing the complex in the comprehensive unit based safety programs. We also have some new resources to help with the treatment of alcohol use disorder. These are research summaries from our [ Indiscernible ] programs that are aimed for clinicians and consumers and it provides information -- information about the option by options used to treat medication drug disorders. We have [ Indiscernible ] for lung cancer screening for patients at clinicians. We also have new brochure that we have developed to help hospitals in particular to say patient safety organization. There is a new regulation that was just issue that allows qualified health plans to meet the ACA requirements when participating in the marketplace by contracting with hospitals that work with patient safety organizations. There are many hospitals that don't currently do that so we developed a brochure to help hospitals make the decision about whether to or not to wear hoping that that will increase participation. Now, data and methods?-- we have published some really interesting papers in the last few months. One, -- I don't know if you have seen this one was around mastectomies. It showed that while breast cancer rates are constant the rate at mastectomies -- of mastectomies increase significantly including double mastectomies. I was very interested.

We also have stats that looked at hospital stays for teenagers that show that one fourth of the hospital is associated tonight stays is associated with a mental or substance abuse disorder which is pretty striking. We've had a paper that I think Rick had talked about before that showed the growing difference between public and private payment rates for patient hospital care. As you can see private health insurance rates have been increasing pretty substantially publicly. This was very new information and went against the grain for a lot of folks. We have also just published something related to trends in capitation for physician visits. As you know, there has been an increase in the number of alternative payment methods that are used. Most of those methods seem to be a variation fee-for-service that capitation for what we seem other pretty significant decrease for capitation for physician payment. This information is a little hard to absorb because it really is just a dichotomy. The only thing we were measuring capitation and [ Indiscernible ] but it doesn't talk about other variations around fees for services. We have map [ Indiscernible ] stead of being field it currently so we will have information about physician payment arraignments hopefully next year or soon after. We're looking forward to receiving a data and vandalizing it -- analyzing it .

This sort of went against what my conception was of what was going on that I realized we were talking about a number of physician visit and is conceivable response to precious -- pressures that [ Indiscernible ] has gone up . Other people were talking about it and look at the dollar spent the one thing that seem to be lacking was a didn't talk about the number of patients covered by capitation versus service arrangement. We keep seeing in Medicare the number of Medicare advantage going up so I didn't get that. Do we know that other factor?

One thing that this data looks at is the actual payments of the patient clinician who provides service received so it's really not looking at the payment from the insured to the group or the entity that the physician as part of. When you look at the Medicare payment arrangement, that has been focused on the payments for Medicare to the physician group. This is a different level.

That was sort of our point is that there is all this talk about alternative payment methods and people think that that is going to be a different way of paying physicians. This was sort of a side effect of what data we collected in the medical expenditure panel survey. We actually new for each visit that was paid to the physician how was the -- how it was paid whether it was capitation or fee-for-service. A Medicare advantage or other types of HMO or other types of program -- programs, the issue is that the plan they get the capitation payment but the way they are paying their physician is fee-for-service that is what we were looking at. There's a disconnect exactly between what you are talking about because people think Medicare advantage is growing and that means we must have more of these six payment or bundling or whatever. Not true. A lot of the growth in Medicare advantage is fee-for-service types of things. Even though we're talking about making these payments and changing the way the payment systems are designed, is really a fee-for-service arrangement so you're going to have to figure out how to deal with these alternative payment arrangements in a fee-for-service kind of system and that is what we were trying to point out in the paper. Your point about -- there are arrangement where things are bundled or the new Medicare kinds of systems -- the organization gets a single payment but the way they pay their providers is still on the fee-for-service basis so you're going to have to build something into those if you want the appropriate incentives.

The other part is the physician visit or the state is all office based visits were could be in a capitated environment where they more than likely use a non-[ Indiscernible ] service provider that provide services in a not capitation environment.

That is true. CXO, does this not include those physicians or businesses that have case management fees attached but it is still covered by the fee-for-service?

Yes, if there is a separate fee that is paid to the physician group in order to manage the case, -- but when the person visits they are getting a fee for service payment. We wouldn't capture that side payment for the management. We were just looking at the way the individual digits were paid -- visits were paid at the point of with lists -- when the services were received.

[ Indiscernible - low volume ]

Exactly.

Does this paper study breakdown by state?

We did a little bit of that. The medical expenditure panel survey is optimized to be a national example. You can make some state estimates. We did regional estimates. Capitation as you might expect is really much more prevalent our rest because of the kinds of -- West because of the kinds organizations that exist out there but even in the organizations like Kaiser, they own the physician groups out West -- they are separate .

I beg to differ

Well, they contract with particular groups -- Permanente . But the physicians their work for that group whereas in the East, they tend to -- the more they will contract out with other physicians so those kinds of groups aren't as common.

The arrangements are slightly different, but it is much more prevalent and we did look a little bit at California but that is a large state.

This is a reflection back and a reflection forward that has something to do with this graph. When I looked at how HR Q makes a difference I began to think, what if somebody in the public see here? I didn't see the word person or patient and how Ark makes a difference. It makes a difference by improving the health and well-being in the healthcare outcome of people who live in the United States. That means that the messaging around very important research like this probably should also be seen through that lens. So, what is the headline for this as it relates -- I know it is not about patients per se, but ultimately everything we do is supposed to improve the health care and well-being of patients. I think about that in general and in communication whether it be about patient safety indicators or whether we are improving the safety of the United States whether it's the disparities and the equity curve, messaging it from the lens of the person -- the public would be really important. This can all be obscure. We are having trouble with it, so you can imagine what people in the public who may not even know what HRQ stands for -- they need that message so that whenever something is put out someone would also think what would a one-liner to from the point of view of the public where the patient? I'm not the patient representative on the panel but tran9 [ Laughter ]

I think that is a great suggestion and I concur. If I can help in any way -- I was also going to say that when we use the dissemination of the ark reports, we very much change it into the voice of the employer and employee because it does resonate much better than a graph like this, but I was going to ask if this is going to be continued because to refund the private side -- what we are understanding is fewer HMO but now APO is becoming more mature and overtime are looking to do more capitation or at least accepting risks of some kind. We may see this -- what we are hearing is going back up in 2015 and 2017 of those continue to mature.

Certainly APO is expanding but they pay on a fee-for-service basis. This paper was actually an update of the previous paper where we had seen the decline. We thought maybe with the new payment methods we thought exactly your point, perhaps things are changing but in fact it was even less. The fee-for-service was more prevalent than it had been in the earlier two years. I think that is an important point because people think that with all of these new arrangements that is going to go back to this kind of a system.

This afternoon we have some time set aside on the agenda to talk about the areas in which we think our investments could be particularly important. I would just say in setting that up, I'm going to come back to some of these issues. I hope we can talk a little bit more not all focused on [ Indiscernible ] abut jolt to be the beneficiary of this. Because I think one of the things that has been interesting is that -- first of all , the engagement of the committee on this topic, but I think also some of the observations about some of the difficulties in the way healthcare is organized and understanding it at a deeper level. Some of these issues that are important for patients, employers, organizations and physicians. I think a lot of the different [ Indiscernible ] around the table has a different interest in being able to better impact some of this information. I think we'll have a chance and that is just also a highlight not the only thing we will talk about in that section but definitely one of the areas I am interested in hearing from other committee members. This is a great preview of that.

I think I would like to introduce Joel before he came up here. This is Jule Collins. He is the director of our center for finance -- the group that [ Indiscernible ] the nest. Think you -- thank you .

We have more great findings coming out from jewels group. We had to articles recently published on ACA and labor market affects. The first article looked at the effect of the ACA requirements for employers to provide health insurance to employees working at least 30 hours and as you may recall there is speculation that that would increase the number of people working part-time as employers would look to cut hours and not have to pay for health insurance. There was no increased boost and people working just below the threshold. They also didn't find any decrease in the number of people working about the 30 hour threshold. The conclusion was there did not seem to be an impact on the labor market as a result of the ACA. Another study looked at the impact of the expanded Medicaid coverage low aged workers try low-wage workers. Again, it found no change in labor market participation either job loss or people at the low end of the wage scale to get onto Medicaid or job switching or transition from full to part-time status or vice versa as a result of Medicaid expansion. Those were really exciting articles that [ Indiscernible ] -- How about that third bullet point ?

So, what they did find was that workers that were just under retirement age, they did see some reduced hours, but they felt that it was because those people were perhaps ready to retire and did it need to work to receive employer-sponsored insurance, they can purchase their own insurance so they didn't have the same job loss necessarily. We are also continuing our work on the ARC quality indicators. Just a reminder we have a number of modules for the quality indicator projects. They are develop based upon district data from the healthcare cost and utilization Project. We have prevention, quality indicators, the inpatient quality indicators, the patient safety indicators in the pediatric quality indicators. The work that is going on now is the advancement and specification for the patient safety indicators. We are going to be releasing a new version in June. Will have new version of the inpatient indicators in the pediatric quality indicators in August. We are also updating the quality indicators for ICD 10. So, quite a lot of work going on in that area some of which is being supported by CMS as they use some of these quality indicators in their [ Indiscernible ].

Can I just ask a question about that?

Certainly.

Are you going to be revisiting the PSI 90 in doing some work there? I know with both the star program and the tax program there is a huge reliance on it and there is no deficiency there.

Yes that is the focus of a lot of the work we're doing right now.

Great, thank you.

So, our next set of questions for the neck -- knack relates to -- a talk a little bit about this data and measures and one question that we have given the policy research questions that are increasingly being asked, are the new data resources we should be thinking about developing that will answer those questions? We have our current data resources we have, [ Indiscernible- Name ], [ Indiscernible- Name ],. Is there new data we should be thinking up that are point answer better the questions of the future and at the corollary, what is the continued value of our existing data resources? We think that they are flagship resources, but with some of the new EHR and claims data resources coming out, how do we continue to make them relevant or do we think about putting our resources elsewhere? I hope this is wasn't -- this wasn't going to cut in this afternoons conversation but these are questions that we have and we would very much like your strategic guidance on [ Captioners Transitioning ].


-- we think they are flagship resources, but with some of the new EHR and claims data resources coming out, how do we continue to make them relevant or do we think about putting resources elsewhere? I hope this wasn't going to cut into this afternoon's conversation, but these are questions we have and we would like your strategic guidance.

I was wondering if there had been any thought given to data sources that aren't so focused on healthcare delivery? As we are thinking more broadly about health and health outcomes and healthcare being part of that solution with cross sector collaboration utilizing data from other federal agencies to map more closely the alignment between the healthcare the people are receiving as well as other material these insecurities and other social determinants of health that can ultimately impact people's health outcomes. Not only looking at the healthcare measures the link at the same time other data sources from other federal agencies to get a more comprehensive picture of how people are experiencing health at the individual life.

I echo that. At Intermountain we are doing a lot with the [Indiscernible] because this need to adjust for social determinants is so crucial, but nevertheless, that measurement will work in highly densely populated areas, but there is a lot of work that needs to be done here.

The data sources AHRQ has provided over the years have been true the catalytic and fellows come down to learn how to use data and it has been a welcoming environment, that I think you put your finger right on the pulse of the future, that we have to constantly keep up with or even ahead of -- when my fellows want to look at pediatric data they use -- and that gives them pretty granular data from a number of pediatric hospitals. It is not as general as some of the data sources that are available from AHRQ that it allows them to ask different kinds of questions. One of the great areas of collaboration I know goes on in HHS is the relationship between what AHRQ does and with CMS does and what the IT and the health Academy does and data space folks. That is all good, that it is constantly getting ahead of that when people stand up and say big data is going to handle all problems.One way to start is to again help people who are not health services researchers understand the richness of data available from HHS at large. I am constantly surprised at what some of my fellows come up with to understand the health of the U.S. population. The economics of how people are living in the United States. Just helping us all see where that data is now and where I can go for large simple trials or for assisting in real-time for the FDA and the Sentinel system. These are somehow related and you can be this sense makers on top of it because of the long and distinguished history in using public data for public good.

I think this is a great question. I don't really have a specific answer, but I have a suggestion about a process. I think we are at an inflection point giving what is going on with healthcare reform. Right now we're sort of fragmented. We can beat this a source for access to all of those and identify if there are gaps.s. Maybe it needs to be expanded or their needs to be a compelling argument for targeted funding if that was needed because I think -- are huge successes of the agency. The other thing I would also ask and again [Indiscernible] I'm not the biggest fan of ICD-9 and ICD 10. One of the reasons is I don't have confidence at least in the near future that data are going to be reliable and valid. I think that is something that could be intramural at AHRQ because you guys understand this stuff better than most people or it could be a targeted opportunity. I think when it comes to data sets, we often integrate these in various ways and that is a big change and I don't know how it is going to play out. The complexity is large and it exceeds the accuracy of medical records because most of us have not been trained in doing it and certainly at the level [Indiscernible] could be -- when it comes to data since those of us who use these in various ways, that is a big change -- the complexity is large and it really exceeds the accuracy of medical records. Most of us code stuff and put stuff in electronic records and not have been trained. What you call congestive heart failure and what I call congestive heart failure, let alone talking about stage 3 or stage 4 or whether it is sunny or rainy today. The larger point is is is a two important question. You guys are the people to convene it and take leadership.

I know there is a lot of work on the transition from ICD-9 to ICD-10 being done by the HCUP team as part of looking at updating the QIs and the PSI 90. They have started to uncover some of that. And you are right, it is a huge issue. I would tell you what we have been thinking in a marginal terms. We haven't really been thinking about the big changes and I think your idea of convening a group to think through this more deeply is a great idea.

One other suggestion is that I'd be included in that.

Absolutely, you are number one. We've been thinking about the possibility of linking HCUP with some kind of market data so when people look at the utilization they can understand more about the market environment in which that utilization is occurring. That might be something that could make the HCUP data more useful. We talked a little bit about adding some information on the kind of physician practice. We are doing that for 1 year because we only had have the money but should we think about an expansion and if there are additional items we should be thinking about capturing that will make it more a rich data source. We've been thinking about these things in terms of marginal improvements on the date of resources we have, but we really haven't taken a few steps back in thinking if we have all the money in the world, what would we do and what is important for the healthcare system? I think your idea would be g reat.

Another unique opportunity i s, I'm not a big fan of it but I don't have any control of it, there is a huge consolidation going on in the payer site. There might be potential partnerships that could be done in terms of public use d atabases. I'm thinking about things that streamlined access to valid and reliable data that could accelerate. The problem with research, by the time we asked the question, right the grant and get funding, we have something of no value because they made policy decisions. Anything to shorten that would be helpful.

I went to make a brief point of information. One of the challenges with this wealth of multiple data is everybody collects things differently and defines things differently. Our evidence now project, we were trying to collect the simple quality indicator of smoking cessation. It was in 50 different places and EHRs. One of the projects I want to make you aware of we're doing is we have funding to the patient centered outcomes -- maintain the registry of registries and embarking on a project to standardize and get a consensus on harmonizing on registries. I think that work and have value in other arenas as well is that moves forward.

If I could speak to that, and it again maybe my opinion coming from another Children's Hospital where I gave a talk and we looked at multiple databases within one institution they were collecting data for. Each asked literally the same question in a different manner. I am curious to see if there is a look to standardize. I may disagree a little, but ICD-10 has been problematic but it is an attempt to standardize the data in a way that we can study it in a more standardized approach. There are so many data sets. I'm putting in -- between two Children's Hospital I can't even look at the EMR the same way because the data is being collected differently and folks have customized EMR's to the point where you can't collect data the same way. I think money well spent is figuring out how we can access that data in a very simple standardized manner if it can be done.

I was glad Sandy mentioned the payer side because I wasn't sure whether or not you could access any of that. I know for example CMS is working on this health care fraud prevention partnership with a bunch of health plans and employers that are trying to pull together data from all places to see how they can identify fraud. If there is any private sources, I know IBM Watson or things like that that are coming together now might be an opportunity to partner as well.

I had similar thoughts. From the patient's point of view and working with multi- centers and the fact even within the centers, the EMRs are not standardized. You have to look at what you can do. I think to standardize the registries and having as Sandy recommended a one-day kind of workshop rather than just trying to cram it into half an hour would be at an excellent idea. In our small field of celiac disease, we knew the validated instruments were not answering the questions. When we brought together the scientists and researchers, industry, government and patience, it was really a day worthwhile. I wanted to again say that is the way to go.

This is Mary. I'm glad there is work being done in harmonizing the registries and measures because -- has made a requirement for CMS to use registry measures. Registry measures will not have to undergo and get the endorsement. It is quite important to focus on that. The second thing is -- requires multi- payer alignment. A multi- payer data source would be great and if it can be done and linked across so we can follow patients sometimes. Lastly about pharmaceutical d ata. Pharmacy data, part D data and also the pride -- private payers. It could be an aggregation database for understanding medication issues.

It's interesting on the harmonization issue that the department is trying to work on this right now with our own surveys. I will tell you this is not an easy process.People hold onto their questions more than they do there c hildren. I'm hopeful, the cochair of the data counsel is trying to make something of this. If we can do it at the federal end, how can we expect you all to do it? I applied your interest. I don't want to throw cold water on the ICD-10 issue, since we didn't go to ICD-10 for a number of years, there could not be changes to the ICD-10 that many groups wanted while it was being implemented. Actually, there is going to be a new influx of new codes into ICD-10 in the coming year. I would just say the federal government is not driving this. There is a great pressure to even be more specific and whether that is advisable or n ot, that is not for me to say. It is an interesting situation.

I would just add a couple of things. I like Sandy's idea about a process. I think is part of the process an important organizing frame --

[Audio Faint/Low Speaker]

Maybe we can make a YouTube video. I think an important frame is to have people come to the table to talk about the important questions. I was a in our organization as we have done that, that is our best insight into what is missing in terms of data availability or structure. You get a little bit biased when you say here is the data source and how can we make it better? Versus one of the questions were trying to answer and our a bility. I think people have stopped asking important questions because it seems too hard to get the information you need. An example that we have started touching on today, and I think it is really challenging, but it is to understand how care is organized from sort of a Microsystems level all the way up to higher levels of aggregation. A lot of analyses that are done lump things together they should not be lumped and that is because we don't have great classification systems. I think the precursor to a data source are classification systems that would enable even organizations like our own to be able to in a systematic and consistent way be able to characterize the way in which we are organized in the way in which we pay. There is a lot of assumptions that lead us down blind paths in terms of drawing conclusions about organizations and payment. I think that is an area that could be worthwhile. I would just say I am part of the -- which is an attempt to bring a lot of care delivery organizations together around the country to create some kind of standardized data and a network that offers an opportunity to do research nationally. The leaders came together, the system leaders from those organizations came together and there is a report at the IOM on some of the questions they are asking. That is one place you might start with in terms of questions they couldn't answer and they touched on Monica's point about wanting to better understand things like the social determinants and their own populations and the communities in which they are operating because that contextual information is really important and they don't feel like they have a systematic way of getting to that. Even understanding the way things are organized. The comment we made earlier about dissemination and implementation, some of those understanding in which the context that thing somebody did that looks cool was successful and thinking about it if we understand about it what it would take to implement someplace else. The final thing. This is something I feel like I have learned about the last two years, there is a difference between a tool like an electronic health record and the way data flow into that tool which is really about clinical workflow. That's more than what the vendor is that you but your tool from is the driver for how the data live in the wild. No amount of attempt to standardize vendors per se will overcome the challenge of variable workflow that means, and I will give a brief example, I have asked three vascular surgeons in our organization that practice in the same medical Center where they document where they made a recommendation patients go on aspirin and I got three different answers. They were together in the same facility and used the same electronic health record. The three of them looked at each other. That is the hard work we do to try to mine the data and getting into a place where we can use i t. I think this idea that we can do it through vendor systems and kind of ignore the work floor thing -- workflow thing, I don't think you should be drawn into the idea there are simple solutions at the back end. Not that we don't need to work on those but that is a critical piece to be aware of. If that is true in organized systems, just imagine what it is like a not so organized systems.

Thank you all for your feedback. I think there is a lot of good ideas. Apropos of the admonition to not start with the data that start with the research questions, I think we will think carefully about marrying up the results from the discussion this afternoon and what data we need to answer that and think going forward, especially with our partners and Charlie and others in the department thinking about where we should go from here so thank you, very much.

That was bullet 1. We have a 15 minute break on the schedule. We can be a little relaxed about that, but we will want to come back in time to hear Arlene talk about AHRQs' work in primary care. There will be an opportunity for public comment before lunch and then we will come back to some of the issues we have talked about in terms of talking about the priorities for the future. I would just say I have asked David Ballard and Jed Weissberg to kickoff that future initiatives thing, but I hope this is an area where everybody will participate so I look forward to hearing of thoughts people might have. I think we touched on a lot of areas already this morning but want to make sure we have an even broader frame this afternoon if that were possible. That is the plan for the rest of the day and you have earned your self a break.



[The AHRQ Webcast is on a break and will resume shortly.]

This is Jamie. Is there anyone else still on the line?

Yes.

We need to disconnect the phone line and had you call back in. If you call back and we should solve this connection problem.

Thank you.

Okay, here we are. Right on time.

For our next session, we are going to have a discussion on advancing research and innovation in primary care focusing on the work AHRQ is doing to advance primary care and lead the discussion. We have got Arlene Bierman. She is the director of evidence and practice improvement at A HRQ.Arlene has been here for seven months and is a whirlwind of activity doing all kinds of work. One of the things she is doing is trying to consolidate the AHRQ work around primary care and continue to give increased visibility and link it to the other things AHRQ is doing.

Thank you. First I want to thank Kevin and Jen for asking for primary care to be on the agenda. I am delighted to be talking about primary care as a primary care physician, a general internist and geriatrician, and early in my career I read for several years a primary care internal medicine residency training program and a large Department of ambulatory care that took care of underserved communities. This is a big part of my p assion.

I'm happy to talk about primary care today because I would like to hear your ideas and input. I was here in the fall and presented some of the work of R H I T division and I in the input I got from the NAC has been valuable in advancing our portfolio.portfolio.

I think primary care is into go to the mission of AHRQ. We are finalizing the -- achieving AHRQ mission of improving quality safety, etc.. Just to say primary care should be first contact provides acute, chronic and preventive care but like a linchpin in terms of connecting people if it works well and helping them navigate the health care system. We know countries that have stronger primary care assistance have better health outcomes and increased efficiency. We know there are increasing demands on the primary care system in the U.S. with expanded insurance coverage as well as an aging and increasing complex population.

This is AHRQ's three buckets, how investments in primary care make a difference. The ultimate goal is to improve the quality of care and outcomes for patients as well as improving population health but to get their we invest in research and evidence to understand how to improve the quality and safety of primary care. Creek materials to teach and train health care systems in primary care professionals to catalyze improvements in primary care and generate measures and data and track improvement performance of primary care and evaluate progress. I will give you examples of our work in that area. Our past portfolio, what we're doing currently and shift the conversation into where we should be heading in the future.

I direct something called the center of evidence and practice improvement. I wanted to put our primary care work in context. This center was created a bit over a year ago by putting together two pre-existing centers, the Center for outcomes and evidence in CP 3 which is the center for primary care prevention and partnerships. We have five divisions. Our divisions are the evidence -based practice Center program, the U.S. preventative services task force program, division of decision science and patient engagement. That is division of health improvement technology, division of practice improvement and the national Center for excellence in primary care research. This is an opportunity and I was struck by how much was going on in these different divisions and how much benefit there could be by aligning this work with the comment impact across these different divisions. For an example of the division of decision patient engagement is known for the national guideline clearinghouse, the national quality measures clearinghouse and how could some of the tools we have better support primary care reform.

With the health information technology we have a new initiative that we announced a learning center around clinical decision support and that is key for primary care. We are putting language in our request for funding for clinical decision support to make sure we get applications that address CDS within primary care practices. The division of practice improvement is home to evidence which is our biggest investment and I think primary care transformation which I will talk to a little bit in a while.

What is the national center of excellence in primary care research? Right now it is a virtual center. Is a webpage which provides links to all of our resources in primary care, and they are quite extensive. George Brust was on IPA with this part-time from Morehouse and we put together a plan for developing and expanding this center. We have recruited a director for that center to lead it is a well-established primary care research or an family physician. We are hoping to announce that fairly soon and he will be coming to join us to lead that center. George wrote a wonderful blog that has been posted on the AHRQ website about his experiences and about what he learned about primary care at AHRQ while he was there.

AHRQ has had a huge portfolio in primary care research. For the sake of time I'm not going to go into that. We will follow-up with summary materials that will give you a brief overview of some of these. Basically the patient centered medical home, there was a large investment. A lot of that work has been picked up by CMS and CMMI. We have a primary care home medical resource center on our website that provides lots of good resources for practices and practitioners and researchers studying primary care. We have a series of grant some primary care transformation. We have developed materials around practice facilitation and interestingly yesterday, some people from the American Cancer Society came to meet with us who just completed a pilot of improving cancer screening in community health centers and other disadvantage services and they used all of our materials to do the practice facilitation and we are going about that. There was also a large amount of funding in a series of grants around caring for people with multiple chronic conditions. That is a priority area for us going forward. Primary care really needs to be the home for Kerry and coordinating care for these complex patients. I also mentioned we have the primary care practice -based research networks and talking about data and capacity. I think that is an underused resource. There is now 174 -- that include over 30,000 practitioners and have a reach of over 80 million patients. The could be ideas of how we could that are tap into that resource as we move forward.

Evidence now is coming to its first year of funding. This is the largest investment in primary care research AHRQ has ever done. It is a group of seven state and regional collaboratives across the country. You can see on a map where they are located. They are aimed at reducing cardiovascular risk in primary care practices. It is linked to the million hearts campaign. There is a couple of interesting things about this program.One is each of the regions has its own intervention and standardizing metrics across all of them. There is a national evaluation center. There are two goals. One is to actually improve performance on measures of quality indicators related to cardiovascular risk and the other is to study practice facilitation and what it takes. I will talk a little bit more about where we are in terms of the evaluation and what we're doing there. I think this could be a good model going forward in terms of implementing and creating change and advancing quality but at the same time developing science about how you do that.

We have a host of tools for primary care clinicians. I just put up the field them on this slide. Team STEPPS has been widely adapted and hospital settings and we recently released team STEPPS for ambulatory care and this is something that is available and I think will be very useful. We actually have an app from the U.S. preventative service task force. It just got a five star rating from the American Academy of family physicians. I have heard residents use this in the clinic all the time to find out what preventative services they should be o rdering. Also another example is our AHRQ literacy universal precautions toolkit to help providers in terms of getting better outcomes with patients who have low health literacy.

Another thing I want to talk about in terms of our tools is we are working closely with CMMI to see which of our tools can support different initiatives. They have the new primary care initiative and a lot of the work AHRQs has developed and is available is really useful for the work they are doing.

Are you working with Kate?

She is actually in CC SQ standards. Also with [Indiscernible]. I think that is one area where we can have a good role in terms of supporting those efforts. We are actually working with both of those centers.

In terms of measurements, we have done a lot of work in terms of developing measures for primary care and have some very useful measures, care coordination measures, database relationships measures database. I know these are used because before coming to AHRQ I sat on the geriatric measurement group and we have care coordination is at the request of CMS. There initial work was based on polling stuff from AHRQs resources. Also items specifically related to the medical home and hasn't ambulatory care improvement guide. That is some of the measurement work we have.

I'm going to shift about where we should be heading. You I talked about evidence now. We are coming to the end of our first year. The target is 1500 practices and 5000 clinicians and we have already enrolled 1200 practices. In doing this, we identified a lot of challenges like measurement issues. Within the next short while, we will be releasing the baseline performance on this practices as well as there capacity to change and there capacity to implement the interventions. Each of the different collaboratives is doing their own interventions. At the same time, the national evaluation center is going to look across these to learn what worked in different settings and is doing a robust method it evaluation. This is a model were AHRQ is going to go in terms of what capacity and support do primary care practices need to up take patient centered research findings as we are tasked with implementation and what can we learn from this project that is sustainable and can be implemented in future work.

We recently had an RFA, this is part of the departments opioid initiative calling for applications to improve the capacity of rule primary care practices to manage opioid addiction. We've gotten a fair number of applications and they will be reviewed in our me funding meeting so we should be announcing again in the upcoming months which projects were funded. This was a great example of the lining, the primary care work with the work of the HIT division because one of the areas of focus was the use of telehealth to provide necessary capacity and resources to will communities and isolated small practices.

We put out a special emphasis notice on primary care research outlining the areas we are interested in. We really want applications. This is a good time for people to apply because there is money in our investigator initiated research funds. The other thing we did y esterday, a special emphasis notice came out about developing plans for people with multiple chronic conditions that is primary care focused. That just came out yesterday afternoon and we are quite excited about that. He other thing we're doing is the agency is in the process of updating its general program announcement. We are making sure as that is updated we includes language from all of these PAs to be permanently part of the general program announcement for the agency.

There is lots of other areas where primary care research has been integrated across different sectors and different divisions. I mentioned about the clinical decision support. We are also doing a lot of work. This grew out of the feedback we got from the NAC last time on patient reported outcomes. Were going to put out patient reported outcomes particularly in ambulatory settings to improve care for complex patients. That is one synergy there. Our division of decision science and patient engagement just put out a special emphasis notice looking at shared decision-making in income and racial and ethnic minority populations because a lot of decision work has not been tailored to those populations and there is limited evidence in that areas. Our center for patient safety has expanded its safety work beyond just acute settings and is focusing across the health system and has a large and emphasis and several funding announcements around patient safety and ambulatory care. There are lots of opportunities I believe for funding.

Just to think about where we could be headed in the future. A lot of our focus is in the primary care practice and the relation of the practice to patient-family caregivers, etc.. One of the areas where we need to head is how we link primary care to the rest of the health system. Some of the other sectors around this is home care, public health, social and community services. On the other side more specialty care. Postacute care and long-term care services. Ideally primary care should be the center for coordinating care for patients as they transition across all these different sectors. Thoughts about how we might do that and also help primary care as the population ages and changes could do a better job of being a home for people who are more complex and have special needs.

Questions for the HIT, what are likely priority areas in the next five to tenures were AHRQ could make unique contributions to improve primary care we also have all the buckets we focus but how can AHRQ do a better job of engaging in getting input from the primary care community and what partnerships we should be thinking about. Partnerships with different primary care groups, providers, patients, other people who are focusing on primary care research. I will stop there.

As I look at the areas you talk about in the new special emphasis notice, I really like it because my biggest concern is some of the things we talked about our I don't know plug-and-play or slice s olutions, and those are easy. How do you improve opioid screening in a primary care practice? We do that and we do a great except then domestic violence screening or exercise or tobacco falls off he goes we have not figured out how to do it all and have primary care do everything we expect that to do. That is by far my biggest issue. I see it in here and I hope we can get to that. We are killing primary care and we keep adding more things and things drop-off and we are still killing them. Medical homes sound great and we put into a seals and they are feeling because primary care can't do we currently expect them to do.

I agree.

I have a question about your sense of courage utilization from your online resources. A lot of things are on the web for people to pull down. Do you track that and have any feedback about the distribution of users or anything that would be helpful in looking back as we want to look forward?

That is such an important question. Yes, we do track that. We know those numbers are widely used, some more than others. We also know we haven't done as great a job of getting them out broadly to the practice community as we can. That is why we have been partnering with CMS about some of these tools. I recently gave a talk in Savannah, Georgia. Part of it I shared even more widely some of the tools we have available for primary care. So many people came up to me. The audience was people involved in health system change. There were practitioners. They said these are great. We were googling your site and we didn't know they were there. We need to do a better job of identifying which of these tools is most helpful for which settings and disseminating them better. There is a huge opportunity there and I welcome ideas for how to do that.

We are a small agency and we try and develop the evidence and tools and then we have to partner with other o rganizations. We partner with CMS and others to help the full dissemination of our tools so we can develop things and test and pilot them, but we are not really able to push implementation across the system and we have to rely on partnerships to do that. Helping us identify appropriate partners and thinking about ways we can take the things we have developed and help disseminate them throughout healthcare or identify other things we can do to help primary care practices is what we are really looking for.

Just to clarify, I was wondering if you had information on the users going to the website? Not so much your ability to push things out.

We are very limited in what we can collect on that. We try and do some kind of case studies of users where we can find them but if you go to the AHRQ website or other government websites, we are kind of limited in the statistics we can collect. We would love to collect lots of things but unfortunately not.

Just so people know, I got Sherry, Mary, Kevin and then I will would check and see if anybody on the phone has input so if you are on the phone, we will turn to you after that.

I have a few thoughts on t his. The first is at the business group we have seen behavioral health on the employer side has become one of the hottest topics and access to treatment and getting people to the right places. Making sure mental health is incorporated into the research we are doing on primary care I think is something we haven't had great data on. Anything that can help support the primary care on that side would be really helpful. I know you have a lot of great data out there for the p roviders. If there is some way you can help us translate it so we can disseminated more broadly to the employer world. I know a lot of our members are bypassing the system and building their own primary care centers making a part of NaCl, going direct to healthcare providers because they are seeing gaps in care or they want to make sure they have the best care they can have. Some are just not moving into the ACO world. If there is information we can provide that says you need to embrace this now, you need to pay the care coordination these and support this initiative, that would be really helpful for us. The last thing is in terms of tools, Incorporated in the workflow for primary care, I really think they need additional data on prescription drug prescribing, what is on the formulary to avoid the prior authorization process when the patient gets to the pharmacy and making sure they know what is available and what is generic and the price of these things. I think there are tools date no that somebody picked up that prescription. There is a whole area on primary care affected safe in time because of the bounce back that happens with those things. Lastly on the quality measures, about a year ago we started an executive committee on value -based care and we have some of the largest health plans, some of the most progressive employers and some of the biggest health integrated systems. We are working on trying to identify what employers can expect from ACOs. The expectations of the newly formed ACO all the way up to a mature market. One of the things as a byproduct I would love to do is to get those health plans to agree on quality measures for primary care. Right now each healthcare plan and pings them with 23 measures or 18 measures and each time we get together I said we are going to work on this. I think that is going to be some hard work and anything you can do to help me be prepared to do the right measures, to have an argument for why they should use this measure versus that m easure, we are going to try to take up some of that work.

Thank you, this is great. I did want to overload you with a laundry list, but we actually have the Academy for the integration of behavioral health and primary care and it's on our website and there are a lot of tools. That leads into the second question, if you want to let me know specifically, this is what we have done with CMS.If you can communicate with me what would be most useful for employers, I would be happy to identify what tools we would have and what would be most useful. Sometimes it can be hard to find them. The other thing in terms of measurement, I think this is one of the areas where the benefit of alignment -- the national quality measures clearinghouse sits in our center. With MEPS, TCP/IP is coming up with measures and the primary care initiative is coming out with measures, it is really an amazing resource because it has functionality that lets you compare measures. Separate from this conversation we had thinking about how we can build on that resource to make it more useful for practices and if there is an agreement on harmonized measures, somebody could go in and pull up the one they are using and easily compare what modifications they need to align the measure with what is being requested or how different measures compare. That is a resource. We are also rethinking where should be headed. It's not part of this conversation but I would also welcome input on how we can enhance and make that resource support the needs of practices.

It's very helpful to have kind of an overview and summary. I have a few thoughts about t his. What I think that is been interesting for me because I realized I was fairly ignorant about a lot of the stuff going on in primary care. That is partly my problem but then I think that is reflective that there is a lot of good stuff happening and it hasn't been as visible to the primary care community as it needs to b e. There has been a lot of effort but I think David has been a bit of a one-man show and that is part of the overarching question, how does AHRQ make this a bit more visible? That also gets back to where the center sits within centers within centers and even at that kind of level how to give this more visibility.If you go to [Indiscernible] it will list the primary care center but you can't directly linked to it. There is no hyperlink so you then have to do another search for primary care. It is little things like that that I think are reflective that it doesn't jump out of you. In the areas, I like the way you are dividing AHRQ. There is reiterating research and there is tools and resources and measures and metrics. I think that is a useful categorization. In research, the impact program, the evidence now, there have been visible important initiatives and I think those are terrific. What I think needs to be balanced is is there still a role for investor initiated research? I didn't realize there was this much opportunity. That is probably the perception if you weren't in on the impact or the evidence now, you were sort of out of the game for doing a research study in primary care. I think the more you can send a message and no that it's not just about -- I'm not trying to minimize those important initiatives, but it's hard when there is a feeling that you are not in the game unless you are part of one of these big major initiatives. If you can send that message o ut, I think that would be helpful. The tools was another thing for me that I felt badly about. There are incredible tools and resources once you get to the AHRQ tool. On practice transformation, the medical home and team -based care. How do you link this to the community to make these more accessible. I think it does take some resources. Add you get this -- I think you want every organization to say AHRQ is one of the first places you want to go before you reinvent everything. I'm not sure it is quite seem that way now but I think you have all these great resources that are unutilized. Every QIO, I'm not sure right now, the first thing you think of is to go to AHRQ. I think they are underutilized. I think that was your question, Monica. Who is using the website and the audience. I think almost a marketing campaign. It's not just enough to say it is there, you almost need social media. You need to be much more clever about how you get people to recognize it and drive them to use these resources because it's a shame they are not probably utilized is much as they could be. The final thing around measures, I think this is a huge issue in primary care. Sherry alluded to one thing, if you use traditional quality metrics or performance or patient experience you wind up with a list and then you were doing every disease prevention, care management, and Paul alluded to this, if you do opiate screening, violence screening and all the stuff. The need now is a listed integrated measures in primary care. That is now the holy Grail. There is a researcher at Virginia Commonwealth who was trying to do that. It is part of a movement in family medicine to distill it down. That is another role for AHRQ, to get out of disease specific measures and help the primary care community to come up to what would be the five holistic measures to say you were doing a great job taking care of a person, not just an organ at a time. Finally I would say the other key role for AHRQ along those lines is non- disease specific research. The problem with primary care is you still have to pitch everything as a disease specific study. Then you have to fake it because really what you want to study is whole person care. It is a cancer study, a diabetes study, but how do we do chronic care management well in primary care? It is the opportunity to say you don't have to fake it. You can say I'm interested in studying people as a whole and you can do a grant like that. I think behavior help integration, the comorbidities, that is the action primary care. How do you screen for all the things you could be screened f or? How do you make decisions about what to prioritize in which populations because you can't screen for 20 things which is what a checklist would say you need to do. That is your unique role, to get beyond disease specific and that kind of reductionist model and say how do we help primary care study and pragmatic implementation of this much more complex world of stuff. There is so much good stuff right now. I realize I have given more of his speech sentiment to that it's an education for me. There is so much good stuff happening. It is encouraging to hear there is more opportunity and I think the primary care -- and you have a unique role you can play that is different from other ones that would be valuable if you could be amplified a bit.

Thank you, very much. Like I said, I am still fairly new to AHRQ and the primary care physician and I was where some of the resources. Coming in and trying to synthesize, I had no idea myself how much is there. I think we need to do a better job of getting it out there. When people use these resources they find them valuable and useful. I think you are right that people don't necessarily know there are currently funding opportunities and that was one of the reasons for putting out the PAs. We have a number of primary care -based list serves and we have started getting calls. I talked to some of the project officer's yesterday. It is having an impact in people are getting in touch with us to ask questions. It doesn't have to say primary care in the title. That's why I put in shared decision-making and racial and ethnic minority communities. The clinical decision support to encourage the primary care community to look beyond things that just say primary care in the title and ways to get the word out would be much appreciated. I think you are right on measures. I think later we are going to be talking about future, and that is another crosscutting. One of the ideas we are floating is should we focus on the care of older adults. They have multiple morbidities and functional impairment and different goals and how you start developing quality measures that action reflect their care across all of their conditions and the different settings. That might be an area of focus and synergy.

Just thinking more broadly about primary care a AHRQ, clearly would love to have more resources. We could come up with lots of things we can do. And my center we're focused mostly on the practices and the patients but then -- is focusing on patient safety and there is a lot of ambulatory care safety issues especially as people have multiple things and drug interactions, etc.. Then we have the -- the does the organizational and systems approach and the financing. AHRQ is a natural home for primary care. If we could actually bridge these different dimensions of primary care research, there is a lot of potential. You are right, at NIH it is much more disease specific.

I'm sorry to see so few people have things to say. I'm going to me just a management suggestion. I have a list in the order in which people with their put their cards up. I'm trying to maintain some sense of order. It is tough with you guys. I accidentally skipped over M ary. I'm going to go to her into a phone check and then I have you in the order which I saw you. Then I will ask our lean to maybe say for comments until we get everything on the table to make sure we have time. It doesn't look like at that last check there was anyone who signed up to make a public comment which means we have like 15 extra minutes to hear from ourselves, which is very exciting. I will let you know if that changes. Mary?

I have a few things to say. In terms of research, diagnostic error in ambulatory care, I don't know if it belongs in your center or the patient safety Center the diagnostic errors are quite important. I think everybody has mentioned that in the previous meetings. In terms of measures, one way to measure, not disease specific but more comprehensive is to measure the core functions of primary care. There are I believe five of t hem, access and continuity, coordination, patient engagement, so want and so f orth. Those are just suggestions. I don't know if you work closely or if you have met with the primary care organizations like the AFP and ACP. They may have ideas, too. We are helping CMS develop the draft measurement development plan in with heard a lot from the physician organizations and surprisingly when we look at the primary care measures, there are more measures in primary care than any specialty. Surprisingly those primary care organizations comments to us is those are not specific to us. We want measure on our core functions. That is just a suggestion.

In terms of alignment of measures that, recently heard about the core measure set that are agreed upon. I wonder if you would make a notation on those measures in the NQMC that these are the agreed-upon measures that, and there will be more coming forward I'm sure.

In terms of postacute, I think there is a real need for postacute transformation as well but I know that is not in your area. However, the primary care role in the postacute is something to look for some evidence of what kind of primary care role should primary care physicians be playing in the postacute world. More and more, CMMIs alternative payment model are bundling payments across acute and postacute and the role of the primary care physicians in that area. Lastly, in terms of the million hearts initiative, the evidence now, it may be bigger than just AHRQ but I'm thinking when the million hearts initiative is announced, it would be nice if all the agencies kind of speak together and disseminate the information that AHRQ is going to be focused on this piece of million hearts. CDC is focusing on this and CMS is focusing on this. Right now at the ground level where we are working as a QIO, we are hearing CDC has XYZ and the million hearts CMMI grants coming out on the patient population using the risk assessment tool and so one. There is evidence now. There are a million things that coming out with some overlap but it would be nice for the public to no how the agencies are working together on this initiative and what role each one is playing. Thank you.

Let me just do a check about the phone. Lucy, are you still on?[Indiscernible], have you joined us? Jen, I'm assuming you have one or two things to say.

Yes, I will keep it brief.

Let me finish my rollcall and then I will come back to you. Paul Ginsburg? Heinrich? Patricia?Okay, back to you Jen.

I think mostly I would echo a couple ideas I heard from Sherry and Kevin and measures and -- would be a great person to speak to Arlene. The -- summit, the first of all fully many with leaders coming together -- all of us working collectively to figure out how to get the research done efficiently and quickly to help primary care transform and accelerate this process. That may help with some of the different initiatives that might help us align them and if we can have a strategy and priority as a federal agency behind that so we can make sense of what is coming from where and why -- I know that is easier said than done but I will put out there. I think the third point, and Kevin was talking about this a little bit, I know AHRQ has a small budget. There's not a lot of money for investigative initiatives. I'd like to see as much as possible funding investigators out there in primary care research thinking about creative ways to find that work as well as the many -- being done internally and get additional funding. There is a lot of creative and exciting ideas going on. Maybe it is AHRQ leading a coalition of folks within NIH and other folks to bring in additional money to that pots. Thanks, everyone. I echo much of what everyone else has said.

Thanks, Jen. Jed?

Paul made the comment that primary care is drowning and Kevin said you can't screen for 20 things at a time and there still are care gaps. I'm -- based on evidence that AHRQ probably funded to develop the first place and we can't bring that evidence into practice. Is the issue the individual failing? I would say it is the way we designer assistant to do these primary tasks which can't all be done by primary care. I think back to Kaiser Permanente which did an internal analysis demonstrating lots of PeopleSoft providers is specialty settings far more often during the course of the year and they did primary care settings and thus came up with a group agreement to try to improve the care of individuals through a group effort and people would take group responsibility for getting a lot of these things done and that calls for system redesign and every defined sense of what primary care and specialty care is and I think in the future specialty care will have to take more responsibility for getting some of these primary tasks done and perhaps it could be investigation and exploration to do that. When the ophthalmologist is -- they are not uninterested in those conditions.

I will give my personal example of that which is I received a fit kit for a colon cancer screening from the nurse practitioner who was doing a printout visit for breast surgery I was about to have.That was followed through by a number of people who saw me in that preop visit. It was clear to me I wasn't going to escape without paying attention to that as well. Once I had it scheduled, all of that harassment stopped. There was an incentive on my part. Don?

It's always dangerous to quota politician but I was particularly struck by something Hillary Clinton said in the recent debate that it's very easy to describe a problem and very hard to do something about it and I think that is the situation we find ourselves in here and I resonate with what a number of people have said. The main problem for today is burnout and overload for primary care at large. That includes the onslaught of measurement in slightly different ways from different sources but also the activated patient appropriately bringing in a memory stick with all of their records on it and to download from whatever app.There is no way of primary care provider can integrate with all the that it needs to be done so that is a description of the problem. It does prompt AHRQ to ask the question, what is the best research we can do? It may be about group accountability for colon cancer screening. I don't know. It is got to be something that looks at the primary care team in a holistic way and the patient in a holistic way through the eyes of the patient and the provider group having a fruitful interaction in a shared decision. It is nothing that wasn't described a long time ago. It is just gotten a lot more complicated. I was thinking about that team STEPPS for ambulatory care and I looked at the app which is fabulous. I looked immediately at stuff that shouldn't be done because I thought that was particularly useful. They you mentioned all the tools which I think are underappreciated and underutilized but somehow an app and team STEPPS training and those tools don't necessarily add up to a holistic picture for either the care team or the patient. How can you fashion research that will get out the holistic issue? It may have something to do with team -based care which is the point of a medical home and ask the question this team STEPPS provide a pathway for true team -based care with everybody doing their appropriate role in service of the goals of individual patients who want to have better health and well-being? That might shift the conversation away from while we have these great tools to what are the residual questions a catalog of tools can't answer? I haven't provided an answer to anything. This diagram I think is powerful and similar to other diagrams I have seen which call out the important question that even the medical home is responsible for a small fraction of what it takes to produce health. All of these links to the community and community service are really important and I still don't see that happening in most primary care practices and certainly isn't happening in the vast healthcare delivery system so there may be research about effective ways to use community health needs assessment. What is the primary care health these assessment look like? Primary care is closer to an interface than the partners in Boston by a long shot. You may be able to have a parsimonious, feasible, affordable set of research questions that would bring us closer to those kinds of initiatives.

To balance your March could you quote on [Indiscernible]?

[Laughter].

[Laughter].

Monica?

In the interest of time most of my comments you just addressed. I wanted to put another plug for the practicality of the Society of internal general medicine which is my professional home. We are many researchers and have sort of a Yoda affect on the primary care trainees coming behind us I think a lot of people would be interested in seeing a workshop and collaborating on a workshop and working closely to help get things disseminated at the ground level two a lot of practicing clinicians and ones who are leaders for those coming behind us.

Sandy?



There are lots and lots of good suggestions and ideas. I just want to focus on a process thing. In light of the discussion we have been having, I think there is a potential opportunity to look at the center in terms of bridging, leveraging and integrating. The U.S. preventative services task force is part of the program and it is also the source and celebration of some of the screening issues your are there joint initiatives the task force could identified as areas that could help or challenges they face that could tie into the research agenda?

In the difficult funding environment the agency is an and will continue to be an, I wonder if there are opportunities to meet with NIH and HIT, I reserve the right to change my mind 180 degrees during the rest of the day but two things that came to mind. See if you can get together with the various disease oriented parts of NIH like blood, cancer and have them go with the joint thing where they would partly fund research that looked at chronic diseases without specifying the chronic disease but if I came in and said I want to look at cardiovascular disease and diabetes, which now has two institutes, is it [Indiscernible]. They will kick in base what they fund type of thing. You would be the coordinating and integrating factor to make sure they would address that agenda which is also there agenda and they recognize they're looking at the phenomenon. A similar thing is HIT which I think has to be a core aspect if we are going to make information available. The national Library of medicine might help and have funding a nap.

The second comment I have, when I was -- when I see three different sets of recommendations or guidelines and what I have to do with an asthmatic to pay for any emergency room, when I see this I say there is no standard or guideline and I ignore it as a primary care physician. Controversy is something to be avoided because you only have a limited amount of time. When people talk about tools I think that is a problem because I don't have a lot of time to look at tools. I need fewer tools. Are we at the stage we can integrate in terms of some usable platform. To do that might require some thinking that is a little different and we have had it the agency as an investigative community. I'm thinking in terms of maybe having an initiative that aims at systems design and design thinking that looks at how we take what we have, and this may require a different dissemination strategy of the initiative. Maybe groups like [Indiscernible] and not the classic investigators. Maybe it is people used to systems integration outside of healthcare but have things to contribute as part of multidisciplinary teams. Those are two ideas. One is bridging integration and the other is thinking about the systems. As I was listening to what other people are saying, the challenges of a primary care physician were Paul is is different than the University of Pennsylvania health system which is very different than the people still in private practice around the country so we are not dealing with one primary care, but I think the system in which we function is the other lens that has to be brought to this.

Christina.

It's interesting to listen to this. I have been a primary care for the last 20 years with the underserved population so this is near and dear to my heart but I survey the tertiary care Institute as a researcher trying to figure out how to better connect ambulatory practices. I may take it a step further, I really think we need to be very innovative and look at new models of care and I think in pediatrics for instance in trying to serve a population of asthmatics in the state of Delaware, we connected our EMR to the school versus, obviously with parental permission. Any changes that occurred, there was a third piece. The primary care provider, the tertiary care provider in school nurse to improve outcomes. There were parts of the community not bringing into our healthcare system and I think as we start to look at those and evaluate how well that has worked, it has been amazing to me when we publish it that it has been extremely effective because it adds the other layer to access a record and make comments about what is happened for that child at school especially when it comes with asthma. The second part is I think we need to look more to more clearly at transition for care and I think that is a big piece especially as we have a population of children that are living longer with significant chronic illness. We are still missing that piece in movement to adult care. I think even in my practice, we have parents still calling us because they don't have that comfort level with the expertise of their adult primary care provider so I think even training care for adult primary care providers is much more in depth for dealing with chronic illness, for at least childhood chronic illness. The third I think as the nurse practitioner in the room we have 23 states that allow for nurse practitioners who have worked the full scope of practice. We pulled into our institution when Delaware became a full scope of practice stay, -- within 2015 we saw 23% increase in access of Medicaid patients to our practices. I think is another innovative strategy we can look at and at least measure the cost effectiveness for that.

David?

I was going to have a comment about the opportunity for AHRQ to engage the world of the non- academic, non- HMO, staff model care delivery environments where primary care is becoming increasingly important. From an operational perspective my operation is doing great in the primary care space. We have an ACL with 5000 positions. It is financial and clinical performance that is great. We are primary care doctors that lead that. The president and see -- a family physician. We have 900 primary care physicians who are employed across -- I don't think we have a fellowship trained primary care physician in our organization. We went to a resource rich medical school 20 years before that. The real world, while we are all doing innovative things in the operational space related to primary care, that needs to be studied and written about. The research isn't happening in that environment. Just to let you finish, I'm probably the only person in our organization that is how the primary care federally funded grant. The first time I submitted it 10 years ago, they called me up and said the person who reviewed your grant said if he didn't work in an academic environment you should receive funds from AHRQ. He said we did a search on the three people who reviewed your grant and we had more publications than the sum of those people. It turns out I won the John Eisenberger wealth in 2012 for that scope of research. My organization has an operating margin that is about five times the AHRQ budget and we are able to save his work is important. We funded it with our operating margin and with resources from foundations. Finally AHRQ funded it after about the 10th submission and funded some part of it. There is this huge opportunity for AHRQ to engage the large regional and national healthcare systems of the U.S. outside of the Kaiser environments and health group environments where a lot of the action is played out in terms of primary care, but that is not being studied and written about.

I just want to say one quick thing in response. David, that is actually the focus of evidence now, small and medium-sized practices and getting beyond the usual suspects that have the capacity to do it. That is clearly an area we need to focus on. Please send us names for people for study sections of get this and could review grants because I think that has been a challenge for us and as we transition and move into this area we need reviewers to get it and have the expertise for the kinds of methods we are going to be using. We are actively collecting lists to nominate for study sections so please help us with that.

And please don't stop giving grants to Kaiser Permanente.

What we are talking about is you might also look at those study sections and say sometimes it is easier -- to change the culture of the study section because there is so much continuity. It is new integrated program that gives you the opportunity. Maybe you want to look at a matrix and cut at a different line or angle. A new structure provide you that opportunity you don't have to explain.

I just wanted to return to the measurement issue for m inute. I was involved early on in the core measure set development. I want to make a plug for that or something. It evolved government, CMS, insurers and national medical societies, Rob a blue less well but still involve consumers in patient groups and businesses. And to make a plug that hopefully we can all focus on that because the last thing we need is two or six different groups saying we have got the set everyone needs to use for primary care or cardiology. My own state, I was involved in the national level and some people I were involved with were -- we want to use this. We will still confuse people because the nationals will be saying to use this. I think it is best we can all figure out one place especially since we don't have the resources to duplicate efforts. What I think AHRQ can do is the new measures because the core measures we are saying we can take existing measures and decide which ones to use instead of six different cardiac measures or whatever. What AHRQ can do is those whose measures and the comprehensiveness of primary care, the things we want to measure primary care and would love to put into a core measure set that no one has come up with yet.

Bob?

I think my comments are really a follow-up on what Sandy was talking about. The person on the phone as she was finishing said something about we need to expand resources for investigators and what we can provide to them. In that vein, I was going to suggest the same thing. How can you leverage funding with other agencies? We struggle with so many of the same kinds of budget issues in the office of research and development at the VA. We are doing some things for example within NIH on aging in place so we will have some funding with NIH on that. It is never easy because it is all the memoranda us that have to be signed off. We have one complimentary and integrative health. I think there was one in development on opioid abuse. What we end up doing is like Sandy was suggesting.We participate in the review. We provide some money and then when the projects get selected, they sort of get split off. If it were us for example, we would have to do the VA and NIH picked up the others but then we all get credit for the work that comes out. If there were five projects, we would get credit for five. Well we for example have to focus on veterans, we take great pride in having more generalizability. If it were something like complementary and integrative health or something like fighting infections. Infections are the same in the VA hospital as there are any other hospital.

[Audio Faint/Low Speaker]

We have been talking to the VA about partnering on some of our HIT work because there is a lot of parallels.

David Atkins was interested in that part.

You have got Patrick and Kate in David and Carolyn at the VA and -- you can strike while the iron is hard because these people know which other and like each other. -- is very broad-based and his research group includes primary care physicians and medical sociologists and philosophers, everybody. I think there is a good opportunity. You have leadership in the key agencies that can do that. Mike at NIH I think would be open to this thing.

We also have a number of centers of innovation and one in Philadelphia. One of the things I find really attractive is very few of the investigators are 100% CA. They are VA and also linked with some academic center. You get the best of both worlds here you mentioned before about mental health. The VA has a very extensive program looking into integration of mental health in primary c are. Maybe that is something to start off of. The last thing I would say is just from my experiences working at the VA in the deal deed, the DOT has actually taken up some interest in health services and maybe that's another place to try to leverage --

We actually met this week with the DOT. There are a lot of new synergies emerging.

I would just put one thoughts on the table. I think this is been a great discussion and hopefully our lean now even has a longer list of things to do with less money but just a couple of points. One is back to the patient reported outcomes. The observation I want to make is I think the easy part is collecting them. The harder part is integrating them into clinical care. We have actually done some work in the area of pain measurement and integrating the brief pain inventory. The thing that was key was to find a way to make the information accessible to physicians in a way that was almost like a new lab test with everything else they were looking at and in the way that didn't require them to be survey researchers and no what the scale meant and didn't require them to open up a separate PDF with the score on it. I think where we get excited about getting the idea of the patient voice in a more systematic way in, and we actually spent probably more time and making the systems talk to each other in putting in the information and making it available in real-time to physicians. What we learned from patients is they were very willing to give us this information if they thought there physician was going to see it and use it, otherwise don't waste my time. That leads to my second point. We talked about the profession and about researchers at I also think in primary care continuing to find ways to hear from patients with their hopes and dreams are for primary care and how they see it as integrating or not integrating into the ways in which they are trying to pursue health and what kind of support we can give answer what kind of mechanisms to patients so as we pay more attention to the health journey, I think we need to be quite humble about the role any kind of clinical care, whether primary care or specialty care can plan that. There might be opportunities to facilitate people's journeys. For instance, we did a survey recently as part of the -- project of our patients who are overweight and obese. In interesting thing was most of them said no one is ever talked to me about my weight problems. This is something that requires a lab test necessarily. No one has offered me ideas about what I could do. The things they were most interested in were lifestyle interventions. We asked them about that versus drugs versus bariatric surgery and other things. Thinking about what patients want from primary care is something that also might be worth trying to work in. I think sometimes we get far down the line in our models of how we think things should be organized and forget the folks who are meeting those services. I would look for way to bring that perspective in.

[Captioners Transitioning]

Partnering with other agencies and seeing if I can help with other agencies, another example that Bob talked about, transforming clinical practice initiatives. On the patient engagement side, there is a whole network that patients are doing around how to engage patients an individual care and in the practice transmission space. There will be no evaluation of that. It will just be folded in with the whole evaluation program. What a perfect opportunity to do a pragmatic cluster. We can look at patient engagement and if it led to different outcomes. You have to strike thickly. When there is an opportunity there that they are funding, that is less than money required to say that we could just drop in some way of doing this as a pragmatic trial of understanding what patient engagement means. That takes nimbleness. What is CMMI doing? Can we layer on something from a AHRQ perspective? You are doing the whole shebang. It is as if someone else was doing practice coaching. Could AHRQ make sure we are evaluating that aspect of this big initiative? I don't know if it is too late. The more inter-agency, the less costly it would be.

The other thing that strikes me about that, and we might talk about this more in the afternoon, and I think this is a two edged sword. If you think about the interagency collaboration, and I know David will touch on this, there are two questions you can ask yourself. One is what is the unique contribution AHRQ can make? That can help to further articulate the case for AHRQ. We heard a good example with the primary care up against NIH. These things work together. Thinking of it from a collaborative perspective, the challenge is to do that in a way that does not add to the charges that are duplicative of what is done in the other agencies. Finding a way to articulate where the unique opportunities are and what the particular skills and contributions are that make you a valued partner. The notion of leveraging the resources do not -- does not help us to have new discoveries and new clinical recommendations if we can't put them into practice. I do think that is something to think about how it could be done more effectively than I suspect it is done today. Sharon and I had an exchange about AHRQ is the mouse that wars. It is trying to get the big kids and the bullies to play well with each other. I still think it is a worthwhile mission.

It is important to appreciate the risk. You don't want it to be an excuse to say don't fund AHRQ to do this stuff when there is money and all the other agencies. It is a little delicate on how to do it in a way that does not diminish the budget.

When Sandy and I were young men, we used to go to the hotel where I stayed last night and we reviewed grants in the late 80s. I remember we had a grant where the priority score did not make the funding level. There was someone sitting in the review and that person picked up the grant and funded it. I don't think we see that kind of stuff today.

It is a unique opportunity. You have primary care leadership now with the VA and CMS. Even though it is a specialty, someone who is very primary care oriented in NQF, this is a good opportunity. There is nothing unique about primary care that the leaders of all of those organizations should be doing. You have allies and supporters of AHRQ. They are all on the record. They have all been important. Several of them are ad hoc members of this group. I think there may be a window of opportunity.

I understand [name indiscernible] is on the line and has a comment.

Everyone has been mentioning fantastic comments. I want to go back to what was mentioned about what CMMI is doing on collaboration around patient and family engagement. If you are not aware, CMS does have the divinity group that is started that is being championed. One is a patient family engagement divinity group being led by [name indiscernible] who is deputy director in the quality improvement group at the Center for clinical standards and quality. I am happy to put her in touch with whomever on the MAT. They are doing tremendous work that AHRQ could benefit from. There are affinity groups on behavioral health . We are thinking about how to push this more as it relates to the reform population health base -- health-based payments and measures and overall framework and approach to looking at non-disease specific issues that are impacting population health improvement. I am happy to provide that information if anyone is interested.

I would love to know who to contact. I should let you know that Darren to Walter has invited David Myers and myself to participate in the weekly calls of his L and D group which has been helpful. We shared some of the work we are doing and we are aware of what they're doing. We are starting to develop those partnerships. There are other people we should be contacting, I would love to know that.

Absolutely. I will let you know.

In the interest of lunch and photos, I am going to bring this to a close. I am getting guidance from Jamie who will correct me if I get this wrong. Photos before food is the motto. We are taking a group photo and individual photos. We do that first just off in the atrium here. I'm sure you will see the paparazzi set up to do this. Just had toward the paparazzi. We will do the group photo first than the individual photos. Get lunch and come back and we will eat in this room and then we will start again at 1 PM. We will do our best. Thanks.

[The event is on a lunch recess. The session will reconvene at 1 P.M. Eastern Standard Time. Captioner on stand by]

We were hoping to think and imagine that there were additional resources you saw in the budget. There was some interest and proposal to restore it back to a level of funding. You can think about that as new funding. On the other hand, you could think about within the resources that we have available, are there areas I should be getting more attention than they get today? I think we all would like to be in a world where the challenge was how to spend more money. Both things are on the table. Because we are doing future visioning, I will not ask you what you would give up in order to do the thing you think is most important. For myself, when I saw the things that you had to cut in order to hit the budget target, it was shocking. I know those weren't swaggered programs.

A number of topics that I suspect that I thought about previously have come up this morning. I think what we will do is put some bullet points out about the things that we think should be on the list. They were in the context of specific discussion items. I think they are important to bring back here. Then we can open it up for everyone to make a contribution to as well. I know David may have the biggest challenge. David, are you there?

I am on the line.

Why don't you get started since I don't know how much time we have you available for. If you would like to go first, that would be great.

I will be brief. We talked about a lot of these things. One point that I wanted to -- under score was how to best employee practices and do so efficiently. We will see what approaches will fit environmental business models. What types of financial and other incentives might be most effective for influencing that behavior of physicians and other healthcare providers? Which of these issues makes sense for AHRQ to address? We covered that. I think some work could be done on accountability thresholds and whenever of financial or other incentives are associated with clinician behavior changes. We need to identify which tactics has the greatest return. Some of the studies as we discussed earlier, you probably want pragmatic organizations in the advocacy [poor audio quality] . Those benefits from this continued augmented benefits helps the organizations messy population health and models may be the way to go in terms of transition from quality-based healthcare to private insurers and larger risk employers and integrated works. CMMI and NIH is beginning to fund some of the work. I have heard from many others. [poor audio quality] I think it is a much more mature -- mature approach from the AHRQ side . [poor audio quality] Those are some initial comments.

David, just to push a little bit on one thing, in the notes that you sent ahead of time, you had talked about the importance of coordination across the agencies. You may have just said it as you were talking, the sound quality is not the finest. I apologize. Could you just say a bit more -- I think this was based on a lot of the experience you have had with different federal agencies over the time that you have been in the field. Your thoughts on the coordination issue?

I'm not pointing fingers at any individuals. As I shared earlier, Sandy and I looked over grants in the late 80s. It seemed to me there was more funding collaboration across federal agencies. I was really struck during my two-year term as chair of the challenge that it was to engage other federal agencies. We had great difficulty getting someone at a leadership level to show up and have a discussion. I felt very strongly that there ought to be co-funding. For whatever reason, that is very difficult to achieve. Hopefully there is some future state down the road where there is clearly a distinct role for AHRQ . There are collaborative efforts among AHRQ and other federal agencies and they are truly partners in co-funding some of this.

That is helpful. Sit on the board of a small foundation. I think that because it has so little money, kind of like the AHRQ situation, we pay attention to leveraging the money that we do have through partnerships. I think they have been very effective in those strategies. It is an important point about the matter with the agency is focused on, how to look for those ways to make the impact even greater.

Beth asked us to think about what we could do with more money. A whole raft of notions came through my mind. I whittle them down to two teams. I am coming up on the beginning of my 30 year of NAC membership . I was coming at this issue of coordinations not from the point of view of funding, but more of the missions aligned and the rules -- roles and responsibilities clear. Sometimes it is unclear who is doing what. I looked back at the AHRQ mission stated on page 6 of the directors report, and I don't see a reference to what has been stated as the coordinating national quality strategy amongst all of the agencies. I wonder whether that is more in principle than it is in practice. If you brought together the mission statements for all the other agencies, CDC, NIH, etc. would you be able to take those mission statements and understand clearly who was doing what and how they were coordinating? My suspicion is no. That suspicion started when we heard a presentation on healthy people a year or two ago and there was so much overlap and confusion about who was doing what in setting the schools for the country. I think that does need some work. Perhaps if they got worked out here, it would not be quite as confusing on the frontline when you spend money on your implementation initiatives. The second bullet point was all-around HR IT. That is why I told Arlene she had come back this afternoon. This agency should double down on each IT research and implantation of the findings. When you think how much money has been spent on the implementation of HIT for meaningful use , plus all of the private money, plus all of the money you can see in every organization being to put to play spent to do the same sorts of things to get better population care management. Then, when you read the literature that says 80% of alerts are ignored, we wonder if we're getting the bang for the buck. It would be nice if we could not demand standardization from the vendor side, but at least have modules that would work with each vendor that would achieve the aim of a screening and recording data on cigarette smoking and cessation activities. All the things that we are supposed to be doing that everyone is trying to do and collect and we could make it again play if that is not some technological fantasy of mine.

Are you trying to overthrow the whole system?

Yes. I think we have to spend money on HIT to get the value of the investment that we have already made. So far, we have medical records that are legible . That is good, except for the copy and paste stuff which makes them a copy and civil. We have a lack of interoperability. We don't have the information following the patient through provider groups for different specialists or different geographies. The clinical decision support, which was the original rationale for the ROI of the systems has not been implanted and proven. I think we have a jump on it.

I will add my three bullet points and then open it up. The first is that I do think we lack both the conceptual models and the classification systems and the data infrastructure to really understand the role that the organization and financing of care play in what we have -- observed. We are locked in definitions and models and data that is so 19th century. I think we need a refresher on that. It is a challenging area, and I have done some work on this characterizing the nature of physician group arrangements in Massachusetts where it is not a one and done activity because they continue to change and evolve. Certainly, they continue to change. Ways that can improve our ability to understand the organization of care from the micro to the macro level. I would say that even within organizations such as the one I work in, there is tremendous variability in terms of the way that care is organized and that care for common areas is delivered. We have a hard time capturing that. Having common systems that would let us categorize that so that we could understand what works best in terms of systems of care, I think that would be a really helpful thing. The second area is in the area of diagnostic care. I was on an IOM committee that led to a report of improving diagnosis and healthcare. I think that there is a whole recommendation related to the research that is required in that area. I think that is research that AHRQ is well-suited to do. It is not exactly the same as patient safety research. It is interesting. The committee -- I think there is a tremendous amount of work that can be done in that area. It runs the gamut. I think it is a hugely important area. It is an underappreciated problem. It is a problem -- it is particularly challenging because if we don't have the right diagnosis, much of the rest of what happens could be wrongheaded. It is so fundamental to good care delivery. With the explosion of tools and technology and the diagnostic area, I would say things are getting worse, not better. The only reason we don't know that for sure his we don't have the research we need to understand the exact size of the problem let alone whether we are making progress in any of the interventions that we might imagine. The third one I was going to put out there as a controversial.experiment. I live the other side of this which is what if it does not get reauthorized. The question is, is there any part of that portfolio that AHRQ should be raising his hand to step into? Should that happen? I think it could be a part of the discussion about the collaboration with P Corey, but we could also be imagining that world and what the things that we would want to make sure did not go away. Those are three ideas that I just wanted to put out there for thinking about future directions. I will also say one thing on the PCORI piece, which is I do think that one of the real contributions is really trying to make much more concrete the notion of engaging patients as partners in research. That is a very different way of working. It is not easy. We are still bringing much of the research committee along with that idea. Those who have had successful experiences really agree that there is no going back. That is the part I'm afraid we would lose. Many other federal agencies and industry is beginning to see the value in this. We are missing a group whose main mission is to push that idea. I fear it is to new -- two new. That is the kind of thing that could be adopted by AHRQ. I think you are well-suited to take on some of that and even if PCORI stays around, you want to continue to engage in that area. With those opening thoughts, I welcome hearing from everyone else.

If I can frame some of the questions that would be help for us to focus on, we are coming to an election. There will be a new administration coming in. Not only telling our message to the outside world, but talking with in the administration and what are our unique strengths. Where should we argue that AHRQ could be a major player in solving the health care system problems. It would be really helpful to get a sense of you as to where you think AHRQ should take a leadership role , given everything we have talked about today. I know there are lots of issues. It is not only what research we should be doing, but how do we argue to both the new administration, a new Congress and the world. What is it that AHRQ should be taking a leadership role in? What -- where can we be important? I am hopeful that if we make the case for that, we make a better case for our existence and we make a better case for or budget.

One thing that resonated with me was when David mentioned evidence-based medicine. Right now, we have AHRQ participating in our evidence-based committee. We take the research and suggest how large self-insured employers who make their own decision about planet design incorporates those into their options. Some folks haven't covered autism before. Some folks haven't covered infertility before. The research shows that if they don't, then there is other expenses and lost productivity. There is no births, etc. If we can make sure that we take the evidence and we incorporate it into plan design, we know that that is what gets covered and how doctors will generally do treatments because that is the way it gets covered. We can incorporate that into the plan design, then I think we will continue to see evidence-based in the delivery system.

What would be helpful for me, as I think about this as we have is transparent conversation as possible about the political realities and the changing administration, it would be to have a clear sense to the degree that we can discuss it. Some of the challenges that AHRQ has had with communicating our mission to a broader audience, particularly with Congress and the people who are funding us, they need to help figure out the best strategies and then overcome some of those.

I will start and I welcome other spots. I served on the Academy help board for eight years. To me, the challenge that faces -- AHRQ faces is one that Academy health has don't with quite a bit, which is trying to make anyone care about health services research or explain it in a way that someone other than someone who does it can appreciate. Academy health did this interesting work with a communications firm several months ago where they talked to health services researchers and ask them how they would describe their work. There were 30 different terms that came up. In the current environment of staying on message, we can't even have common language among ourselves to describe what it is that we do. We have not been very good at saying what a difference it makes. Why should patients care? Why should doctors care? Why should your mother care? I think of it as a wonky endeavor. We have not spent a lot of time trying to get help and I'm not sure we are the best to do the communication thing, but we have not gotten help with people translating why it is important. I think that is why AHRQ has been working on that messaging about evidence and teaching and trying to begin using what we know about modern communication methods to say what it is that we do. The second challenge is the belief that what AHRQ does is duplicate it. I think there is very little understanding of the continuum in whatever model you want to use from a basic science and discovery research to getting to the front lines and thinking about the particular agencies that plate rolls along that continuum. I would say in challenging budget times, and I don't think that the federal government is that unique, I think there is a tendency for people to make sure my agency's first. I don't care about my fellow agencies. I think there has been a little bit of a playing off of how we best spend our money. The other thing that I've seen in the political dialogue is why isn't the private sector doing this. When we talk about the way the healthcare system is operating, one of the political responses is why is that a federal responsibility? Why is that something the private sector should be funding? I will end by saying that it is political. It is not that all of these conversations are the logical. I am an overly logical person and I think there is a logic we can bring to this. I think there are things I have talked about. I think there is a piece of this that has to do with the current environment and the ways in which budgets and priorities get set that probably is not about logic.

Just to make sure I understand you, it is not so much that there is certain topics or areas that we think are critical and key like health equity that have some politically charged items, but it is more so an inability to translate very complex things that we do into common language that is accessible to everyone.

I think the translation thing is underappreciated. There are other issues. I think that even some of those could be managed more effectively if we could learn to tell a better and more effective a story about why our work makes a difference and why it is essential in the continuum. I don't think we should be pitting basic science versus translational work. The reality is that we don't have both pieces. We don't get the benefits of any of those investments. I think the other thing is having been in two organizations that are trying to say something quantitative about the impact that research has, it is not a straight line. It is very rare that you can say I did this piece of work and with this one study, these seven changes happened that saved millions of lives. We have a few of those and we go to them. It is such a compelling story because it is so easy to tell. Most of the work we do is not like that at all. That is why we have things like meta-analysis. We would not want people to be reacting to a single study here I do think it is the complexity in helping people understand that it takes a body of work to move the system along. It is not about having for big studies a year that all the money goes into. That would really solve things. It is really understanding the scientific process. A lot of the work that we do is not quite as sexy as discovering a new gene, even if it does not ever translate. There is stuff that gets the imagination of the public going that seems magical. In our work, maybe we need to hire Disney or someone to help us with the magical part.

I want to say one last thing and that I won't say anything else, I promise. We might want to consider doing what Academy health did or bringing in extra people who are in advertising. They could help us with the mission. I think that we all love evidence and data. We might want to shift a little bit in our tagline for public messaging purposes. People like a convincing narrative. If we are trying to have clarity internally, that is one thing. If we are trying to make sure that we are selling it effectively, I don't know that evidence drives a lot of passion. You might want to think about bigger framing.

I do think we are trying to have a lot of discussion about this today. The internal making sense of and thinking about is a position to do the most important work. When we think about selling it to the Congress, then we need to go in to marketing mode and not integrate that. That is not our core confidence. I will leave you with one other piece of research that I am aware of which was talking to consumers about their reaction to evidence-based medicine, which was isn't that what my doctor has been doing? It is hard -- I'm not sure we want to tell people what we really know about that. Alice? Just to piggyback that for one second, when you are thinking about maximizing the quality and accessibility and the affordability of health care, instead of reframing it from saving billions of lives, I look at it more like living better, longer. We are talking about chronic illness. People want to live better, longer. With a tighter budget, we are looking at partnerships and maybe there is a working partnership with the business school that has communications and PR. My board members will say clear, concise, elevator pitch. Otherwise you will lose people.

70% of people out there make their decision in the first two or three sentences.

I had a second question, which was when you're talking about diagnoses, can you expand upon that to the extent of when you are talking about your three bullet points and diagnostic care?

It was in the area of errors in a diagnosis. The work includes -- it is an all in approach. Diagnosis is a complex process the committee worked out a conceptual framework for that. There was a time during the committee's work that I had that how do they ever get it right moment? I learned that it was not just about where diagnoses. Even the common diagnoses can be missed but frequently. It is just something we have not paid that much attention to, starting with medical education and training all the way through to practice. It is an area where -- I gave a lecture on this the other day. Listen to your patient. He will tell you his diagnosis. We have lost the time and the art of listening. There are some things about how we more effectively engage patients in the diagnostic journey. There is a ton of research that is basic research. There are multiple data sources and most of those data sources are not obtained in a way that helps you understand the denominator like how that case even came to be reviewed or whether there was a diagnosis error or not. That is not unusual. You actually need to rely on multiple sources. It gave you multiple perspectives on other things. I won't try to summarize the whole report. I think it is -- the committee try to frame the report in the context of improving diagnostics and healthcare. We thought it was about avoiding errors that we would miss the boat on what the opportunities were. I think there is a lot of research to be done on how we can develop systems and how we can train teams. A lot of it is familiar stuff that we see in the quality world. The stakes are high when we are trying to get the right diagnosis. That is what that was about.

From an autoimmune standpoint, from the patient's point of view, when they enter into the system, they are going into -- it is very siloed. For celiac disease, it would be a gastroenterologist. Rheumatoid arthritis -- if you come into a rheumatologist and you happen to have whatever your symptoms are -- I saw it talking to medical school students. That was something that when you're looking at chronic illness and early diagnosis, it has the siloed approach. It is something that I hear again. We have not done a survey in this area. We have had other surveys, but that is what we are hearing from the community. It is so siloed. Your thinking from your point of view.

I have a couple of meta-thoughts. One is, if I had $35 million I would not start a new program. I would try to strengthen the programs I had. That may mean swapping out a program or redefining a program. Unless I was sure that the funding was going to be there long-term, I have seen organizations get into trouble when you give away money. There is room to expand with the doing and incorporating things that have been talked about. If I had to define a new program, -- I'm going to use a term differently than it is currently used. I call it implementation science with an emphasis on science. That means that you will develop evidence as part of that. Then you won't stop there. What makes AHRQ different than much of the rest of the NSF or NIH is that we are not satisfied to discover new things. It is important that we then go and get them implemented. That is how patient care and outcome in the system is improved. It may be rearranging chairs on the deck of the ship to some degree. I think it provides a focus and it may have a salient message to understand what is different about AHRQ and some of the other things. I wouldn't depend on Academy health or any other organization to communicate the message of what AHRQ does. Health services research and AHRQ overlap, but they are not the same thing. While they are each crucial organizations to the other in the image of constituencies, it is too important to be left to general. I think about it and I think the implantation science also highlights the importance of the communication piece of risk communication and how we communicate information. It would have to involve IT. It would have to involve systems of healthcare organizations and how they are involved today. It may be an organizing principle that spans a lot of what AHRQ does. The other thing along those lines with what to do -- I hate to say this. I think it would be a disaster, policy wise and for the health care system. Hopefully there will be constituencies that would buy this. I think there has to be some contingency planning -- the second thing is there is money from PCORI that comes in from communications. You get the transfers to support research and communication, right?

We get the trust fund, but PCORI gets 80% of the trust fund. We get 60% for dissemination and implantation.

The question there is, you can't go wrong on government policy and its application. You can't do that. You have to ask us before you do that next time. Learning about -- I have done work on risk communication. I work with people in the business school and marketing -- I think there is something to learn about how to be more effective communicators of the information. I wonder if some of that money could be focused to study how to improve the dissemination process. We could put a science center in. The third part is given that what Tran might -- AHRQ my do if PCORI was not funded, the impact would be less money available, not more. It would be solved in an urgent need to figure out how to further refocus. It may be premature to think about it at this point. Hopefully it will not come to that. It would have profound impacts across the federal enterprise about how we are developing the strategy for developing rigorous evidence-based for we do. Those are the three things that I was thinking.

I have what I am afraid will be a long and discursive comment. First, it a short one. I have an aversion to the science of implementation and dissemination. What is missing in most people's mind when they hear that is scale up. We are really bad at going from implementation of a prototype to scale up. Dissemination is spreading knowledge, which does not necessarily lead to scale up. Here are three dangerous myths that I think we have to counter. One is that legislation, even well-intentioned and beautifully crafted, will lead to optimal care at the point of care. That is a myth. The value-based payment will magically lead physicians and care teams to provide the best possible care to their patients and improve the health of the population. Three, an entity that is responsible for the basic science and the genomics and development of a new Biologics and doing experiments in rats and experience at the bedside can also do implementation and scale up and dissemination. That is a myth. There is no country in the world that has such an entity. We certainly should not. It is asking too much. There is too much imbalance. Those are three myths. I think the purpose here is to have patience and their providers feel that we are not wasting their money. It is their money. It is the taxpayer money. We are not harming them. As Bob says, we need to give them the care they want and need when and where and in the form that they want. That is the unique contribution of each arc you -- HRQ. Provided, it is acted upon in an effective way by the agency. The confusion is around things like CMMI and its current basket of primary care demonstration, for example. I know that contains an evaluation component because there is an ID IQ on the street to help with that. There is an open question to whether HRQ is the best agency to help with that evaluation using sound health services or whether CMMI can also do that as well as being the policy and financing arm for the demonstration. I'm just not sure. That is where collaboration could occur. We talked last night briefly about the example of the Pennsylvania rollout of medical homes where quantitative analysis was done by investigators at random and a qualitative analysis was done and funded by HRQ which used a positive deviance approach to show why some practices were succeeding and some others weren't. I thought that was an ideal partnership that could be replicated . What I am not sure of is where the data functionality of HRQ fits into that. It always felt to me to be an enabler for the research. It is also really different. I can more easily see how IT, in the service of more effective care, is part of one whole thing. Having large databases is a little bit different and has to be thought of differently. It is explained differently to people. Then there is the training piece. I think what could easily be lost would be training the people who can do the effectiveness research that is needed to ensure the patients get the care they want and need when they need it. That is always the primary thought when I think of HRQ. I don't know that people in general realize that it is the pipeline of investigators. We are trained by a large by HRQ money and resources. Those are just some thoughts. I see most of HRQ 's work to be on the far right hand of the translational pathway. We need to let the people know how to develop cures for cancer. There is one crisis here that I want to point out. I am certainly advising every fellow who will listen to me who is interested in shared decision-making. The administration and their wisdom has decided that personalized medicine is really important, or precision medicine, whatever you want to call it. Of course there is confusion because he health service research community, we think of personalized medicine as informed, shared decisions. That is really different. I don't know -- I don't hear in the billions of dollars that are being spent on precision medicine attention to what the conversation will be between doctors who don't have a clue as to what this is all about and their patients who called out miraculous visions of how they individually will be cured.

They and all of their descendents.

Right. I think that is a huge opportunity. The entire agency could be funded on what it will take to do that well. It won't be done by NIH anyway.

Kevin and then I will go to the phones.

Politics is a funny business. I would be very careful about how much we think we are going to be able to shape that were give you guidance. Why is the NIH budget $30 billion a year and the AHRQ budget is $3 billion a year? Do you think science research has any better evidence or any better communications about it? It is the culture. It is the political forces. It is interest. It is not that they just had a better messaging campaign. When AHRQ and its predecessor was formed, it had the worst political consequence. I think it is good to message. Think that Sharon, some of the stuff that you said and being able to re-articulate that -- there is plenty of amazing stuff. No one else really does that. The NIH has implementation of dissemination. It is not available.. I think it is more complicated than getting the message right. The message, what I would suggest, is that it is what is it that HRQ does uniquely ? You need to make sure that is what is reiterated. We had several things. I think it is what Donald and Sandy talked about. It is more on the implementation and that side of translation. The system's translation. No one else really does that. The NIH is trying to do fermentation dissemination. It is not at that level. There is no place else where you can go with non-diseased-based research. Maybe there is some other agencies. I would really look at what it is that is unique. With legislature, it is a bit -- everyone can do this. It is being able to come back and say this is a unique thing. Who else is doing the disparities report? It is both the reports and products as well as who else is doing this kind of inquiry and research. This is what makes it unique. There is no agency that has a specific charge around research and primary care or even some of the application work, maybe CMMI a bit. I think no one else does financing payment issues. There is no one else in the space. I think I would try to articulate those. The other missing element is workforce. There is the Bureau of health professions and HRSA. I guess they changed the name. It was the birth -- Bureau of health workforce. It is not research. It is training grants that is another area. There is probably more than I have not mentioned. I think it is trying to very clearly say this is what is unique about HRQ and this is what would not be done. I agree with Sandy. I'm not sure if there were more resources that it would launch the next big initiative. We have had so many things we have talked about today. It would be more investigator driven grants with emphasis. I think you can do this emphasis. It is a general thing we think is important. As we talked about it, how do you get the tools that you developed into people's hands? That is where I put some of the investments and how to translate this and engage the stakeholder communities that are. I am not sure my first thing would be to launch big initiatives that were totally separate.

[Captioners transitioning]Is there anyone on the phone on the committee that would like to add to the commentary?

This is Lucy, I would concur with some of your earlier comments you made about Kaiser and Lucy and I was just wondering, I know in some early work I did with AHRQ with the patient center it was a dissemination plan and I'm wondering if in some of the AHRQ RFK we could put out some requirements on dissemination that might include a story or something that could be used in communications but Court at the point at which the research was being done, how it is being used. I actually am the vice chair of the committee of Academy health right now and I do a lot of congressional staff and representative meetings and one of the arguments and make with someone on the value of AHRQ is would you make a major investment like the investments we make for the kind of research that is supported their without assessing the value of that? If they could be a line of research on the ROI on the new evidence-based devices and drugs, we talk a lot about the number of devices and drugs that have been recalled because of the kind of research AHRQ does hasn't necessarily been done in the real world in terms of implementation took it has been talked a lot about. The next area I think would be really important that we could make a huge contribution on that affects policymakers, payers, clinicians, is to really put a lot of effort into thinking about understanding some of the patients reported measures. I know in the CCK are episodic bundled payments there is a requirement for use of commerce -- measure and a lot of these patient reported measures we have a very limited understanding of what is a clinically relevant change in that patient reported measure? Are the thresholds? From a research and improvement perspective there is limited evidence on a number of reported measures in terms of the validity of looking at them longitudinally and what the [Indiscernible] is collecting that kind of data and really understanding those measures as we start moving to a population health perspective that could be used across the continuum from use of community-based resources to the healthcare delivery system.

Thank you. Sandy I know you have to take off so will go back to you.

Thought about the communication. We need a clear message what people have been talking about is exactly the message. If you look at the top 25 -- NIH model some stuff is in state three but by definition most of what they do is proof of concept, the ideological course and randomized trial, and then it has to be proved in the real world, brought to scale, the patient centered brand so to speak, but everyone's to do the research evaluated in the real world and look at its impact. I think one of the things that we also have to do as a community is to demonstrate the value in a tangible way and I have always been impressed with how impressed Congress is in having some metric that's meaningful to them. People could estimate and recognized all the vagaries of doing it but it doesn't stop other scientific groups from doing it, about how many lives have been saved because of safety initiatives or quality initiatives to translate those in two years of lives saved in comparison to other things. Some things never make sense to me and I will go to my grave not understanding how can spend the amount of money they do on trying to come up -- with a treatment for a particular type of lung cancer and not spend a small fraction of that money to prevent people from smoking to begin with which would save many more lives . but I think we can want more of these comparative methods not too sure that somebody else isn't doing a good job but that we are doing something that is extremely valuable and has better payback. I think need to be able to generate some of those metrics.

But in all fairness Rick did a splendid job and that is what you get for the whole campaign around the budget was the patient safety, the number of lives saved, so I think you have done actually quite an effective job of saying here is the human consequences that Americans have benefited from in a huge way. I am just not sure -- I think you are doing a good job at that I am just trying to be realistic.

The other thing is I think it is almost like talking to your kids when they are teenagers. You have to save a lot of times to get noticed. To make it part of the landscape .

It's even better if someone not you says it.

Yeah, somebody who is in you the parent at least always want to roll your their eyes as soon as you start to say something . [Laughter]

I'm not sure who you met with in Congress [Laughter]

We've done a really good job with everything and I think it needs to be sustained and built upon and tied in with where the agency is going to be coming in terms of its priorities and its vision for what the impact is going to be made. That's what I think motivates -- like you were saying you don't want to be the parent but it's really important for the stakeholders, the providers, hospitals Association, the research community, to let people know how important AHRQ is. Every government agency will argue for its own existence and survival but I think have some unique opportunities. Again AHRQ has done a really good job over the last decade or so of building stronger relationships with the stakeholders in the various communities that AHRQ seeks to serve and inform. And it is going to be important. I have to go because I have a friend who died yesterday and I have to get back for a memorial. But -- anyhow. To meet next? July. Have a good summer, have a good spring.

Did your comment?

Somebody ate the last cookie. [Laughter]

As an outsider, if you think you have a hard time trying to sell data. You would think people would want to know what the hell status the United States is but that is another story. I really do agree with the comments about being unique and aiming at that. After all one organization has been very successful in promising future. That is an easy sell. The agency that my standalone group sits in is needed when things are wrong. And people want fast action. We have another organization that is responsible for whether those medications, if in fact you take them, work. I am in my 70s now and I think I am in excellent health. But I have to schedule my work around my physician visits it seems and I have to have a list of all my physician's names and when I'm going to be visiting them and why. I think Congress and others either they are in this situation or their parents are in the situation and I think they can be convinced very easily -- maybe not easily but they can be convinced that the existing system isn't optimal and that we need to be able to fine tune it. I think you can make that pitch on an individual basis for the system as a whole. And public health hasn't helped out. That has been my area for well over four decades. We have either pitched the idea that it is all your personal -- if you would take care of yourself and do these things and eat these kinds of things then everything will be fine and then we say but if that doesn't do the case then at least you have to have health insurance so you get into appropriate care. But as we know, we just came out with something this morning that was on the third page of the Washington Post and had to do with life expectancy and showing that it is not going up in one very interesting group of women and going to be coming out with several other reports and is a perfect -- preventable types of activities and the system itself as -- needs to be fine-tuned for the knowledge we currently have. I would be very willing from my end to try to provide data or provide reports that the document more issues of preventable morbidity and mortality in groups that one wouldn't normally think about it. Maybe that is where we could shake hands. There are some other things but still I do think, don't give up. This is needed staff . -- staff. -- stuff. And I think it can be sold .

I wanted to react best to something you said about the issue of -- sorry I lost my train of thought -- in talking about second opinion or misdiagnosis of treatment. One of the biggest trends on the employer side is to incorporate eight navigator adequate for a second opinion program into the health plan. And we see when employees to that where they have a difficult diagnosis or even eight standard diagnosis but just want to make sure that it is correct that a third of the time the diagnosis is changed and two thirds of the time the treatment is changed. So it works. The situation though is that very few people use it. So they just assume that Dr. knows how to diagnose everything and even rare and chronic diseases, complex diseases, and so I'm not sure if this is a project but we know that when those things happen, and you have an expert navigator or someone who can help you, one employer calls it a cancer shopper . regular breast cancer, or if you have something very rare they want to get to the right facility or a center of excellence. It certainly is about trying to improve diagnosis over use.

So is something to think about.

I apologize for having gotten us into the communication realm. The conversation really was to think about how AHRQ can most effectively use the resources it has today or should there be a happy day when they get a little bit more, having a plan for that. And so I think, and I guess one of the things I would like to reflect on what was I think it was easier for asked to talk about when we were doing a deep dive into the primary care program and I think this is kind of a theme we might return to but I think you might -- we actually gave you more constructive thought in the context of particular programs so I would urge that at the next meeting we think about a couple of those areas and that seems to be a lens are naturally inclined to bring to the conversation. But I do think you've heard some advice, maybe not to start the things but to think about and get give you the double down as to how to work on an area you are already working in. So hopefully some of this was helpful and we could have a debate all day about the best way to sell your favorite thing to the cranky Congress. But -- sorry if we got a little bit sidetracked on that .

I think this was incredibly helpful. There were lots of important messages and themes that came out and I think it is good to know that a lot of the important areas that were discussed we actually have work ongoing in these areas so it reinforces that it's not just about starting something brand-new but really thinking about how to focus what we're doing and make sure that what we are doing is the most effective and that we are really thinking about the continuum from evidence through implementation and use an evaluation for all those things and just making sure we are following through more completely in the things that we're doing.

Today will comment about that, you sort of set this but there is a critical amount of work in any given area that is required to be highly visible and to be impactful as everybody knows. But there is that, always that check, they are doing something here, here, and here and can we do enough given increased funding or discretionary funding that can have a visible impact the kind of messaging that occurred around [Indiscernible] for example, it would be great if in primary care or some use of the data, there were equally compelling stories with an element of we supported the scale and spread and now it is national. That is where you have to aim for a think and as somebody who is a jack of all trades and a lousy gardener, I can tell you that if I planned to many kinds of plants they all do okay but there is not enough for a salad. [Laughter]

And were going to come to your house for dinner. [Laughter]

I would encourage people to keep thinking about this particularly in the context of the programs that can one has and I think this will be an ongoing theme and they also do think that the other piece I wanted to highlight is the notion of collaboration and hand offs across the federal agencies that could bring, even if you are the only ones that can describe it, I do think that is important and it is all the taxpayer dollar and thinking about how all of that could be better if there was more mindfulness and attention we know that handoffs are often were failures happen and I think that is the T-1 for ---four model, and in many ways the agency can think through that conceptually and practically.

We asked Francis to come back because people were stunned at what an exciting mother lode the investigator initiated grants at AHRQ were but we didn't have exactly the fact we were looking for so we brought the guy who knows to give us just a little bit of update on the funding profile.

Good afternoon, I'm happy to provide any information I can. With a specific questions that I missed?

I think we were trying to understand the headlines and asked a couple of questions, one was how to better get the word out on investigator funding. And how can improve the grants, but some questions came up around pay lines and things like that.

Sure. So AHRQ about five years ago made a concerted effort to move of one findings -- fundings that are three years cause you go to our website and the funding announcements will see a family of announcements that are focused in three areas because in the general research funding announcements which are classic are a one investigator initiated grant opportunity and there are applications of focused on patient safety and health IT. That component is no training portfolio applications which include can more traditional training applications so having a standing funding announcement allows us to fund a portfolio of applications that come in overtime. Currently the investigator and one, we fund at a competitive range of about 30%. Which means applications at the 30th percentiles so applications that fall within the 30th percentile strong chance of funding. Some people on a pay language is the point at which are funded applications the money allocated for that would run out and for us it around the 20th allocated for that would run out and for us it around the 20th percentile, it can go up two or three or down two or three pending on the number of applications. We generally allocate grants budget across three cataclysm grant. The large applications which are the traditional or 01 -- R -- R01 which are $100,000 a year. And the media training portfolio and conference grants which receive a small amount of money.

[Indiscernible - low volume]

We have R 21. We did that in the health IT portfolio area. And we allocate about 80% of your property to grant large grants, the R: investigator grant applications and those are about $40,000 a year maximum and spent 35 years. Any of the 25% of the grant budget is divided across training and some conferencing. Some conference grants .

So for the R a one the limit is $40,000?

Pardon me, $400,000.

I don't get about $40,000 a year. [Laughter]

So on the competitive end of the funding line we see about 1000 applications annually despite the budget that the agency may have appropriated in a given year to the percentile when we reach in terms of funding has been pretty stable over the last year and the 30th percentile. Of this Court applications. And that is a good point because in any round of funding percent of the applications are unscored and unavailable .

And by design right? You are aiming not to score 50% so that your time to discuss the others. Yes.

Absolutely. That is it in a nutshell but I'm happy if there are questions that come up if you would like a specific or deep dive into the initiated grant portfolio I would be happy to do that but I would say that applicants know that there are funds available and get the applications in withstand those chances of getting funded.

The other chance -- question that came up and I think you heard some comments and sent sheep in the room about the study section competition, I don't know if you have any thoughts about the kinds of people that are harder for you to recruit to study section that would be helpful for the kind of work that AHRQ does .

Let me just say by way of background we have five study sections that span the agency's priority areas and I think that is the agency's priorities shift that is the point at which it becomes difficult to have the study sections catch up with those new priorities. So economics research would be a good example, we focused energy and attention there in getting appropriate study section numbers to the committee has been a bit of a challenge. It is a balancing act for those in the academic health centers to devote the time and attention and decide where in their career trajectory is the right time to do it. And more junior stage where they are in their publisher mode or when they more senior than the response abilities are greater. I would say where they suffer most is peer reviewers who can step out of just looking at the scientific merit of an application and can think about the potential impact of what that research is. So we have done an analysis to look at how reviewers look at the impact scores compared to the significance of the research design and they always fall back on the research design so oftentimes we struggle because we see highly scientific meritorious applications that would have little impact in real-world settings. So in finding reviewers can balance relevant and science, but his sweet spot for us.

As you know very well I have a special interest in equity and try to be helpful in how can one might frame its funding requests around equity. It seems to me that is an area where funding currently is really tough to get . so we're particularly interested in fraternal infant health right now and this is a real opportunity if you want to push that because we go, or if I go, to Robert Johnson or Aetna or Commonwealth fund whatever it is you know what the and directs their arm. And it's not nearly as favorable as getting a grant from AHRQ. So is that something you really want to push in your priority population? I'm prompting you to say yes. [Laughter]

There is a fairly straightforward answer to that question. I would say yes and I would say we taken a two-pronged approach to try to integrate equity across our portfolio research grants. The first was in updating the agency's mission statement to explicitly include equity as part of our mission. The second one is to push a priority policy and to partner with our staff in the agency centers were there is a focused research area to say what is the relevant in this research area for equity? And her colleagues in patient safety have been wonderful in allowing those kinds of collaborations which resulted in funding announcements to draw out a specific priority in that area. And second in the is published a couple of notices to remind folks that in the context of grant application tournament to include in a meaningful way not just the focus of the application but its impact across diverse populations.

This is very helpful. I think more about the study sections. Are they -- does eat -- each have its own character for can you say more? I want to make sure how to get people recruited to them and what we could do to help them with that. And I'm going to go back to the primary care discussion earlier, this didn't really come on but should there be one of the six study sections that has primary care or does one already exist. Five --

Five study sections focus on training one of the qualities focused on one has a research focus, one is focused on our patient safety study section and in a large degree on health economics research. So that is how we are divided. We don't have a primary care study section, there are primary care research study sections practitioners for sure and we could create one for sure, that would be -- we recruit in at least three ways, the first is a public announcement to invite nominations, the second is to work with the subject experts within the organization to have them come up support. The third is to work with in the study sections in the academic research communities to advertise the possibility to communicate. We shared database with the national institutes of health, use that as just another place to look for researchers. As arcs portfolio research expands we can add or subtract the portfolio . so he asked me how to the applications distributed across the [Indiscernible] the patient safety and quality section receives twice as many applications as the health economics studies section. So right now I'm trained to decide whether we should split it into two to manage the workload. At the same time they are figuring out how to come up with the applications and economics resource research and make sure the research as much. So to give you a sense of how it plays out.

Maybe it's something we should keep in mind. I am just wondering where the study sections are one way to help shape some of the emphasis or how --

That's an excellent point. What they serve for us as ambassadors for understanding the agency priorities and then taking that back to their respective research communities. So for example part of the reason why the patient safety and quality section enjoys but the full contrary of reviewers but also a full commitment applications is because they are both the ambassadors for the agencies as well as researchers in that area so there are professional societies, they submit study section donations to us without us having to ask. You can serve as well, this is a shameless advertisement for you to serve.

[Laughter]

As I looked around this morning many of you have already sent in a study section so I wasn't going to ask you to serve but to submit nominations to the study sections.

Any other questions? I really appreciate you coming and I actually think the message about the potential for funding is a positive one. I intend to take advantage -- no. Anyway. Any other questions for Francis?

If I may just one more shameless plug. [No Audio]

We can spend a little bit more time talking about, like they did with a primary care physician discussion this morning. So let us know what you nominations are. Thank you.

Thank you very much. Back is there other stuff that we can do is area cycling off the [Indiscernible], I'm not a researcher.

[No Audio]. [Laughter]

A permanent ambassador. Okay. I don't think. Francis that was really helpful we should have had you as an agenda items -- and me. They want. So anyway I just want to thank everybody for participating today we had some really nice conversations and hopefully they have helped Sharon and her team and I look forward to seeing everyone in July. And the one thing we didn't take time to do and I apologize because I am trying to make sure that I catch a flight so I can get home so that I can get on another plane tomorrow. The one thing we didn't do is if you have ideas about things for your next meeting. Please send either Jamie or Sharon or myself an email, proposing that topic. So that as we are putting the agendas together we can take into account areas that would like to spend more time talking for example like the primary care physician were talking about today .

Thank you very much.

[Event Concluded]