Session Information:

Determining Value and Benefits Realized for the Veterans Health Administration: Optimizing Health Care Valuation Methodology

• VHA evaluates health care investment transformation and optimization using an unique approach that can be utilized by private and public health care systems that includes:  (1)  a whole system evidence-based framework that supports strategic planning, (2) a comprehensive enterprise-wide measurement plan that measures progression over time, (3) an independent direct causation-focused analysis of drivers, people, processes, technology and outcomes, and  (4) an analysis that identifies benefits realized facility by facility, specialty by specialty to determine the return on investment and promote the organization being a learning organization.
• VHA has validated successes in transformation and optimization through knowledge management that address access, patient safety, quality and high reliability.
• VHA provides lessons learned from their transformation and optimization methodology, which begins with a business case and baseline assessment and takes the audience through benefits realization and investment evaluation. 

Session Transcript:

Super excited to have Jennifer Ford with us. Jennifer Ford is the Director of Product Effectiveness at the Veterans Health Administration. Jennifer has nearly two decades of healthcare experience. She optimize health care investments by producing evidenced based assessments to maximize the intended value for America's largest integrated health care system.

I want you to pay attention to this.

They provide care at 1255 health care facilities serving more than nine million enroll veterans each year.

Jennifer is a published author in frequent speaker and national and international conferences, and is an integral player in early stage legislative policymaking.

Jennifer, I'm beyond thrilled to have you with us. Say, Thanks so much for taking the time of your incredibly busy schedule to share your insights and expertise of our global audience today.

Thank you, Josie, and thank you all for being here.

You may have attended some of my presentations in the past, and if so, I am grateful that you have returned.

In past speeches, I've discussed large scalable healthcare transformations here at the Department of Veterans Affairs, But today I'd like to discuss just how my organization actually optimizes healthcare value, as Josie said for the largest healthcare organization in the US.

So, to start I quote, John Maxwell. Change is inevitable, growth is optional.

It is the growth that leads to positive innovations you see all around you.

Businesses can choose to either lead the change or and grow, or they can decide to adapt to other people's changes and conform to other standards.

For a business though, effectively transforming means you can maintain your current operations while exploring new initiatives that are meant to improve your business models and add value to your organization, and, of course, to your stakeholders.

But how do you know you're improving and maximizing your growth opportunities, or, in other words, improving and maximizing the value from your investments by even transforming?

This presentation attempts to provide you with this insight from our proven and unique methods in determining value and success in healthcare transformation?

In healthcare, innovation, and transformation is happening all around you and in real time.

The VA, the VA, has led healthcare innovation from the creation and inception of the electronic health record that was integrated across our Nation over 50 years ago, to leading the country in patient centered care.

We, here at the VA, really take to heart that transforming is keeping our Veterans in wellness versus illness.

We spend billions on health care each year and need to answer to our stakeholders, which, by the way, are a pretty tough crowd being the veteran, US. Citizen and Congress.

They all want to know, and rightfully so, what evidence do you have that what you're doing is working or a successful?

As I mentioned, the Department of Veterans Affairs is the largest health care system in America. We have a 70 to $90 billion budget, a year, any given year, and serves over nine million Veterans in over one thousand facilities. Just imagine what 90 billion means, if I, If I lined up dollar bills, back to back or side-by-side with each other, it goes around the world 360 times.

And if you, as you can see here, we have over 377,000 employees in Veterans Health Administration alone. Just to compare that, Google has just over 135,000.

We're talking about large-scale transformations.

So where do we even start evaluating this? How do we even get going? Well, in 2003, the VA embarked on its major health information and information technology transformation. The idea was to overhaul the complete entire health IT infrastructure and all the systems with it amounting to a multi-billion dollar investment. Therefore, my organization Pocket product effectiveness was born.

I'd like to share with you a video that Foundationally describes the organization.

This video was produced to help our internal stakeholders at the VA understand how the service, or how the services we provide to them can help them realize the value of their efforts.

If you're willing to do something, you want to do it as well as you. Can you invest your time, energy equipment and money, and you want all those investments. Pay off with exceptional performance coaches. Improve performance by measuring what you're doing and helping you figure out ways to do it better.

The HA has its own group of coaches a team of performance specialists who work for an office called Product Effectiveness product. Effectiveness or P can provide the data. You need to improve your performance. Helping to ensure your facility team or program is effective, efficient and valuable to VA product. Effectiveness helps you collaborate with other BHA organizations, validates that technology provides the benefits users expect takes, advantage of lessons learned and measures customer satisfaction. Product effectiveness independently provides empirical data to enhance the effectiveness of your investment decisions. It.

Takes innovation to keep up with emerging healthcare technologies, our industry knowledge and expertise with, collecting lessons learned from past experiences, serves as valuable input to future efforts for improving veterans health care. It takes sound investment decisions to achieve healthcare value. It takes performance measurement to understand if you're on the right track, for healthcare investment and Technology success. If you can't measure it, you can't manage it. Measurement is about reducing risk, and product effectiveness is expertise, provides BHA leadership with evidence based information for maximum decision making power. P is people, Technology, investments, innovation, and Performance Measurement. Through highly focused measurements, product effectiveness can mind for valuable data that can help leadership make decisions that improve the performance and lead to better health care.

Product effectiveness works in four domains: Benefits realization: Customer Satisfaction, Functional Review, Lessons learned, Benefits realization is a way to figure out whether a new program or product actually does what it's intended to do. Customer satisfaction is another measure of a program success. Do people like the new program or service? Is it easy to use? The PE Functional review team can help you avoid, identify, and mitigate risk. The lessons learned domain helps you learn from others in ... and carry those lessons forward to future endeavors. Product effectiveness strives to help customers achieve their investment and program goals By providing them with an independent view of evidenced based results. Product effectiveness has deploy these services across the enterprise to help customers deliver optimal performance, efficiency, and value.

Like a Coach, P can help you unlock your potential and ensure that your time, money, the effort, result in efficiency, effectiveness, and value for V J healthcare. And.

All right.

So, obviously, the video is intended to help easily digest some of these complex methodologies that we use. But what's interesting about the evolution of transform transformational programs is that my own organization also had to transform as the VA was transforming.

Back in 2010, the then administration brought ideas.

two, sorry, I had to increase my screen, the video minimize, to make broad ideas of what transformation of focus they wanted to focus on, which included healthcare efficiencies and new models of care.

Again, the real question we kept asking ourselves is, how do we know we're going to be successful? What tangible evidence can our stakeholders have and use that these major initiatives are benefiting our veterans?

Well, at the time, the principal Deputy undersecretary of health turned to me and simply said, hey, I know you do these on health IT investments.

But can it B also apply to non IT healthcare transformations? Of course, I'm going to say, yes.

And then, so my organization transform from product effectiveness to optimizing health care valuation. And, of course, we're going to have to go back and change our video, as well.

But you may wonder why I still use product effectiveness? Well, the problem is, right now in the government, there's been a lot of changes and what transformational wise, but what doesn't change sometimes is our organization and we need and organizational changes. We just did go through a re-organization. So we are planning on changing the name, and hopefully maybe the next time you see me, I'll have the official title changed.

But, so, what does it mean to go from a product effectiveness type of organization, which focused on IT products to a more general, optimizing overall healthcare value?

Well, it means I take a whole systems approach, an evidenced based approach, to determining value.

Instead of just focusing on technology, we focus on people, processes, technology, and policies.

You might say, Why policies?

Well, because some of our policies and directives here at the VA are still mandated, but they're outdated according to our transformation. So, you need evidence to be able to either go to Congress or to leadership, to show that you need to change the policy and director to match the transformation.

But with this whole systems approach comes a comprehensive, enterprise wide measurement plan that in competence, encompasses all measurements that determined the return on investment if the investments or transformation is optimized, or if it's reached its potential for all benefits that were intended.

In contrast, the measurement also yields unintended consequences that prohibits, perhaps, change, perhaps prohibits change management, cost effectiveness, and brings up any risks that need to be attended to to continue the transformation from to progress forward.

So, this comprehensive measurement plan gets analyzed to determine the key performance indicators that demonstrates causation.

one of our issues that needed to be addressed early on in our methodology was this exact problem of collecting data, but did it, Did it have a causation factor to what we were trying to change?

We assumed that we could use national quantitative measures as evidence based information to determine whether a benefit had been realized.

The one project I'm going to reference is actually our ICU technology that was being integrated into our clinical care environment.

The objective once implemented was to really see these notable, positive changes in no quality measures that people follow such as central line associated bloodstream infection rates and murcia rate, MRSA infection rates.

However, six months after the implementation, those measurements barely moved.

We especially couldn't tell that the implementation of the project caused any of these measures to be effected.

So, are you telling me we just spent millions of dollars on a system that yielded no measurable results? What happened?

Well, it's pretty easy.

Most of the measures are really measures that take time to change and are predicated on complex change management in order to optimize before any causation can be achieved.

So, then, you say, well, what are you going to do? You can't wait years before you prove that. A multi-million dollar investment is yielding results.

So, therefore, we adjusted the methodology to monitor these longer term strategic measures and created more immediate, impactful, optimizing measures. If you will, We call them foundational measures.

We did more detailed workflow and business rule analysis, facility by facility, specialty by specialty.

We are unable to uncover immediate return on investment because we can capture efficiencies right away.

So, for instance, if an ICU doctor required 10 minutes to review and track certain markers within a chart, and now the doctor can track and review in a minute.

We've recoups significant deficiencies in the process and also, just as importantly, provider satisfaction.

And, finally, since we have multi tiered implementations, we also collect best practices and lessons learned to improve quality, effectiveness, and timely of the next implementation.

With the billions of dollars we spend, per year, and close to 400,000 employees, we have thousands of innovations, pilots, and ideas, for change per year, from, no, from the entire system.

So how do I focus my resources, which are limited to the most valuable investments?

Well, each administration comes in and has sometimes similar goals and different goals and objectives that they want, the V: a, and healthcare tube to progressing.

They want to achieve in VA, much like a company's Board of Directors.

So what you see here, are the Leading portfolio's that aligned with Leadership's Transformational goals. I will discuss a few of them today.

So one of our signature programs is learning optimization.

This team collects all assessments that we produce, analyzes them across other assessments, maintains our institutional knowledge, mines' data for best practices and lessons learned.

Disseminate findings across the enterprise and reduces risk investment by ensuring that we don't repeat results that didn't go so well.

In our Knowledge databank, we have 350 transformational projects encompassing 15,000 evidenced based assessments in several categories, as you can see here.

The main categories in health care, really, access pharmacy lab, going down mobile technologies, telehealth technologies, interoperability, innovations, inpatient, outpatient surgery, the list goes on.

So we're gonna look at a few of them that are more current, that we're working on today, or just completed, just to give you, it's just scratching the surface, but it gives you an idea of how this methodology yields results.

So as the VA started to replace our EHR electronic health record, which is, by the way, 100% customisable, we realized that moving to a purchase EHR, we did not have an inventory of our pharmacy automation.

Therefore, we were tasked to perform a nationwide inventory analysis for evidence and decision making of our future automation needs.

We found that the VA possess different models and robotics with different configurations, interface's, capabilities, et cetera.

We also performed a capability and benefit study so that other facilities could use the knowledge to invest in beneficial pharmacy robotics.

two years later, recent time, we were asked to repeat the study because we were uncertain of the changes across the enterprise, and in particular, we needed to prepare for the new EHR modernization now that we actually knew our vendor.

It turns out, the pharmacy automation tools needed more analysis.

Some systems were legacy models that worked with our current EHR, but not with newer technologies, so therefore, the results are now being used to determine one, what machines really just need to be sunset and repurchased.

two, which interfaces are complying right now and we could just go ahead and convert or three, what machines can be used But we now need to create new interfaces in order for them to talk to the new system.

In either case, you can see that this can become an enormous expense to either create new interfaces or to have to repurchase these types of huge automation or robotics machines in general.

So this business case that we're putting together really cannot be achieved without this whole system approach.

Also, N tau, we completed a nationwide pharmacy workforce assessment with over 3200 respondents.

So here at the VA VA, we process close to 600,000 prescriptions per day and safely, I might add.

But how many times have we ever even check to see how the pharmacist's were doing all of our pharmacies to fill them.

So we needed to understand the educational and training needs of the workforce.

So the results of our study told us that our pharmacists were not necessarily 100% satisfied, they did have some burnout and lacked necessary training and promotional opportunities for that satisfaction level.

In fact, 85% work said they were stressed, 85% said they would move to a different role if they could. And 78% said that there were barriers to advancement.

So you can only imagine how valuable this evidence based information was for us to put in plans, so that our pharmacist could then start to realize the benefits of our training and our promotional opportunities.

Without the evidence, we might not be able to even get funding to look into these matters.

Another example in this bucket is, lessons learned is a lesson learned study we we did regarding bed management and capacity management.

So as you know, how, you know, healthcare facilities around the world were overwhelmed with the overload of kovac cases, and not being able to point, you know, and being able to quarantine coven versus not uncovered patients.

So initially, we found that our medical medical centers were actually not consistently using the bed management system, which was created to document the capacity management in real time. Further, we realized that there was no requirement for national reporting.

So covert kind of made this a catalyst for us. So, our assess, the assessment surface these issues, and put in place, recommendations for future efficiencies and effectiveness in our bed management, and process and reporting.

So, high reliability organizations, or HR RO, as you see here on the left, focus on leadership that are fully invested in an organization's effective communications, culture of safety and process improvements. Becoming an ... is one of our under secretaries major transformational initiatives today.

one of our facilities embarked on a journey toward high reliability that we, but but the problem is, is they weren't measuring their success.

So, in taking a baseline, their efforts yielded over 3000 process improvement ideas and projects, which resulted in numerous efficiencies.

Employee and veteran satisfaction, or reduction of risk and patient safety events and even an Roi over two point $8 million over two years in just under two years.

And as you may know, the VA is highly focused on access to veterans, since our issues, kinda that were back in 20 13.

This led to another major initiative involving care management, care co-ordination, and referral, and consult management.

We're now developing a health care failure mode effects analysis to detect bottlenecks and failure points in the system, placing key performance indicators at those potential failure points, which will then help us expose risks in order for us to address the risk before the delay any delays happen.

In doing so, we assessed two providers from 10 specialties. three providers from primary care, at five facilities, amounting to over 100 in-depth process and failure effects analysis.

This process provides us an incredible insight to allow the process to be optimized.

I have to take a quick breath. That's a lot, and the thing is, is this just barely scratches the surface I could go on, and on and on about all the efforts that we do from a transformational perspective.

But, the real question I think you're here for is how do we do this?

Optimizing investments and values requires you to measure and build on your successes.

Once your stakeholders take hold of an innovation, the possibilities are endless.

As such business process re-engineering begins, change management ensues.

Optimization gets cycled into the process and the organization transforms.

But here's the question, by how much, and to what extent is the evidence that my, or, is the evidence that my organization provides, and also determines, if more value can be achieved from the transformation.

So, this is our model, and as you can see, some of the names have changed from our video. We are kind of moving towards our new model, Benefits realization, is the same.

Customer Satisfaction, change the stakeholder impact, and experience, Functional review's Changing the systems, impact, and analysis, and lessons learned, just simply Change the learning optimization, which is a better overall name for that organization.

So, the blue chevron's here represent the innovation or the transformation.

And as you can see, we first assess to the far left, the as is environment.

What are the outputs of the as's environment? Why does it even need to change?

This creates the baseline and the business case.

As the project gets implemented, as you can see by those smaller down arrows, we take interim measurements to reduce risk and ensure project success at deployment.

This entails a deep dive into the process, flows and business rules. So, what's the difference between a process flow and a business rule?

So, for instance, a process step may be, Schedule A patient, But underneath the process box are business rules that may be unknown, that if it's not explained on the process box and could cause failure points.

For example, a scheduler schedules an appointment, which is a step, but only if they perform several manual lookup processes, that cannot be quantified how long that took.

And because it's manual, the manual could have errors.

So, as you can surmise this level of detailed measurement is required throughout the implementation nowadays, not only to ensure the success, but if there's a hiccup at deployment, we have a better chance at looking back at the root cause analysis versus not having these incremental measures as insights.

Once the transformation gets released to the enterprise, we really have our work cut out for us.

So, our first step is to measure the 30 day mark.

We take an initial measurement, call it a post implementation review.

We're looking for adoption, normalization. Any quick wins and lessons learned?

It is also our first opportunity, and determining when the next full measure should be taken.

So, in a very complex, transformational implementation, we found that a minimum of six months is required to wait in order to show any meaningful results.

Sometimes we've even had to take nine months.

Then, we measure for about three years, year over year because we found that that pretty much maxes your evaluation.

But for a less complex implementation, we have found that in about three months, we can find meaning real meaningful results and then put it on a measurement interval after that.

Then finally, you can see at the bottom, we take lessons learned all along the continuum.

So, now, what I'm gonna do, now, that I've, you've seen the video, we kind of put the names too, The different process boxes. What I'm gonna do, is I'll pull it all together by walking one of our innovations through each step of this methodology.

So, because our scheduling and access to care transformation was so significant and it pretty much went through every single one of these pieces for years and unfortunately, covert has slowed down some of our transformations, so that the measurements aren't as prevalent today, in general. I'm gonna go ahead and use this Case Study to walk you through the process.

So, first, the Business case, What is the need for this change?

Our scheduling system, believe it or not, was on a roll and scroll blue screen.

You may say, hey, that alone will cause me to change, but it's actually a little bit more complex than that.

First, our scheduling system is highly integrated into our EHR and is very, very difficult to decouple and desegregate that system from the entire system.

Also to implement just this piece of the EHR, we need over a billion dollars.

So, really, we need to figure out how great, how great is this change needed today.

So, the next set of slides, I'm gonna go over with you, These are visuals just to show you a teeny little piece of the pie, but it does give you a visual at each step of the way.

So, the first thing is, is the best way, as I mentioned before to analyze was to document the process at each specialty within the Medical Center.

So how does each specialty schedule within the medics within the Medical Medical Center to understand their specific scheduling needs?

What is urgent, and what is necessary as a prerequisite before the visit?

Our initial goal was to analyze all of the specialties, over 25 of them with self and sub specialties.

However, that endeavor was overly ambitious for the timeframe that we had, so we had to settle actually on 12 of them, which were complex and we knew would yield us the data that would give us a really good sampling for our assessment.

It is also important to know that we are a closed system, so we must use the most effective means to schedule into our specialty areas, making sure that we maximize provider productivity.

Therefore, our primary care providers order labs and images for the specialists ahead of time, based on how our patient is presenting.

Then our specialists can have the necessary information to make an informed decision for the patient on the first visit versus going to the specialists, getting your scripts to go get either an image or a lab work waiting, and then coming back to see.

Know what your next step in your care plan is gonna be.

Although each specialty do they do have their particulars for scheduling.

It really just boiled down to a few core reasons for the need to change, and the basis for the business case.

So, what we found was each process, regardless of specialty, had multiple complex steps that were performed outside of the system. So, in other words, they were done manually.

Therefore, we needed to modernize our functionality to perform complex tasks, and also the ability to accept all these business rules and verified process guidance and hard stops at potential failure points.

The current system essentially acted like a calculator.

It basically was only that the results from it was basically only as good as the user gave its inputs, we would have no idea how it was kept what those inputs were, prior to.

Therefore, it was really challenging to determine the value of those strategic measures, such as how long it really takes for a patient to be scheduled, or how long has a patient been waiting.

In this workflow, we were able to pinpoint where the potential points of failure occurred, What functionality was required to ensure we could meet our strategic measures of access.

So, what were some of those strategic measures?

These are the categories.

Now, there's lots of measures under these, but Tylee, timely access to care, clinical practice management, How do we operate clinically?

Clinical integrated enterprise scheduling, enhanced reporting and decision support, provider productivity, overall system efficiency, stakeholder satisfaction, and, of course, any kind of cost, effectiveness or cost avoidance.

When we looked at the functionality of potential systems to ensure the functionality was available to overcome these challenges, we also analyzed other assessment or other enhancements, sorry, That could help us determine access success.

So this work, flow analysis and business Rule analysis, plus the enhanced features, plus the intendant strategic measures that we expected made up about 75% of our measurement plan quantitatively, But we still needed a balanced plan.

That balanced plan had to also, if you remember from our slide before, how to demonstrate stakeholder impact.

So qualitatively, we needed to know how satisfied our users were and using the current system.

What are their frustrations?

What did they think can make this system better?

Much of our workflow analysis led us to this information. But we still needed an anonymous survey that was scientifically and statistically sound.

So, both quantitative and qualitative components lead to over 300 measures in this comprehensive measurement plan that it was to intended to provide the evidence necessary for the business case and baseline justifying the need for this change.

In addition, the work for an workflow and business rule analysis is also hugely beneficial to our stakeholders. It provides them with the evidence to make the necessary changes to re-engineer their processes, to start to optimize for efficiency and effectiveness.

As the new transformation is being implemented, we are ensuring that the functionality and processes are changing as they are intended with qualitative functional assessments and lessons learned at key milestones.

We also started, as you see here, collecting all baseline, qualitative, and quantitative data from our measurement plan.

We conduct the survey and make any enhancements to our measurement plan.

Based on those interim measures from our qualitative assessments and lessons learned, we analyze the data and create a baseline report that serves as the basis for comparison and the business case for change.

You may have noticed that this measurement is a huge project in and of itself, which we then compensate with a program integration team to ensure that all the methodologies that I'm explaining here today are executed flawlessly.

So, after the transformation is implemented, as I mentioned previously, we'll do a 30 day post implementation review to determine if the function, that functional improvements and adoption of the new re-engineered process have been achieved.

nine times out of 10, we find similar results, no matter what the transformation is.

We document usually early adopters that try to use the new implementation to the fullest.

They fully take advantage of all the new of all the new functionality and processes, others may adopt change a little bit more slowly.

But essentially, they kinda continue their current processes until they're more comfortable with the change. And then of course, you're always going to have those that are resistant to change and may need a little bit more incentive.

Now, these were this result, follows the usual bell curve of change and adoption.

So we share the initial assessment with our stakeholders and determine when the complex, more complex post measurement should, can and should begin.

In this case, for our scheduling project, we actually only waited for months. Even though I initially said six months, because there was a lot of, there were a lot of eyes on this, quite frankly, and they wanted results as fast as possible.

However, it was too soon of a measure.

So we repeated the measurement lists, specific measures at the seven month mark that gave us some very tangible results.

So, we repeated the process, again by collecting data, analyzing it for causation, and compares the results to the baseline and determined how much value was obtained from the transformation.

We were now able to effectively measure wait times, productivity of providers, and access to care more accurately.

We can obtain reports in real time that gives us data for real-time decision making.

The second measure also gives the stakeholders another evidence based assessment for decision making and further optimization.

But even more importantly, this report was enough evidence for Congress to grant us the funds to replace our scheduling system nationwide years ahead of the planned implementation that coincided with the EHR implementation.

So, just to summarize our reports, validate business cases, report key findings and results, provide tangible evidence for decision making, and allow for recommendations for future optimization and investment.

So this is a quick and dirty, in a nutshell, what the entire process is. So this is the final summary.

First major measure, look at business flows, and technology, workflow analysis.

If the technology, if there is a technology involved, create a comprehensive, enterprise wide measurement plan which contains quantitative and qualitative measures, strategic and foundational measures.

These frameworks are customized to every project since every project is unique.

We collect and analyze data and report on results and recommendations for evidenced based decision making.

The process is measured before implementation, then tweaking the measurement plan to incorporate unforeseen benefits or unintended consequences along the way, and measure it again for post implementation results.

Those results are compared to the baseline for valuation and recommendations for optimization.

The beginning of a transformation can start high level and quite frankly, big, fuzzy in terms of potential results.

Leaders and stakeholders may not exactly see a concrete future from this transformation.

Through our unique methodology, tailored to the healthcare industry, we bring transformation to light for clarity and value.

Thank you for spending the time. Learning about how the largest integrated health care network in the United States produces an return on investment for our veterans.

Absolutely Fantastic, Jennifer.

What what a great masterclass on large scale transformations, I mean, everything in life is, is, is based on references and, you know, we hear often even conferences like this one, someone who's very proud because they get a Kaizen abandoned kind of an innovation project on their clinic, and they're kind of showcase that.

Now, you are doing this across 1255 health care facilities.

We've, with nearly 380,000 employees, The scale is massive.

So, the question that comes from the audience is, and this could be a long answer, so maybe just kinda pick some, you know, bits and pieces of it. How do you keep employees engaged in this transformation, within such a large system, and all the way from the ideation P? So maybe we start with that. How do you get people involved in for an ideation perspective, on what are the things around the funnel, for you to look at and for team to look at and prioritize.

And then how you get, You keep people engage, and sustain the momentum on the transformation.

Hmm, hmm, hmm, hmm, that's a great question, because I think this is the number one thing that fails and transformations is, and has kind of haunted the healthcare industry in some respects where we go ahead and we make these huge transformations, and we forget to ask the people right on the ground floor using who are the ones we're asking to change? And I think one of our key success factors is that we don't assume that the changes are going to necessarily 100%, be beneficial. That's why I mentioned unintended consequences.

So, what we do is, we do think that the claim to fame is that, that bucket where you sell specialty by specialty, or facility by facility specialty, by specialty, you know, we have facilities across the United States. They have different cultures, different, no ways they want to operate. You can't assume that a facility in New York will operate the same as a facility in Kentucky. And you can't, You can't assume that the people are gonna be as motivated in those areas. That's kinda number one. Number two, when you do the in-depth analysis that I was describing, you really empower people, It gives people an empowerment to say, Hey, I, my inputs, actually counting? I'm watching you change the process and re-engineer.

It actually baked based on my input and a lot of people fail in that they don't even ask how it's gonna work for them at the frontline, and that really is a, is A, is a claim to fame and it is a key critical success factor.

Once you have this many transformational initiatives going on and, and, you know, making progress on them. How do you keep the, the organization, you know, updated not only not only the leaders who have to communicate, you know, the results, but really, the employees? What type of feedback mechanisms you have so that people can know, you know, what the status is a certain transformation initiatives?

Yes, I think a lot of our leaders are excellent about engaging our employees through town halls, or, you know, direct presentations, panels, Q&A environments, that really allow them to absorb the information that they have. A lot of leaders will go ahead and put posters up for some of the changes in the positive results. You know, look at our results, and really encouraging and incentivizing environment.

When it comes to your validation of the results, you have a very well defined process. I mean, you're looking at these things, If I got it correctly during your presentation, you discuss, you're looking at a three year timeframe on the implementation because to your point, you know, well, good intentions fail at implementation, right? You have the unintended consequences of those implementations that can be really hard to do. Do you have a special kind of like audit, like group that's that's focusing on that portion of the process. Josie, audits a bad word for us. So we'd like to use consultative processes because we are not an auditing team. What we do is, when we take our measures year after year, up, to about three years, like, I mentioned, it's really to help the organization optimize and expose how much more value we can eke out of our transformations Now.

Like I also mentioned, you're gonna get your early adopters. And they are the people in the VA are the most amazing innovators, and they really will take these transformations to another level.

The folks that I've mentioned that are kind of usually in the middle of the bell curve then start to emulate and copy those early adopters, and more, and more value, is this.

It is absorbed, even to the point where maybe a year or 2 and 3 were operating that way as a given.

But if you don't have that baseline to look back to where you came from, you would have no idea how much value really is now a given in your organization.

And it's really amazing how much optimization happens year over year. Now, at some point.

Is it worth the investment to measure it?

No versus optimizing?

And that's why I said, we normally get about a three year kind of mark, on that, Where, you know, they're now just sort of learning yet. They're optimizing their learning.

In the last quick one here to wrap up, Jennifer, Is there a place there are lots of questions about great work. You know, is there a place where the audience can go? Maybe a website, or, or maybe there was a report that's issue, you know, over a certain period of time, where people can go and kind of follow this journey, somehow see what's going on. Do you have any external facing type of publications? So, that's a great question. We are actually going to be published for that scheduling one that I just mentioned that's coming out in September. And we are now getting on a cadence to publish all of these findings, which will be able to be, you know, obviously obtained by the outside world. Unfortunately, a lot of this stuff is for VA, internal use, It's within our firewall and it's not necessarily published on the outside.

But a lot of our work as well, not, you know, you're not gonna go find exactly me, specifically if you search, But there's a lot of congressional, you know, type of, like the scheduling one where the decision was made based off, of, you know, our documents that we sent in. So, I guess if you searched really hard or requested that, like, maybe you can find them, but mainly, it's inside our firewall. And, unfortunately, we can't really share it that way, because it's for internal use, but we are publishing, as I'm doing these presentations, as much as we can, to educate the outside world.

And I would always be happy to answer any questions of, you know, any kinds of challenges or opportunities you may have to talk through.

That is so generous of you, thank you so much, Jennifer, for taking the time, mister, this tremendous transformation that that's going on in the, in, the in the system, share your expertise of all of us. We're very grateful for that.

Well, thank you, everybody.

Ladies and gentlemen, that was Jennifer Ford leader for this incredible transformation that's happened in the Veterans Health Administration. Multi-year transformation multi-billion dollar.

Potentially the largest scale health transformation happening in the United States, if not in the world. So, grateful for that. We're going to be taking a break now and we're going to be back up at the top of the hour with a panel discussion on innovating the patient journey and we're gonna get some great people here with us. We're gonna have Blue Prism and ebi and we're going to have leaders on process improvement in technology talking to us about this innovation on the patient journey and what we can learn from that.

And the, getting your questions in for them in real time.

Not only during the live Q&A in the end, but throughout their discussion, so that we can learn how we can bring new technology, is, how we can bring new perspectives and approaches to the patient journey and innovating on that patient journey. So, we're gonna close this session for now. We open back up at the top of the hour. Thank you.