Session Information:

Assessment as an Accelerant to Operational Excellence

One of the US’s largest energy companies shares what drove them to pursue Operational Excellence, and how they’re using assessment to accelerate their journey.

• Duke Energy’s journey toward Operational Excellence
• Why assessment and oversight are critical to Operational Excellence
• Critical parts of an assessment program
• Helping leaders value assessment
  • Self-assessment and
  • External perspectives

Session Transcript:

Those oil and gas live We had a little bit of technical difficulties with the launching of the presentation for ad but we got it working now and we're ready to go. So I'm very excited about having an brewer here with us today. He is the Vice President of Operational Excellence Assessment, and Oversight for Duke Energy. Edge is one of those unique leaders in the industry. He has been with the energy in electricity production, specifically, gas distribution, and over 37 years. And that's true leadership and experience, ranging from operations, engineering, strategy, if he has a Bachelor's of Science and Mechanical Engineer for North Carolina State, and an MBA from the University of North Carolina. So, he has gone on both sides. That is gray and add, It's a real privilege to have you with us, a real leader in the industry, and very much looking forward to your presentation.

Thank you, Chuck, and good morning to everyone, and I apologize for the technical delays. As we know, challenges working from home through virtual, private networks, or you have to be creative and innovative as far as I know. So I appreciate the patience to everyone. So it is a pleasure to be here and talk with you this morning. Our operational excellence journey at Duke Energy has been a fascinating one.

And we are at a phase where we are using assessment, which is my primary role, as an accelerant to how we continue on the journey to operational excellence. So, I don't want to talk about assessment without, sort of going back and and talking about why we're where we are and how we got here.

And, unfortunately, at Duke, it starts with one significant event back in 20 14 we, one of our power plants had a pipe failure under one of the coal ash Repository ponds, and it spilled the coal ash into the Dan River.

Just South of Virginia, 30,000 tons of coal ash went into the river, obviously polluted the river.

There was no long term damage to the river, but from a corporate reputation perspective is where so much of the damage occurred. We're a 100 year company, Fortune 250, Like to thank one of the leaders in the energy industry, and this was not our best day, Later, that same year, in 20 14, we had an oil spill into the Ohio River.

And so, this was really the wakeup call, the fallout from the Dan River event.

As you can imagine, was pretty far reaching.

Obviously, negative media coverage, customer outcry, environmental issues, the government, both state and federal, and launched investigations, the EPA lost an investigation, and, again, this was not what our company was built on, our Chief Executive Officer.

Our chief operating officer still talks about this being the, one of the worst days in his career. You can see all the follow on effects, There were actually a criminal charges at a misdemeanor level and we were put on a five year probation period for doing any work with the Federal Government. So, again, a huge issue and not our best day. So, but it was the wakeup call, something needed to change, perhaps that we had become complacent as being the operator that we thought we were, and what needed to change.

And so, our Chief Operating Officer at the time, knew that we needed to start down a path of operational excellence, and we did some benchmarking, put together some teams. First off, what are we trying to achieve? How do you make sure you get the right level of operational discipline further, focused on the results that matter, and this is the chart reviews. And, as, you know, it's nothing scientific, it's just a four blocker, Making sure that we understood that we did want a balanced operational discipline. Again, focused on results that matter, we considered ourselves at that time in 20 14, sort of in that lower left quadrant. Our operational disciplined wasn't where we needed to be in our results were not good.

Performance lacking 2015 and 2016, we actually kinda went up the results on the left side. And, but we knew that our principals and our practices and our discipline still wasn't where it needed to be. So, we were sort of in the unsustainable quadrant on the upper left.

Um, the other thing to think about is when you put in place procedures and processes, do you swing the pendulum too far on the operational discipline piece and perhaps lose focus on the results that you're trying to achieve? And we operate six nuclear stations, which have a high operational discipline, as you can imagine, high government regulation from the Nuclear Regulatory Commission.

And there is a lot of procedural basis. And if you swing too far there and you're sort of into the bureaucracy mode, and you, you can't be really sustainable there either. So, again, our, our desired state was that upper right quadrant, the right amount of operational discipline focused on the results that matter to the utility matter to our customers, manage all of our stakeholders. So again, what our definition is, operational excellence as results plus operational discipline.

And so we started an objective at the time. The objective opic's at Duke is to exceed stakeholder expectations about achieving the highest standards and safety, reliability and affordability.

This has morphed over the last couple of years the affordability piece We work back in because we can out, again, we're still in many parts of our business are regulated. And we have to make sure that our cost stay in line with our what our regulators expect and what are truly what our customers can afford.

Again, as we started down the OpEx journey, we did benchmarking with, obviously the chemical industry, oil and gas industry. You see some of the names here, Chevron? dupont. We leaned heavily, again, as a large part of our operation as nuclear. We leaned into the nuclear organization with the Institute of Nuclear Power Operations and we saw some comments to common components across those, right?

They They all have standards, they off training, they do some sort of evaluation, and we'll call that assessments as we move forward, and then there's got to be some support back from some central body, a center of excellence or something like that.

So with that, we developed what we call it a scalable OPEC's framework. And the reason we talk so heavily about scalability is that there are very different risks in different part of our businesses. We are a electric utility in seven states of the United States.

So we produce, transmit, and deliver electrical generation.

We also have a gas distribution business, and at the time of this, we're actually in some international businesses that we have actually exited. But again, when you when we were rolling out the framework, we knew that every task, every role didn't need the same level of operational discipline example. We like to use, and, and actually this is a power plant saying, but you may or may not know, that actually. Hydrogen is used to cool electrical generators, it is very combustible. So when we're handling hydrogen, that is a very, very high risk operation and it requires an extreme amount of operational discipline. However we also have technicians who change out gas meters out of residential home.

The risks there even though it was gas, it's a different risk and we need to make sure that our operational excellence practices are scalable to the risk that the organization can see or that the employee can see.

I heard josiah's presentation a while ago and he talked so much about culture and obviously innovation as he pointed out. And as OpEx, as we're talking about, operating, discipline is heavily steeped in culture, but how you attack culture directly.

We knew we couldn't do it that way. So, again, we worked to develop a framework that would basically foster the principles that we knew were important. And over time of applying that framework, then the culture will evolve to be how work is done at Duke.

This is our framework. I won't go into a lot of detail here. This will be shareable later if anybody's interested. Again, as we benchmark those other companies, some people would call this an operational access management system. We called management systems a little bit different in our framework, but we divide our OpEx framework up into four major attributes, accountability management systems, employed tools and training, and then continuous improvement tools. Each of these four attributes are then broken down into 15 separate elements that you'll see here. Again, I won't go into all the detail, but just to point out that under the continuous improvement attribute, you will see assessment program as one of the required elements. And so that's what we'll be talking a little bit more as we move forward.

Built on accountability, you'll see that in the pyramid graphic before, the culture of accountability is so important. And we developed this is our accountability model for Duke, I'm sure other companies have different accountability models. But it starts with the accountability and the center being the personal accountability of every employee, every leader. And then it is bolstered on the outside by leadership and organizations accountability which is a continuous improvement cycle of plan do check and adjust just a different way of saying PDCA. Or maybe another way of saying it is working the PDCA model into actual accountability model.

So, again, this is the accountability model for Duke. When we started down the path, different business units within our organization said, well, we have our own accountability model. And so, as you guys know, as you're trying to move a large enterprise, some consistency at the top, some overall sponsorship to say, this is the way we're going to do this is so important. And we were fortunate to have that at the time.

The other part of the framework is, or another required element is actually the leadership imperatives. And you'll see some of the same words that GSA use here. You know, you want all your leaders to inspire people, carry out the purpose of what your company, in our case, Duke Energy, is all about. We're always looking to the future and need to be able to transform.

Delivering the right results, the right way is where we really dig into where we talk about operational discipline, doing the right thing the right way every time.

Then, obviously, collaborations work as one. These leadership imperatives were already in place before we started the apex journey. But we made sure to include these as part of an integrated part of our operational excellence framework. And every leader in the enterprise has these, as their imperatives, and they are discussed with their manager at every review cycle.

So when we think about the framework and there are 15 different elements, but you gotta think back to our original objective, which is the highest standards and safety reliability affordability. So we like to think that each element plays into all three of those.

And the example we use is we have observations are required. We have a lot of highly technical work with technicians performing anything from high voltage work to again, pressurized natural gas to operating a nuclear plant.

And so, it is required by our framework that leaders will be in the field, observing those activities, making sure that our practices are being followed. And, but while we're there, the reason we want to think about this in the framework is while they're there, they they should also be thinking about reliability, is, is the task that's taking place the most reliable way to do it? Is it moving the enterprise to a more reliable product for our service, for our customers? And then also keeping in mind affordability.

If they're out observing the human behaviors of that person doing the task, they also need to be engaging with that employee and finding, is there a better way to do it? Are there any barriers to them getting their job done? And then that obviously those barriers and inefficiencies high right, back to the affordability of our product, to our customers. So, again, we have to think about using those elements that affect all three of our major objectives of operational excellence.

This is just another chart, kind of showing a little bit of our progress since the initial event that I talked about in 20 14. We started down a path of what we called FO event free Operations. That was really our focus. Stop having big events that we had in 20 14. Ideal C 15 and 16 is when we actually develop the framework, rolled it out across our major business or major operational business units. We did stand up a program office in 20 17. We started doing assessments and late 17, across 18 and 19.

And then, as again, as we've talked about, we'll talk about how assessments are currently being used, and how we're using them, and the way we report out up into the organizations, but this is sort of our timeline.

So I mentioned this before, assessment, assessment program, is what we call a required element of the framework. The 15 elements that I talked about, four, are operational units. They are required to be followed.

And the assessment element itself can go anywhere from self assessments of an operation of a business unit, all the way up to a large, independent operational excellence assessment that Mike actually facilitates. So, you'll see, again, required in the framework.

This basically says, no, it's looking for unexpected performance gaps, Is looking to see, are you doing work according to the standards that have been established for your work, either by regulatory bodies, environmental permits, the Nuclear Regulatory Commission, or perhaps by internal processes and practices that are focused on customer service, as an example. But it's all about looking for gaps, either to performance or gaps, to standardize The way we talk about it. I did add one quote here, from, obviously, some of our Lean manufacturing, Don't look with your eyes, look with your feet. So, this is our assessment model is very much about go, see, data is important. and data informs where we go look, but when we get out there.

Again, we're, we're looking with our feet by getting out into the field as much as possible.

So now, this is looking a little bit more at what I'll call the, the overall operational excellence assessment.

The way we do this, and it is formed by an independent review panel, my small group facilitators, we pull in experienced operational leaders from across the enterprise. So we'll pull in, typically, 15 to 20 operational leader from all of our business units, nuclear, transmission, customer delivery. That's our distribution, the smaller guys, fossil hydro generation, information technology, So we get, again, seasoned, experienced operational leaders, come in and serve as assessors of one of those particular business humans. So, we're right now, still, planning, to go back out to our natural gas business unit. These guys operate pipelines, natural gas pipelines, all the way down to distribution, to individual homes. Right up to the meter of the natural gas, coming into someone's home.

So, that is our current assessment, we're planning, it, will, that assessment team, will be formed, again, from operational leaders from across all of Duke.

Again, the idea here is to see one when we first started, it is, how are they applying the framework, have they applied the framework, are they using all the elements, are those elements effective in effecting the objectives that they're trying to achieve?

So, those were initial baseline type assessments. We're now pass that into cycle two and now, we're looking more into those performance gaps. And the whole idea here is, again, focusing on continuous improvement, moving up, that journey of operational excellence. The chevrons you'll see down at the bottom are really just how we how we do how we consider our process. The preparation is all about looking at data from the business unit. What what's their performance been like.

We will then actually conduct an assessment. We'll write up a report.

The business unit will get that report, they will be expected to respond with corrective actions as we call them. And then we go back on later, with a validation verification that they actually did what they said they were going to do, and have those corrective action has been effective.

So that's sort of a high level process of how we conduct the operational excellence assessment.

one of the things that, and we did borrow this from the nuclear industry, if you haven't seen it, or at least that's where we saw it, when, you think about assessments and oversight, which, again, is, is my role.

We're trying to look for early signs of decline before they become huge performance gaps, right, And the earlier you find those, obviously the, you put corrective actions before they become big events. If you don't see, if you'll see here as you progress through, if you don't see those early signs of decline, then you could be leading into something more significant As an event through oversight of practices oversight of your metrics and your KPIs and assessments, You're trying to shorten that cycle and see those faint signals early, too. So that they do not become events. What are those early warning signs? And that's the objective here. Don't wait for an event to happen, to react and respond, find the early signals that may be leading to an event. So that's why, again, we put so much emphasis on our assessments.

Oh, I mentioned this before, but that we put together teams of experienced operational leaders from across the enterprise. What this brings to us is a diversity of thought. They bring in very different perspectives. If you compare the way our customer delivery organization, these are the alignment that you see out on the poles. These are the guys that are responding during hurricanes.

Like we just had, their background is very different than that person who has operated a nuclear power plant for 25 or 30 years. However, they are both grounded and operational discipline practices, and any operational leader has those practices are already ingrained in their experience base. And those are, what we try to do is leverage that experience and that expertise, and now put it to the continuous improvement of the business units that were actually assessing. And I can tell you that there's real power in that diversity of thought.

And, you know, we'll talk more about that in a second, but So. our methods, as we mentioned, performance data, review starts. You may be thinking right now, how long does this assessment take?

We do it quickly. We basically cut out a two week timeframe. The first week, actually, more like three days, we'll bring in our assessment team, will train them in our methodologies. We will train them and how to do observations, how to document those observations. Make sure that they know the difference between an opinion, perhaps, an actual fact. We really strive to say, If we're going to bring an example, If we're going to create an area for improvement for our business unit, It has to be based solidly in something that you saw or that you can point to or something in the data.

We'd like your opinions, but they have to be backed up by facts. Basically. We want your opinions, actually.

So again, we try to teach them how to do our process for structured observations. And another key part is, obviously the interviews. When I saw it shows a graph of the, the happy people, and not so happy people, when you go out and pulse the organization, that was happy. People are the ones that are probably aligned all the way through the organization. But those unhappy people, or possibly pointing to, a problem in alignment to what the overall objectives that that leader is trying to achieve? And we don't necessarily go looking for those people. But obviously, if we hear from those folks, we will listen to that. And if we corroborate that with perhaps the performance gap, then that is going to be highlighted back to the leader as an area that they need to improve them.

As I said, so that's the first week. Second week, we hit the field, typically, three days of field observations, and again, you know, we're in seven states, so there is some travel involved. We go out for three days, We bring that team back. And we have a session that basically starts with trying to say, what did you see? And what are the themes that you saw? How do they compare back to the framework? What are the gaps and the elements, perhaps, that they should have been using that may allow this gap?

Actually happen. So, that starts the report, writing phase. We spend half a day to a full day operating on those common themes, and then, by Friday of that second week, we're basically writing what we call area for areas for improvement or strengths.

The areas for improvement again, have to be based on solid facts and examples of what we saw.

It's not atypical to see an assessment team will come back in our case from Florida. Another assessment sub team will come back from Ohio, they will have some observations that tie together to a common theme, if there is a gap in that organization, and that's what we're really looking for.

So again we assemble that into a report um one of the key things that we've been able to fortunately have and I saw that you know, Josiah uses governance.

So importantly in his, in the innovation chain, we have a strong governing body for operational excellence and that's what we call the Operations Council. Our senior exec of each of our business operational business units, report into this council. They meet every month. It is chair. It is sponsored by the COO.

And then he appoints a chair every year to actually lead that whole year. And that group is highly accountable for any operational issue across Duke Energy. So what we do on our Operational Excellence assessments, that is our audience. We give the report to the business unit that's being assessed. We also give it to the Operations Council. And the Operations Council then, can both look at learnings from that assessment for apps, perhaps for their own organization, but also to ensure that all of Duke Energy is is benefiting from this assessment. And that that business unit as accountable for those performance gaps that we may have highlighted, and that they're on a journey toward excellence.

Uh, I don't want to, I mean, I go into a lot of detail here about these Assessors and what we're looking for.

I heard some of the same characteristics that that Josie was talking about, but I pulled a couple of Malcolm Gladwell quotes here because one thing you want is is a passionate leader, no question about it, you want them to be an operational discipline and have some passion for continuous improvement, That's so important.

But the other thing is the that experience level that they bring, they may not be comfortable in the beginning translating that. It's not unusual for us to hear, you know, if we're going into a fossil hydro station and it is a customer delivery, though, the wires folks, they say, well, this is the first time I've ever been in a power plant, and we say, that's OK, that's actually great.

Because what we need is that objective opinion, And it is not uncommon for one of those leaders to look at an activity that's going on and, say, while I that looks, that looks difficult to me, or that looks perhaps hazardous to me? And that independence of thought is so, so valuable to our process.

So I talked about the journey. This is the way we describe the journey. It looked a little bit like Josiah four steps but we do look at what we call on a four point scale. Again, when we're having events back in 20 14, we call that you know, we need to get in shape and stop arming ourselves. So that would be what we call a level one level. To be better each year, Is getting the organization into a continuous improvement cycle on continuous improvement mindset. Level three is really what we, we know that in order to be the best regulated utility, we've gotta get to a level three, but we also know that regulated utilities may not be around forever. And we've got to understand that. We may need to be transforming to a different model. So that level four of op X is where we are looking outside the industry. Our industry to say, yeah, there's, there's something going on in the sector that we've gotta get to that level.

You know, maybe we're comparing ourselves more, eat, the classic. How are you comparing to an Amazon, or Google, or something like that? But a level three is, we should be at a solid level three. We're not in all cases, but, again, this is our journey.

And when we do the assessments, we actually do give our teams actually give scores based on this four point scale right here on each element. It's primarily used as feedback to the business unit. We are tracking them over time. But it's not a hard and fast metric that we're going to incentivize or anything like that. It's more of, are we moving forward on the journey.

So looking back on having done this for a couple of years now, I wanted to say, well, what are some of the keys that, that have been vital to us, and how we've done the assessment program?

And the first thing, and I mentioned this a little bit before, is that executive sponsorship has to be there and we have that through our current Chief Operating Officer and the Operations Council.

They again, they own the operations of Duke Energy. And we were fortunate to have a discussion from our CEO limb, good to the Operations Council, which, again, have these operational senior leaders in it. And she is looking for them to ensure that all operations within Duke, our event free, and are focused on affordability reliability, as we said, in their objectives. And whereas she can rely on that operational group to operate while she can look out forward and see what's the strategy for the company, What is shaping within government regulation that she needs to be aware of. And if we are dragged back into operational events that take the time of the whole senior leadership team, then she cannot focus on that strategy moving forward. And that's so, so vital for us. So, again, the Arts Council is an enabler to the CO two and make sure that operations are going the way they should go within Duke Energy.

This quote I put on here about, Get on the Dartboard. I mentioned that sometimes, when you bring an operational leader in, and they're not familiar with the business unit of which they're assessing, you'll hear some uncomfortable, no feelings, and they may not feel like they can add some value. And our COO kicks off every operational excellence assessment with a, you know, a charge to the team, if you will.

And one of the things he says, and it gives them a little bit of grace, is that, I don't expect you to find the root cause of what you see. Or, I don't expect you to find. The reason that it's there, what I need you to do is to get on the dartboard. I'm not asking for bulls eyes out of this team. I'm asking you to get on the dartboard and when you get on the dartboard, we hand that back to the business unit and there are expected to then find the wise Or that why is this the way it is, and narrow that into the bull's eye. And I can just feel. When we're doing this charge to the team from our chief operating officer, there is a relaxation in the room, because they know they've got grace. They've got grace to go out and look hard and find things. And they don't have to know all the wise as to why it's there and they also don't have to find the solutions. So, again, him giving that, that speech to them and saying, Get on the dart board, is so helpful in getting our team started on the assessment.

We mentioned this a while ago.

You want your assessors to have a continuous improvement mindset, you know, go find the gaps. That's what they're there for. And, again, you've heard this in Lean and six sigma type circles. You know, problems are like gold. That's really what you're looking for. And you really want to find it. If you can find the things that the business unit isn't seeing for their own, then that is true gold, because they didn't know it was there, in the beginning.

Yeah, you gotta have really good business unit engagement, And, again, some of this engagement is sort of driven by the fact that this is endorsed all the way up to our chief operating officer. Right? He also challenges the business units to say, OK, don't just roll. Don't just rollover on what this assessment team is telling you because you are going to have to go correct that. So, if you don't understand it, and you don't think the facts back that up, then he expects a healthy pushback.

And this is, this has been an interesting dynamic because, as an assessment team, you know, you're gonna get some defensiveness on the the business units is being assessed or from them. But that's expected as part of the process. There should be healthy dialog, because you don't want that All that dialog is doing, is helping you narrow in, on actually what that gap is, and why, and perhaps it exists so that, that business unit can then attack it FTE, after the assessment team leaves the Assessors, what are you looking for? Again, we borrow those from across our business units, and it has been, again, it's been such an exciting thing to see some of the growth of an assessment team. As we've done more of these, you know, our facilitators and are engaged in everyone. And, you know, we started out thinking, Oh, my gosh, this team's not going to get there. But they get their every single time, because their business unit leadership has put them in there. For a reason.

they know they have experience, they know they're passionate, and they know that they have credibility across their own organization, or they wouldn't have put them in the room. So, that's important.

The other thing we realized as an ancillary benefit is that every assessor that comes in and goes and assesses another business unit, comes away learning and knowing more about the company. Again, if you've been a, if you've been a wires guy, your whole career, and you've never been in a power plant. You just your experience, your horizon just got broadened significantly. And so now we're, again, this is an ancillary benefit, We didn't design it that way, but what a great developmental opportunity for a mid-level leader or even an upper level leader to go out into another business unit and look for performance gaps.

That, again, won't do that. That will help move Duke energy along that journey to operational excellence. So, it's been very, very powerful. The last thing is, you know the business unit has to be accountable for action. And again, that accountability is sort of thrust upon them by the Operations Council. And the Ops Council holds them accountable for correcting those of areas for improvement.

And again, that sustained oversight. Again, by having a clear tie back to the Operations Council, we do have this ongoing sponsorship.

Yep.

And there is constant feedback, as to are they on the way where they need to go? Are they seeing other gaps that they can be fed back into the assessment process for the next round, if you will?

So, how are we doing? These charts are just some performance, the ways, the way we measure some of our performance. The top two charts are all about safety and environmental performance. TCR is totally isn't the case, right? You'll see it has declined during our periods since we started off X. An assessment is obviously not taking all the credit for this, the Operational Excellence Foundation, fundamentals, along with a lot of hard work that had to come.

And the environmental arena following the Dan River Asheville that I mentioned. It carries over into safety as well. R E is what we measure in regard to reportable environmental events. You'll see a category one and category to their category ones or worse. That's where we get a lot of external media attention. You can see, we, we haven't had any for the last couple of years, and we, again, Category two is sort of that leading indicator. Perhaps, it was just lucky that the oil didn't get to a storm drain, perhaps.

But so the RISD TUEs, have been coming down as well. and so we're proud of that reliability. You will see two things here.

one is the reliability going up, and also our O&M being flat.

So, when we looked at reliability, we wanted to make sure that reliability, We weren't shooting for top levels of reliability of all of our assets.

And this, I'll try to explain that a little bit in that when you run a fleet of powerplants, they're not all running all the time. Imagine your hot, summer day. They're all running, but imagine you're cool spring day. They're not all running. And so every asset isn't necessarily treated the same way, and but they've got to be able to perform when you need them. So our reliability measures are structured across that. And so we we have what we call a reliability index, to make sure that we're not swinging the pendulum too far And saying, we don't want top decile reliability numbers out of all of our assets. Now, for the nuclear assets, we do, because they run pretty much all the time. And then the bottom right is our Net Promoter Score. This is just this is our measure, basically, of customer satisfaction. And you can see that it's been increasing over the period, as well. I will say this. We know this is a journey, and, you know, we're proud of these results that we show on this chart right here. But, I will tell you that we've had two significant failures within our company over the last two years, and that is that we've lost two employees while they were at work for doing.

So, I can tell you that the operational excellence team, and all the members of the operation's Council take this very personally.

And so, we know that we've got areas where we haven't penetrated the level of operational discipline that we need, and also probably mid-level leaders who perhaps haven't gotten the amount of operational discipline applying in their own workforce that we need. So, you know, we're nowhere near Level four across the enterprise, and in some areas, we know we're not at a Level three, but that is our continued journey. And, again, we are, we work on that every single day. So, I think, just say that concludes my presentation, apologize for the original technical difficulties I had, but I've enjoyed speaking. I would love to take any questions.

Add Your are very much worth the wait.

There's a masterclass on the theory and practice of operational excellence and operational discipline the reality on the field and the and the how how it's how it's done and the I love your, your openness, your vulnerability about what works and what doesn't work and and so really, I think those who, who have who are experienced practitioners Not textbook practitioners of operational excellence Will, Will, are thoroughly impressed by what you have done here. So That's That's incredible. It's a testament to to the great leadership you, and the team at Duke have had on this.

So the very first question I had was about measuring results, but you just talked about that, so you preempted the question, because you show some of the results there. You show for parameters, and there are probably more than that. Are there other things that you look at in terms of results beyond the four Panels that you share with us? Yeah. There's a whole suite of matrix, obviously. These are kind of the big four that we were choosing to look at, but, but, so, one way we do that is the operational council that I mentioned. They actually have a dashboard that they look at every month that goes, obviously, broader. And then, also deeper down to what we would call a tier two metric as well. So, if you think about, you know, each one of those has a lower level. Like nuclear reliability is very, very important. The Net Promoter Score has all kinds of subcomponents right?

On what's driving the customer satisfaction.

I will tell you that the affordability is still a little bit in more and conceptual space. We obviously know what our prices are, and we try very hard to maintain that was at a good level. But, when you say, how do you measure affordability now, you, you might get into things like what we call percent of wallet, right? Because we supply electric electricity and natural gas to, you know, very different groups, right? And so, different levels of socioeconomic status. And so I will tell you that we haven't cracked that nut on, are we truly affordable, because how do you, how do you break that down? But yeah, that there is a whole suite of metrics that we track as part of the operational council review every month.

OK, good, another question that we have here had to do about the frequency of this assessments, really kind of a flavor of how many assessments do you do on an annualized basis? Typically, how many people are involved in each one of these assessments? You talk about the teams, but I'm curious about what the size of those teams may look like.

Right, So, you can tell it does take quite a bit of work. So, our team, and, again, we look at our, our, our five major business units. So, we're trying to get them on a cycle of every 18 months, so that we would post. They've gotta have time to react, right? They've got to form an action plan, they've got to create corrective actions, they've got to implement those, and then there's gotta be some saturation time. So, we look on an 18 month cycle. when we would go back to that operational business units. So we target six per year, and we use, it's a team, typically a 15 to 20.

So that's basically the resource requirement. I will tell you in the beginning.

Know, when you ask for, I want to borrow your one of your leaders for two weeks. That's not an easy ask. Again, that goes back to that sponsorship, where we really have to have that senior leader sponsorship to say this is important. And as we talked about, there are ancillary benefits for your own organization. What business leader wouldn't truly, if they could get out of the myopic view of today? What business leader wouldn't want one of their leaders to go out and look at another business units to see what they can learn, Right? It's it's not just about finding gaps in the business units about bringing back that, bringing what you saw back into your own organization as well. So typically, we target six per year and the teams are made up of 15 to 20.

Farewell. Well, we have time for one more question here. So, ... talked about, about, about this, I made some commentary about the system that you have right now, for the assessments and, and how you become a bit more proactive for your organization. So, they don't explicitly ask a question, but he was talking about his interesting on the, on how, maybe how you're moving for towards the continuous improvement phase that you highlighted on your timeline? That you're moving more into that continuous improvement and innovation phase that you have such richness or if there's assessments of opportunities and different perspectives, as you said. That generate opportunities for improvement and innovation. Talk a little bit more how you envision. Move into this next next phase, where you accelerate improvement and innovation.

Yeah, it's, it is definitely, it's, it's not easy, right? And I think one of the things that we're recognizing, that we need to do, is dig deeper into the data and lower down into the key performance indicators, right? And have systems to do that. What, When we started this effort? Again, when I say business units, you might as well have said they were different companies right when we started. this operated very differently, very much in silos. And so when you start saying, I want to be able to dig deeper into the data, now you have all these different data systems that you're trying to deal with. So I think the opportunity for us is to really focus more, no, not at that top level of events. But get down into those precursor events. And where's the data for those precursors and those lower level KPIs. And how do you mind that? And not just mine it, but mine it in an efficient way, right? So, I think that's our challenge, and that's what, we got to work hard wrong.

And it's very thankful for your willingness to share the journey of Duke energy, and your leadership of this incredible assessment program. Thank you very much for that, where we're all very better off as a result of those insights. Thank you, guys. I appreciate the time. Thank you very much.

Thank you.

Ladies and gentlemen, this completes this segment of oil and gas live, Our final segment of the day is coming directly from the UK, where John Carpenter, the head of transformation for petrofac, is going to talk about their digital journey. So, this is going to bring a more technological flavor to the experiences that we have share with you here today. So, I'll see you back at the top of the hour with John and Petrofac.