By way of introduction I’m Charles Radclyffe, Head of AI at Fidelity. My background is 2 things:

1. To help organizations like Fidelity get the most from an early investment in technologies like AI. So I’ve done roles like this at Deutsche Bank, I was head of technology in their innovation lab.

2. The other half of my career has been working in the startup community, I built and sold 3 tech businesses, all B2B, selling into large financial organizations like Fidelity, which was a client of mine in a previous firm.

When we last spoke, you spoke about the different ways that AI is being used at Fidelity - different parts of the AI/ML lifecycle. You mentioned a matrix you used to frame the way that you think about this and the way that you plan it. Could you go through that for our readers? I’m sure it’ll be very interesting to understand how a thought leader like yourself is thinking about this in an enterprise context.

Sure. I’d start off by saying, regardless of whatever technology you’re working with, you’ve got to get to the money bit as fast as possible. Otherwise it’s just playtime. And playtime is fun, but playtime doesn’t pay the bills.

So I think the key piece is to understand what the art of the possible is for a particular capability, and where it can be used by the organisation in order to achieve financial objectives.

There’s really only three things that any firm can hope to achieve:

1. Revenue/Growth
2. Cost reduction
3. Risk mitigation

And the great thing about AI is that it can help with all three of these.

Particularly a firm like ours in the financial sector - we manage investments on behalf of lots of types of people and organizations. The opportunity for us is to use machine learning to be better at making investment decisions.

My colleagues in the investment management side of the business will say hey - there’s nothing new about this - we’ve been doing this for years - all the kind of fancy maths that takes place in the investment decision making side of things. It is the same as what we would call data science & ML. But then the problem is that if you only have capabilities that’s front-office focused and can’t be leveraged by the rest of the organisation at least you’re being inefficient, and not making the most of your investments. At worst, you’re leaving gaps on the table for potential capabilities that could be reused elsewhere.

So a large part of what I’ve been doing is trying to understand where the best pockets of excellence lie, and then to potentially help raise the bar and set up multiple centres of excellence.

What’s really interesting about AI vs. data analytics, which is probably what we’d be talking about if we were having this conversation a decade ago, is that with data analytics you were looking at data integration or visualisation as just one set of capabilities you were building. But AI is the whole set of things - everything from chatbots and robotic process automation to natural language processing and natural language generation to cybersecurity etc. So actually the roadmap for a firm like ours probably looks like multiple centres of excellence. So my job is to get set up for that.

The last point at a high level to mention is - I mentioned financial objectives, there are obviously things where it’s difficult to measure the financial impact. What we’ve got to keep an eye on is another objective which is how well our AI initiatives are aligned to strategic priorities for the firm. It may be that for any particular initiative that it’s hard to quantify how but it’s easier to see how something aligns strategically to customer satisfaction for an example.

It’s important to keep that lens.

There’s two things in tension - strategic alignment and financial impact. My job is to make sure we’re able to quantify how it aligns to strategy or delivers value.

That’s the high level piece.

What we’ve developed is essentially a maturity matrix. My personal learning in this role, at least to me, it’s been quite clear as to what’s important and what “good” looks like.

Something about AI makes it a bit more confusing for people.

What’s interesting is that some people come at this from a data background, some people come at it from a maths background, other people come at it from a software background. Each of those lenses is important but none of them give you enough - so we’ve had to figure out how to get everyone on the same page. One thing that was helpful is to look at what is best practice, in our industry and broadly, and what does “good” look like and where are we and where we are and then putting a plan together to get us to a higher level. And that exercise we’ve just completed, got a really clear mandate for what our ambition looks like for our CTO, and I’m just finalising that plan.

It won’t come as a surprise, it must sound obvious what we measure against. There’s three major themes:

1. Demand: Execution, and focus on demand.
2. Scale and efficiency
3. Governance

Within each theme there are three work streams which we’re organising and verbalising against.

Demand is is really about the execution on demand:

1. Prioritization
2. Skill & ability, capability - our internal capability and our relationships with SIs or boutiques because that can be important
3. Ecosystem engagement

What’s really interesting comparing AI… AI is not like Java. It’s not one thing. To get a best time to market you’ve got to make a build vs. buy decision. Sometimes your best time to market is to take Azure and get some people to develop something on your data, and sometimes the best thing is to bring something in from the outside. Vendors, academia and start-ups/scale-ups is a critical part of what will enable us to execute.

The third one needs a little more explanation because some people don’t get it intuitively.

To play it back to you, when you say demand you’re really referring to the demand from the business for AI solutions and the ability for the business to execute on delivering the solutions.

Yes. It’s important to recognise, at this level of maturity - at least in our firm - AI is part of our innovation world. Different to our BAU IT function and separate to our change function. I see us working in partnership to those two groups and of course the business. But the way we’d go about uncovering problems and building a backlog is different. We take a design thinking approach and what I’ve been trying to develop is consistency so that we’ve got confidence that if our budget is X, then X is being spent on the most important things.

If you don’t do that you end up in the situation where the things that are in the portfolio are either senior peoples’ pet projects, or junior people who have a lot of passion and who are managed by permission and that’s not using the org’s resources in the best way. The piece that’s really important is to make sure that every initiative we have is measured across those three vectors: risk, cost … objectives.

When you start those conversations you don’t walk into the chief people officer or the head of finance and say - hey we’re experts in AI … how can we help you? Or we’d like to build some AI solutions to fix your problems. Instead you start from a problem perspective and you don’t at all think about the technology solution. A lot of what we come up with is nothing that ML is ever going to fix but that’s fine because we’ve got other colleagues who we can push things to. But it’s the purest way of making sure we’re focusing on the right stuff.

That’s really interesting - it sounds like there’s a parallel there with finding product market fit as a startup.

Yeah, completely, totally. Only people from a startup background will recognise that. The job of an entrepreneur is exactly to achieve that as early as early as possible in the lifecycle. And it’s the same job as an intrapreneur and - a large part of my role is places like Fidelity is exactly the same as if I was out in the wild building a product - but I’ve got one customer, one patient, not many patients.

Sticking on the demand theme for a second, I guess that relates really closely to use cases in the business. Maybe you could highlight for our readers, the top 3 or so use cases you’re looking and where AI might get the best bang for buck in Fidelity?

The things that are most valuable to a firm like ours are the things that are the most sensitive. Some things you’ve got to think - the art is - some things you’ve got to focus in on the problems that people give you. But other times people won’t give you the answer because they don’t know what good looks like and they can’t imagine things being solved in a different way. It’s a bit like the cliché of Steve Jobs - no one asked for an iPad - the job in product-market fit, sometimes you’ve gotta create the market or help the market see that they need something they wouldn’t have articulated themselves.

For example - chatbots are one of the most transformational applications of this new generation of technology. For many reasons, but one big reason - the ecosystem/marketplace around the channel is the same fundamental shift we saw 20 years ago with the move from brochures to web, and we saw a decade ago from web to apps. And the next few years are going to be the years where chat and conversation agents really define the marketplace. That’s really interesting but it’s hard to bring that to people and say “look, hey this is going to change your world” and they just don’t care.

A friend of mine once said to me “the trick is to find the party trick” if you want to communicate impact. You’ve got to find something that everyone will resonate with. He used the example of Google DeepMind, no one really understands how it works apart from the people who built it but everyone understands what it can do - a machine playing video games better than humans. At that level that was good enough.

So I think finding these party tricks is the most valuable things you can do. There’s a couple of good example party tricks that every org will resonate with. Any firm over a certain size will struggle with things like meeting room booking - most people above a certain seniority get other people to do for them. How you want to do it is very much a conversational interface and how we do get to do it is through a clumsy web interface. And so saying to a chat bot “hey I need a meeting room for four people and video conferencing capability” - you can say “when do you need it for and how long do you need it for?” - “oh I need it at 11 and for an hour”. That’s the perfect interface. And if you have decent APIs internally you can find loads of those sorts of use cases.

Another good one is - “how many days holiday do I have left this year” - most people might have asked themselves - but they can’t be bothered to log into the HR system because it’s clumsy. Chat interfaces are good at those sorts of things.

And regardless of what it is, ML in its purest, or NLP, you’ve got to find a party trick. If you recognise potential transformational value but haven’t yet found use cases that are going to do it. Meeting room booking and holiday lookup are never going to change the world but what they do do is help people to understand what the opportunities look like and get better at bringing them.

That’s really interesting and I guess it points to a great deal of internal efficiencies that potentially can be gained from AI - and I think the other interesting thing about that example is that it helps people - or it helps me at least - consider - well if you can do that - what else can you do with it? And you can imagine all sorts of more advanced internal applications which people would find it easier to think about having already thought about the first example.

Completely agree.

So we spent some time talking about the demand side of the equation, shall we look at the other two?

Let’s pick governance next. It’s fair to say it’s pretty critical to most orgs not just to get the value but to protect the firm in the process. What’s really interesting about the way we’ve put the matrix together:

1. Demand
2. Scale
3. Governance

If you look at the nine work streams you can get a sense of how bullish people feel, if it’s a diagonal leading from the top right to the bottom left, or equally top-to-bottom, or heavy bottom right and lighter top left - you can see if people are feeling bullish or cautious. I’d say most orgs in the financial sector are likely to be feeling more cautious than bullish. But you never know. But useful to be able to identify that.

Within the Governance framework, I really thought when I started this that this would be the easiest and we’d just be able to take off the shelf some thinking that a consultant firm had done and just implement it and it would be no more thinking than that. But we realised pretty early on and we were disappointed that we hadn’t seen this as mature as we hoped it would be in the market.

Couple of things that will be a nuance to Fidelity, we operate in many countries, and counties that are diverse culturally. Japan, Singapore, HK, Mainland China, UK. Different to a company that operates across the EU or UK/US.

What we need to understand is how to protect the org in a way that’s sympathetic to each of those regions. So we’ve identified three workstreams

Regulatory - quite easy to understand - but the nuance is that we need to map out the regulatory regime and if you want to get ahead of the curve not just what the regulators are telling you today (there aren’t many today) but the question is what’s coming down the pipe tomorrow? We have to make sure we’re building things - if the monetary policy of Singapore says you can’t do this for this type of application - or might say you “have to use ML for AML” and a human to check - you have to make sure you have early sight of that. And you may decide you want to have a public policy discussion influencing regulators - may be a lobbying point there if you get particularly advanced.

Other two areas: we’re a bit ahead of the curve here, we split out Risk and Safety from Ethics.

What we’ve realised is that what a lot of people call Ethics isn’t really ethics, it’s just engineering best practice. So it’s really important to make sure that algorithms do not become discriminatory on gender, race, religion or any other protected category, and the values that we have as a firm get inadvertently breached as a firm by our code. That’s obvious and every firm is in that place where they understand that. There’s a lot of noise about explainability. No one’s ever explained to me why we call it explainability and not explicability which I’m pretty sure is the word that’s in the Oxford dictionary. Maybe just data scientists are illiterate. Maybe one of your readers will explain it and put me in my place.

People get a bit carried away with that, humans aren’t very good at explaining their decisions. What we do typically when making decisions is that we post-rationalise it. Why did I cut in front of that Tesla? Well there was plenty of space - and it slowed down? Actually why I did it? No way of knowing. We get so hung up about algos being black boxes but I think it’s maybe a little bit overplayed - that’s my personal view not Fidelity’s view.

But there are emerging best practices about how you manage the risk and safety aspects.

And then what we’ve been building is effectively a materiality framework - we want to make it easy for an engineer to look at what they’re doing - is this a high level of materiality and do I need to risk assess it in a particular way and follow some process, to make sure the firm is protected? Should be quite formulaic and the engineers should like that and that’s our risk and safety exercise.

Whereas Ethics we break out as something different - more around the application of technology, this is actually the area that’s under-addressed by the market. What I’ve certainly identified is that in the last few years we’ve seen a growing techlash, especially towards big tech. And I think a large reason for that is that because people feel uncomfortable about the encroachment of technology into various parts of their lives. And whether that’s rational or irrational we need to have a dialog with people as to where the boundaries should lie, if there should be boundaries and what people feel ok with. Unless you have that dialog we’re going to see people pushing back - and facial recognition is definitely in the crosshairs at the moment. And we’re going to see firms with the best of intentions make mistakes and we’ve seen Facebook really lose a lot of good will over the last few years because of that. The risk is that we make those sorts of mistakes and we don’t want to.

By separating out ethics from risk and safety it makes it easier for us to recognise that we need a different strategy to handle that as we do to handle risk - risk is all about standards and process and engineering best practices, ethics is all about how do we have a structured conversations at scale across our constituent stakeholder groups.

We’re ahead of most firms because we’ve thought about this and we’ve got a clear articulation of how these things interrelate. But we’re not yet at the point where we feel that we’ve got that nailed. And I think what’s most concerning say “hey we’ve got the ethics bit nailed, we’ve got an ethics board” and yet - don’t want to pick on Facebook too much - but they spend a huge amount on ML and Ethics and yet still have pretty big reputational problems because they keep getting it wrong.

They might not be joined up inside the organisation.

Exactly. The piece emerging to me - an observation for me at this point - the ethics piece has an impact & relationship with Demand. By part of having an Ethics conversation with your colleagues you have to explain to them what the art of the possible is. In non technology roles they might be aware of facial recognition, they might use it to unlock their iPhones, but they might not have considered how it could be used by the firm in different uses and therefore never have been able to articulate what they feel comfortable or uncomfortable with. And if they can do feel uncomfortable, they might not be able to articulate why. And by enabling people to be more articulate you can be clear about which use cases you might want to accelerate vs. those you might want to de-invest from.

You might be surprised - “I never realised that was possible - in which case I’ve got this other problem which is totally fair game to solve”. So I think there’s actually a commercial advantage besides from just doing the right thing.

Interesting - you don’t want to start down the road of projects once they’ve got some way down the road, people realise they’re not comfortable with - you’ll just end up thrashing.

Yes. The thing about projects is that people tend to get emotionally attached to what they’re working on. Often this is the biggest problem for innovators - you’re working on something, it’s a really bad idea - everyone knows it’s a bad idea - but you will not accept it because it’s the thing you just devoted the last 3 months to. And you’ll find every excuse and justification to cling on for dear life to it. That’s why it’s useful to have this conversation because often people will be working on stuff they shouldn’t and that’s the most dangerous thing.

That’s really interesting, and I can certainly identify that in my own life.

Yeah we all can.

And that points to the value of having the conversation about the art of the possible up front with the stakeholders so you can have those decisions points - whether to invest - prior to people getting emotionally invested in a project that’s already started. Cool - so those are the three topics under governance and scale is the middle pillar?

Three things under Scale: this is the one I found it hardest to articulate initially, but it struck me if you think about what ML really is, it’s

1. Lots of data
2. Fancy maths
3. Powerful compute on tap to be able to do the fancy maths

That’s the scale and efficiency piece, so these are our three work streams:

1. Model lifecycle, how do we originate the fancy maths and deploy it and monitor it and make sure it doesn’t go wrong.
2. Data strategy and is it mature enough to support the use of machine learning across the firm
3. What does our compute roadmap look like

Another dimension to ML - not just about having access to a cluster of CPUs you can push workloads to, you need to think about GPUs or ASICs which adds more complexity. Keep it simple - those three things.

So how you achieve it from a technology perspective - the maths (models), data & compute.

That’s been helpful for me to understand the 3x3 work stream.

It’s actually a 9x6 - 6 levels of maturity from 0 to 5, and 9 work streams. So it looks like this:

Passive/Active/Strategic:

• 0-1 Passive
• 2-3 Active
• 4-5 Strategic

This helps particularly non techie people understand “we’re only passive on the governance” so even without looking at the substance they feel inclined to push somewhere.

That can help to identify and motivate the areas which require the most investment.

I talk about these as gears, how strong do we feel about this?

A lot of orgs, when they’re first playing with the tech, will be doing the same thing - give people permission to use company resources in their spare time to develop skills; using company resources in their spare time or as part of their … and even have a bit of budget to bring in some external capability to test & evaluate. That’s the early stage stuff.

The stuff is when you really can identify there is value - and you have a plan for how to attack that in a structured way. That’s the major difference in the gearing. And the strategic is when you realise that actually when you’re good at this and top quadrant in your industry at this is a competitive advantage & opportunity for you. I’ll give you an example: level 5 in our matrix for demand is “to be able to deliver products or services or address markets in such a way that would have been impossible but for AI”

This really speaks to what good looks like - if we didn’t have this muscle built - this kind of muscle wouldn’t be possible for us.

In each of these - I’m sure this is a really helpful way of thinking about AI in the enterprise that will interest many of our readers. For each of the maturity scales, do you have where you are today and where you want to get to?

Really good question. I assumed that of course we’ve got to get to 5. Why wouldn’t we? But of course I don’t think it’s necessarily that clear that some people do. The way that we’ve done this is we’ve basically said where do people feel where we are now?

Rather than me saying - get some of our stakeholders where they feel that they are. And sometimes they might have a different view and that in itself is interesting, especially if they think they’re more mature than they really are. What I’ve definitely learned from this process is not to assume that everyone wants to get to 5, in fact sometimes people can feel quite threatened by the idea of 5, but it’s OK to say we want to be somewhere in the middle here. It’s really here. But what you absolutely must do and I wouldn’t compromise on this is you have to have that goal stated. Even if you’re further ahead than we want to be - it’s critical to know that. I think you can probably - easily spot the difference between firms that have the clear articulation of ambition vs. those who are just moving forwards with the best intentions.

I’d like to connect what we’ve spoken about today with Dotscience if that’s OK with you. What are the things that you’ve seen in the work that we’re doing that are particularly of interest to you in the context of this matrix?

Two things - firstly around the risk & safety aspect - to be able to do that in a fully developed way you need to have an inventory of all of the models that you have in development - either in experimentation, or at some level of deployment. In our industry we’re expecting that this is going to be mandated at some point. We’ve already gone through a voluntary process with the FCA/BoE (financial conduct authority/bank of England) here in the UK this year where they’ve asked 300 firms to participate in a survey to confirm where they think they are - to be able to answer that survey you need to have an inventory. That’s one piece.

The second thing around the model management - we obviously want to allow as much innovation to happen and make it easy for people to engineer models and - not necessarily deploy but in a beta environment testing them on real data and behind the scenes using them - we want to build CoEs, we don’t know how many they will be but to be able to do this, we need to be able to do this in production. At some point we need a platform to be able to manage the path towards production and we’re on the hunt. The hunch is that Dotscience is going to be able to help us there. (Either you guys or someone like you guys.)

In fact it touches three workstreams - we wouldn’t have met you and found unless we had a reasonably developed ecosystem management approach. It’s partly like someone like myself scouting pretty actively, but it’s also about having an org that recognises that you can’t just buy everything from Microsoft - as wonderful as they are to work with - they’re not the answer to everything. And that’s why AI’s not like Java. It’s much more exciting than Java because there’s so much diversity in the ecosystem you’ve got to tap into.

So you talked about a model inventory - I’d like to dig into a little to that - what do you see as the requirements for making that really useful?

My vision for what this looks like - a view of the org and understand who’s experimenting, who’s got serious applications that we think are going to go from lab to live. What is in production? Able to manage the lifecycle. And also able to manage where we are against the risk perspective. To have a single view on how much of what we’re working is in compliance, have these things actually gone through the materiality assessment we’ve set out? Has it been documented? Have they done a risk assessment (high/medium/low)? Have we accounted for model drift? Do we have high level RAG (red/amber/green) status on a single page - it’s all good today - here are our weaknesses and … where the risks are buried.

And you want to be able to sit down with a technology audit team, if you’re a firm like ours, with enough scale to have such a team, and be able to say “we’re good and this is why we’re good” - you want to have that platform to be able to say that. And the truth is that we all do that but we do it in Excel. And I think I’ve created more in Excel in my lifetime than I’ve killed, and I would like - it’s like a carbon footprint, it’s one metric I’d like to do better at in my life.

In terms of that inventory, one aspect I’m particularly interested in, whether it’s possible for a given model that’s making decisions - to be able to track the exact provenance of that model - is provenance a word that comes into your vocabulary? What are your thoughts on that with respect to model inventory?

I’m like a kid growing up in a poor part of town dreaming about the day I might be able to ride a bike to school. You’re asking me how do you feel about having any open top sports car?

It’s hard to fully appreciate - we’ve got to get the basics in place first. That’s where talking within my peer group - other orgs, the mistake people can make is to overengineer this from day one. And there’s a huge amount you can do just being pragmatic. This is why the maturity assessment matrix is so useful. If you’re on a 1 and you want to get to a 3, 2 might just be getting that inventory into a place where it’s not being manually created by someone and manually checked by someone.

Is it important to have provenance? I’m sure it will be. And to get to higher state of maturity it will be. But the risk is that it’s going to add manual work to people and they’re not going to be happy with it. Giving them tools to be able to control and govern and not feel like this guy Charles is taking away their time to innovate and now they’ve got to do admin stuff. That’s my immediate need.

It comes back to the beginning of the point - particularly on the money making side of things - why is everyone getting excited AI? It’s really ML. We’ve been doing it for years already. We’ve got quants in our firm. They’d hate if they have to deal with me who imposes a bunch of process and admin on them because we’ve put together a risk management policy. You will have them kicking and screaming all the way unless you make life easier for them.

To play it back to you, in your mind, the word provenance can be associated negatively with additional manual keeping track of things. And the consequence of that is that people push back, and say no I want to get on with innovation, I don’t want to do all this manual documentation.

It’s not just about provenance - what I understand provenance to be desirable for us - in particular having that technology audit conversation - but where I am at the moment there are some basic things to get done and the only solution is to insist that people do it as part of their day job and that will take away time from their ability to innovate.

I think you guys help me with that - and if it goes further even better. Just trying to look at it.

The model inventory is a non-negotiable. We need to have a single repository of where our stuff is. Even if we have 4, someone has to manually turn it into one!

But if you are saying we want to take a very strong view on governance and risk, and an ambition and a timeline, risk is you end up with a whole bunch of manual steps which defeats the point - the whole point is to automate.

Balancing the tradeoff between innovation and risk - and about having provenance - it’s nice to have in a sense - but there are more fundamental needs you have right now - it sounds like the ability to have an inventory showing which models are being developed where and what stage of the lifecycle they’re at and the model management piece which is the ease of being able to deploy those models into test and dev environments, then staging and prod, which is lightweight and not too onerous and which every data scientist has accessible to them.

Yes. There’s an elegance - what’s attractive about Dotscience is - if I ask for provenance right now I’ll get 60 people showing me where their models have their provenance and why they’ve got their shit under control. And my next conversation will them will be, that’s great - Sanjay, Sarah and Simon, but I need to make sure there’s consistency across the whole firm. Then everyone says “use my method!” And that’s equally as useless because then I’ve got a massive change management problem ahead of me.

Right, because everyone’s keeping track of things in their own way, with their own documentation and folder structure and spreadsheets.

You also touched on the topic of infrastructure management, the need for data scientists to be able to self serve compute resources? Do you have thoughts around the requirements on that?

One place again where our current state of maturity is quite low. Anything is better than where we are. Where we are right now, we have a whole bunch of things which are developed and have tactical needs and there’s a huge inconsistency. There are some things which are built in cloud, some things run in our data centres, some servers under peoples’ desks, which they’ll have found ways of getting in, and other things where we could take advantage of GPU acceleration but the teams building the models won’t be able to access that. Everywhere we look there’s either duplication or there’s gaps. We need to move to a place where everyone gets access to what they need and the firm is able to be confident that we’re making the most of what’s available.

I’ve definitely got colleagues and other people in my role will make very strong cases for why they need a DGX-1. And that’s the thing that’s holding them back. Unless you’ve got a fairly well advanced compute roadmap you won’t be able to hold your own against that. There are definitely firms that are spending money that they’re not ready for - the techies are going to be most happy - but my job is to make sure the firm is getting the most value all the time. You don’t want to have a new Tesla when there’s nowhere to charge it up, and you don’t want to have a leaded petrol car when there’s a fuel crisis and there’s nowhere to fill it up. You want to be in the mainstream most of the time with this stuff.

On the infra management point I guess a related point is that you shouldn’t end up in a situation where you’re buying specific bits of infrastructure - you need to be able to manage heterogeneous compute resources in a way that gets shared sensibly, across multiple teams, across multiples CoEs.

Yes, that seems like something I would say myself! We’ve got to get our roadmap in place. Need a better sense of current state. It doesn’t stop you from doing not stuff having this well understood and well built. But we don’t have a good enough sense for our current state to make a case for why better than current state should be pushed for.

Need to make sure we’re getting a positive ROI and managing the risk.

To recap requirements where you feel like Dotscience can help:

1. Having a model inventory, rather than many systems where people are keeping track of things manually, you can see the different models that are being developed across the org and the state of those models.
2. Model management in the sense of being able to easily deploy models into test & staging & production environments.
3. Infrastructure management, having data scientists have access to the compute they need when they need it across both on prem and on cloud, rather than an ad-hoc approach to provisioning and consuming resources.

I recognise the latter need but I recognise that the current approach leaves inefficiencies. But I think in terms of the next conversations and the models we have planned - the feature/model engineering/monitoring/post deployment, if the data scientists say - this helps me do more - and delivers the governance you’re asking me for with no extra work - then we’re in the win. I’m pretty excited!