Everyone is talking about AI adoption, but who’s talking about governance?
In this episode of PLATO Panel Talks, host Mike Hrycyk is joined by Will McIntosh (AI Governance & Adoption Lead, PLATO) and Stuart Ashman (AI Adoption Consultant, Leading with Quality) to discuss how organizations can embrace AI while putting the right guardrails in place.
From understanding how employees are already using AI to defining accountability, testing guardrails, and adapting policies as technologies evolve, the panel shares lessons learned from PLATO’s ongoing governance journey. The conversation also highlights the critical role QA professionals play in evaluating AI systems, validating controls, and helping organizations manage risk.
Whether you’re developing your first AI policy or refining an existing governance framework, this episode offers practical insights into creating guardrails that support safe, effective, and sustainable AI adoption.
Can’t use the player?
Listen to this episode on Spotify (opens in a new tab)
This option provides a more reliable playback experience on mobile devices.
Episode Transcript:
Mike Hrycyk (00:00):
Hello everyone. Welcome to another episode of PLATO Panel Talks. I’m your host, Mike Hrycyk. And today we’re going to talk about AI, specifically AI governance. Everyone out there is starting AI in some way or another, but I think that there are lots of topics, and one of the ones that doesn’t get talked about enough is governance and guardrails. So, I think today we’re going to talk about how we are approaching governance guardrails here and how maybe you should look at it and think about it. With me today, I have a panel of experts to help us in our talk. And with that, I’m going to introduce you to Stuart.
Stuart Ashman (00:30):
Thanks, Mike. Hi, I’m Stuart Ashman. I’m currently consulting with PLATO on AI adoption and the most efficient use of AI.
Mike Hrycyk (00:39):
Cool. Alright, Will?
Will McIntosh (00:40):
Sure, thanks, Mike. My name is Will McIntosh. I’m working at PLATO as the AI Governance and Adoption Lead, facilitating and corralling all the conversations going on around the company about AI.
Mike Hrycyk (00:51):
Cool. Alright, so I almost always like to level set when we get these things kicked off. So, the first question we’re going to talk about: why does AI governance matter now? And what does AI governance mean? And let’s start with you, Will.
Will McIntosh (01:03):
Yeah, AI governance matters now. I think we’re all familiar with just how fast this AI world is moving and that there’s an endless list of things that AI could do for you, but it’s not necessarily the right application to use AI all the time. So, understanding how to use it safely, productively, and in ways that actually bring value and not just extra noise.
Stuart Ashman (01:25):
Yeah, I think very similar to what Will’s saying, I think AI governance is how an organization decides which uses of AI are okay, who’s accountable for them, and how it knows they’re working. I think it’s now because AI has moved from suggesting to acting. We’re using AI to send, to book, to share on your behalf, and the model underneath can change without anyone deciding to change it.
Mike Hrycyk (01:52):
Yeah, I think the long and the short of it is people are using AI. They’re using it in their outer lives, and they’re using it in their work lives, and they’re using it. And some of that is good, and some of that is a little scary. And so, people who are running forward to adopt is great, but if they don’t understand what’s safe and what’s not, then you’re putting your company at risk. So, that segues nicely into this question. Traditionally, governance is something you set up before a practice exists. That might be arguable. But I think that ship has sailed with AI. People in companies are using AI. For good or for worse, they’re there, and they’re using it. Is it too late? Are we closing the barn doors after the horses have already escaped? Is it valuable to do it now? Stuart?
Stuart Ashman (02:41):
The simple answer is no. I think as testers, we do this all the time. We’re usually handed software that’s already been built, and we kind of start from what it actually does, not what the spec says it should do. Governance after practice is the same. You start from what people are actually doing with AI day to day, and I think that’s better evidence than a policy written in advance, which you’re probably making up.
Mike Hrycyk (03:03):
Great. Will, anything to add with that?
Will McIntosh (03:06):
Yeah, reiterating what Stuart has shared, it’s not too late. AI tomorrow is dramatically different than what it is today. It’s just changing so fast. So, that being said, nobody is really ahead or behind in this whole AI world, with the nature of how quickly everything’s moving. A good governance structure will lay the foundation, and that sturdy foundation allows you to be prepared for when that change comes tomorrow, or the next day, or the week after that. So, having that governance foundation can let your organization grow and adapt to those changes as they come to you.
Mike Hrycyk (03:37):
Yeah, I think that’s a good point. It brings to mind an example, and I should have went and looked up the term. So, in urban planning, one of the things that some interesting campuses have done that are at the forefront is the idea that you don’t lay the pathways in your campus beforehand, because what happens is you put your sidewalks, and then students and faculty come, and they walk around the campus. And what you discover when you look at an aerial shot is you have all these nice sidewalks and people probably use the sidewalks, and then you have these paths that are formed in the grass because they’re shortcuts. And the shortcuts are not always easy to predict which way people are going to go. And so ,some urban planning on new creations I’ve seen is they just don’t put the sidewalks down. They let people walk around, and then they find out where people are going to walk, and then they lay the sidewalks.
(04:20):
And so, I think that’s something you said there, Stuart, which is really interesting: if people are already starting to use AI, let’s see where they’re using AI. And if it’s not terrifying, then let’s put the guardrails around that. So, the natural proclivities is going to make it happen better. People are already embracing it, they’re already engaging with it. Let’s put some guardrails around it so they can do what they’re already going to do safely. Learning what people are doing so you can build governance around it.
(04:47):
So, how does governance take into account what’s already happening within your org? So, maybe before we even get to that, Will, why don’t you tell us a little bit about PLATO’s journey into governance and where we’re at in that process? And then we’ll talk about, and maybe we can include us as we go with that, how does governance take into account what we’ve already been doing?
Will McIntosh (05:06):
Yeah, so, we started from scratch basically a couple months ago in the middle of this past summer. I think Stuart came on board with PLATO, and he was working with the team and looking to drive engagement and get the conversation started. There were no governance policies or directions in place yet, really. We didn’t know what we wanted to do. We didn’t know what we were comfortable doing. So, since I’ve come on board, it’s been, okay, how do we put this directional governance framework together that thinks about what we’re comfortable with and thinks about what we want to accomplish as a business with AI?
Mike Hrycyk (05:43):
Stuart, so you came, and you joined, and we’ve been doing this. How have you been taking into account how we have been using AI, and how is that driving the governance process?
Stuart Ashman (05:53):
First of all, you need to focus on making it safe for people to tell you what they’re already using. You don’t want shadow AI. If they think what they’re doing could end in a ban of some kind, then they’ll hide it. So, I did a bit of discovery first: what are people using, how are they using it, and what are you using it for? And then you can sort by what the AI is doing for folks. So, if it’s only suggesting, then a person checks the output, and you’re okay. If it acts- sending, booking or sharing, as we mentioned before- that’s where you need some testing first. And that’s kind of where we’re at right now is defining some of the policy and the governance around those pieces.
Will McIntosh (06:31):
We can’t govern what’s already happened. I guess, like Stuart was saying, you can inventory and get a lay of the land on what people have been doing. And then as this governance comes together, you can adopt what holds, what changes, what happens next with the work that’s already been ongoing.
Mike Hrycyk (06:48):
So, I’ve read some governance documents from other places, and sometimes they read like your typical Microsoft ULA [User License Agreement]. They’re 900 pages, and they’re written by lawyers. And when you read it, you’re like, I don’t know what I’m allowed to do or what I’m not to do, which have one of two effects. Either people are terrified, and they just turn away, and they don’t use AI, or people, like, they use a Microsoft ULA, they put I agree and then do whatever they want. So, how are you guys approaching building an accessible, usable governance that people will follow so that it actually acts as a safety guardrail?
Will McIntosh (07:24):
Yeah, no, that’s a good point. So, I’ll be completely honest and transparent: my governance reports are quite tedious and long reads, but my intention with them was never to put them in front of somebody and say, ” Read these governance documents and adhere to them. It’s a mechanism to form direction on policy items- policy that can be much shorter to read, much more tangible direction. And in the governance, it kind of lays out the ideas of what we’re comfortable doing and what we hope to accomplish, and what we aren’t so comfortable doing. And then that, in turn, allows you to form those directional and authoritative statements, such as the use policy.
Mike Hrycyk (08:04):
And I think a really important thing for us to include in our conversation is AI is not done. It’s not done evolving. We don’t know what it’s going to look like in eight months. Everything is always changing, which means you can’t write governance that covers every situation. So, it’s much better to write general guidelines that, accompanied by common sense, will try to keep you safe, I think.
Stuart Ashman (08:26):
I don’t know how well I’ve socialized this idea, but my internal rubric for this is the day-to-day governance: the practice of applying governance to AI needs to be like an under-a-minute test, short enough that while you’re making a decision about can I use AI for this, you can answer that really quickly and really easily. And a 900-page document will never do that. So, if you can’t answer that with some simple guidance, then the guidance is failing, not the people. We will probably need to have legal documentation that we train people on annually or biannually that goes through much more detail, but the day-to-day needs to be something someone can really make a quick decision on. You mentioned common sense, but sometimes common sense isn’t common, so it needs some training to back it up.
Mike Hrycyk (09:12):
So, how useful is it to use AI to write your governance framework? I am certain you’ve both tested it. Has it worked out? Is it a good choice? Will?
Will McIntosh (09:23):
Yeah, I guess that’s kind of the meta of this discussion is I certainly did use AI to help me produce our governance documentation. Not simply saying, “Hey, I need a governance report for a medium-sized software testing organization go.” It’s back-and-forth conversations with others and feedback and lots of human edits and tweaking as well. But that’s kind of the beauty of AI is it can synthesize those thoughts.
Mike Hrycyk (09:52):
So, Stuart, any stories where I went way off base?
Stuart Ashman (09:56):
It easily can. A lot of it is around, as Will was saying, how much context do you give it about what it is you’re looking for. But we’ve also talked a lot about the guidance, and I have a bit of a bias here coming from a QA background of test first. So, knowing what you want the output to look like before you ask AI to produce anything. So, if you want a 900-page policy document, you can ask AI to produce a 900-page policy document. It will do it quite happily. But if you come at it saying, “I want something someone can actually reference and use and make a quick decision on to know clearly, yes, this is appropriate, this is not appropriate, or I can or I can’t,” then giving it that guidance will typically get you a better outcome.
Mike Hrycyk (10:38):
So, a big question that people should ask: who should own AI governance in your company and who is accountable when something goes wrong?
Stuart Ashman (10:47):
So, one named owner is important. Naming somebody is important. You don’t want a committee. A committee can advise; they can guide, but someone has to be able to say yes or no. The AI did it is not an answer that we should be accepting from anybody in the sense that you were just saying if the AI writes the policy document, is AI the actual owner of the policy? Well, no, we need a named person.
Mike Hrycyk (11:09):
What skills does that person need to have?
Stuart Ashman (11:11):
That’s a good question. I think, like in any role, they need to know what the ultimate responsibility is and how they need to enact that. So, in terms of policy and governance, then it is that. It’s how are you helping people make appropriate decisions, and how are you owning the outcomes of those decisions?
Mike Hrycyk (11:29):
Any thoughts, Will?
Will McIntosh (11:31):
Yeah, there are kind of two halves to this solution, I think. As Stuart was mentioning, there’s an accountable owner. So, who owns the policies, documents, and the governance documents? The organization probably owns it, but you also need buy-in from all the members of the organization, and that’s ownership in its own respect as well. Does the governance align with how they want to be using AI? And does it let them benefit from using AI, or is it something that’s a thorn in their side, saying, “I have to be so careful, I don’t know what I’m doing,” things like that. So having buy-in from the entire organization is important as well.
Mike Hrycyk (12:07):
Also important to remember that AI isn’t as monitorable as security, right? So, one thought that people have, I think, is that the CSO, the security office, should own it because they’re already tracking things like that. And I’m not against that idea, but one thing that I think needs to be layered with that single individual idea that is the owners they have to be engaged, and they have to care, and they have to be fluent enough with AI that they understand what people are trying to do, or else they might just say no, and that can really get in the way of moving forward. But I think there needs to be training offered to your managers. And the reason for that is it’s not as monitorable. And if you don’t talk to people and know what they’re doing, you’re not going to see flags that say that something might be going off the rails.
(12:53):
And we’re IT folks, so a lot of the time our conversations are centered in the IT area. And the IT area can be super scary because people can do so much more, and they can get into so much more trouble. But a person who’s curious in sales or wherever could be using AI for all sorts of things and could be dumping your super-secret insider trading information into AI, and it could be getting around. And that’s not necessarily something that’s super easy to spot. So, by training managers so that they have conversations, so they say, “Hey, what are you working on that’s cool?” And listening for those flags is important. As well as giving people the tools that they may want to use, give them enough access to the things they might want to use such that they can do things. Sign an agreement and use Copilot so people don’t just go to ChatGPT, or so they do stuff within the guardrails. So, it’s set up good guardrails that help people get to where they want to go.
Stuart Ashman (13:47):
I think one additional point: there’s a difference between who’s accountable at the business or organizational level and who’s accountable for the individual things that you might be doing with AI. So, you mentioned the sales [team], for example. I think there’s individual accountability needed for anything they’re producing or doing with AI. So, if someone produces something which does leak company or client data, that individual is responsible for that. The governance covers how they’re accountable for that and what they should and shouldn’t do.
Will McIntosh (14:16):
Yeah, first of all, PLATO owns the AI culture within PLATO, and part of that culture is the governance on the ground, real use cases. The governance should be able to tell you, when you have something in practice, who is responsible for maintaining it and the results.
Mike Hrycyk (14:33):
So, let’s talk about QA. So, QA needs to be a part of AI, and QA needs to use AI as part of their roles. But more specifically, does QA have a role in AI governance? Other than following the governance on their own, does AI have a broader role, or are they just like anyone else?
Stuart Ashman (14:52):
Yeah, I think they absolutely do. I think the way I think about it is governance writes the rules, and QA produces the evidence that they’re being followed and that they work. So, drawing on that previous analogy, I can reference a recent public news article if you like. It was a Postgres[SQL] team that created a list of 40 defect reports, and the tool found all of these. So, AI actually found these defects. What ran out was the human time to decide what mattered. So, the release slipped, and this was public news, and that judgment is QA’s job, in my opinion. And talked about in this article. And it’s becoming the scarce part, right? Like the judgment on what you should or shouldn’t be shipping or how you should triage or understand issues or risks.
Will McIntosh (15:34):
So, governance is providing direction on what is allowed and what we hope to accomplish. And QA is, like Stuart said, proving that it holds, that the guardrails we put in place are valid and being respected, practicing what we preach, I guess, as a software testing company.
Mike Hrycyk (15:52):
Yeah. I mean, I think QA in a lot of ways in all of our roles are watchdogs. We’re advocating for the client, for the end user, and we’re making sure that we’re doing that. Just another way that we can do that is, if we’re seeing AI being used, we’re advocating to make sure it’s being used in a responsible way that’s going to make the requirements filled in a way that is safe and matches the needs. So, I think QA doesn’t get to step aside. Maybe they’re not the owner-owner, but I think there’s definitely a role.
(16:19):
How do you test something when you can’t guarantee the same answer every time? And to sort of add to that, does governance play a realistic role in the practice of reducing hallucinations, bias, security vulnerabilities, inappropriate responses, and performance degradation, or is that outside of governance and just part of the role of QA?
Stuart Ashman (16:38):
The last one’s a hard one to have a definitive yes/no answer to, but in order to answer the how do you test a non-deterministic system or something that’s going to give you potentially a different answer each time, you stop testing for an answer, and you start testing for more of an acceptable answer. I think it’s the easy way to think about it.
(16:57):
So, we’ve talked a lot in PLATO about deciding what good looks like and what wrong looks like before you start running something, interacting with AI in some way. And you need to do that more than once. Because it’s non-deterministic, you need to do it any number of times. And we could go for an infinite number of times here, but each time you are running something that’s non-deterministic, you’re checking against something real, but something acceptable, as opposed to I expected the answer to be three kinds of thing. You’re checking against: I expect the answer to be in this range or to be something in this frame. This isn’t a new problem, I don’t think, for QAs. We’ve always had an Oracle problem knowing what the source of truth is or what the correct answer should be. So, we already know how to handle variable results. I think performance runs, flaky timing, and flaky automation- these are all things we deal with every day. This is in the same sort of space. I think AI makes the Oracle problem bigger, perhaps more of the whole job.
Mike Hrycyk (18:00):
Any thoughts around that, Will?
Will McIntosh (18:02):
Yeah, just how do you wrap your arms around or put something in a box that doesn’t really have a starting or an end? So, you have limitless numbers of inputs and what is supposed to come out on the other side. So, I think what you can do is test the boundaries and test those guardrails that you’ve put in place. That’s probably more attainable than predicting and testing every possible input that you could put into it.
Mike Hrycyk (18:30):
Yeah. I’m reading a book right now called the Eval Mindset, and it’s testing the assumptions of how you’re using AI as you’re moving along. And one of the ideas it has is when you are doing something new and you have one assumption you’re going to test; you test it 10 times. And if in those 10 times you get 10 answers, they’re not going to be the same 10 answers because AI doesn’t work that way, but if there’s 10 answers that you like, that gives you a certain amount of confidence and you move on. As you go through your testing life and you test that specific thing more and more often, you can test it less times because you raise confidence. Does that mean that the AI is not going to become a wingnut at some point and do it differently? No, but you raise your confidence.
(19:09):
And then in the same way as when we test an area in a program, and we find a couple of bugs, then we test more in that area of the program. So, if you do those 10 tests and three of the answers that you get are not really acceptable to you, maybe you make a change, maybe you don’t, but what you certainly do is test 10 more times and start to see a pattern. And then ask within yourself, ask on the team, figure out how you’re going to determine is 70% good answers good enough? And sometimes it will be. And you look at how bad are the 30% of the times when it’s not good enough? Is it almost good enough? And you have to make assumptions. And we start off with ourselves and figure out if it’s 70% wrong answers. I don’t care what a product owner’s going to say; that’s not good enough. But maybe the 70% is good enough, and maybe at that point you can include a product owner and start having discussions about what tolerance is and stuff like that. But it’s still risk-based testing, and it’s still understanding what the risks are and the confidence that you built.
Will McIntosh (20:09):
Yeah.
Stuart Ashman (20:11):
And the variability. Do you want me to answer the second part of your question, Mike?
Mike Hrycyk (20:15):
Yeah. Well, sure. That was the scary part.
Stuart Ashman (20:18):
Well, I put it off because I said it wasn’t a straight answer. And I think governance doesn’t reduce hallucinations. I think testing does, but reduce is the keyword, I think. Governance makes sure the testing happens. That’s part of your policy to say, we’re going to make sure we test in these conditions or with these people or in this circumstance. And it should happen every time something changes because we’ve already talked a bit about this, but the models can change without someone deciding to change it, and that could introduce drift in whatever answer you’re getting out of AI, whatever you’re asking of it. So, one of the things we’re looking at is how often and when do we retest? What triggers that, basically?
Mike Hrycyk (21:00):
I mean, it does make automation more difficult if you change a model and your answer changes. Automation is built on the idea that it has to equal exactly what you’re looking for, exactly a list of what you’re looking for. But AI likes to be creative and use different ways of saying the same thing unless you rail it in so hard that it can only respond with these five words in this particular order. That’s just another place that human eyes are demonstrating value, which is good. It’s good for us anyway.
Will McIntosh (21:32):
Mike, if I could add to this, just governance isn’t itself reducing the likelihood of hallucinations or bias or performance degradation. It gates the organization’s appetite for those inherent risks, similar to that 70% wrong example you were using. Governance can say whether that’s acceptable or not. And also, out of governance, it’s creating the policies and the processes and controls that will reduce the likelihood of failure and increase your confidence in your ability to understand what might go wrong. I guess. What the potential harms are. So, one of those processes would be your testing strategy of your AI solutions.
Mike Hrycyk (22:15):
Put a question to you guys. Do model changes, do they have minor and major revs like software normally does? Do they do that, or is it just this is a new rev?
Stuart Ashman (22:23):
I guess it’s a bit of both. I think each Frontier Lab might have its own way of thinking about this, but we typically see two versions: a major and a minor. We don’t see the kind of dot revisions below that.
Mike Hrycyk (22:35):
So, I think that there’s maybe room within governance to also indicate a major rev requires that you go back and test it 10 times again, and a minor, maybe you only have to do a couple of tests. Or whatever, the super, super high-level idea of the example. But I mean, revs aside, a lot of your AI is learning all the time, which means you’re not necessarily going to get the same answer on Tuesday that you got on Monday.
(23:03):
Alright. How should you set your governance up to evolve and change as the organization moves, as AI changes? How are you guys, I’m going to say, future-proofing, but that’s super laughable with AI? Will, how are you attempting to future-proof your governance?
Will McIntosh (23:21):
Yeah, not pretending to have all the answers, I think, is the mindset to have. So, one job of governance is creating rules, directions to create rules from, but another important part of it is the innovation and stewardship piece. Having an eye on the future, not trying to predict it, but reasonably be prepared for what the industry’s leaning towards. AI today will not be the same as AI tomorrow. So it’s just important that your governance doesn’t claim to be perfect and that it’s open for assessment and changes to it as the need arises.
Stuart Ashman (23:59):
Yeah, I think again, the tester’s mindset comes to me of being risk-based. So, I think we can’t predict the future at all, so we shouldn’t try, but what we can do is evaluate the risks. So, as the reach or the impact of how we’re using AI grows, I think the rules need to tighten up in consideration of that. So, if we’re using AI to make very critical business decisions, then the rules around that, including when and how often and how much you retest something, become more important. If it’s just helping you decide what you’re going to have for lunch, then maybe not so much, unless you have a serious allergy. It’s really just around that sort of how do we see the risk as things change and it becomes more and more embedded into our everyday, into every part of our workflow or our business, the reach of it, the impact of it grows, and therefore the risk grows. We need to tie it up around those bigger impacting pieces.
Mike Hrycyk (24:47):
I’m going to say something that’s potentially controversial. So, I think it’s important to get governance out there, write it. Don’t take two years to write it. Get something out there that people can start using. But what I think is also important: AI is changing a lot. Don’t release a new version every week, or I’m okay if it’s a living document; update it, but don’t get people to read it with the new stuff every week because eventually they’re going to stop reading it and they’re going to stop caring. And so, when you push the new versions out, wait until it’s big and it’s important and make it seem important so that people will ingest that.
(25:19):
And then the additional part, I just thought of this during this conversation, so it could be wingnutty or impossible. I think it’s important when you have that one person who owns governance, that they’re a governance mentor. So, they stay in touch with the organization. They use managers to funnel them who is doing interesting things with AI, and then they go and talk to those people and act interested, engaged in what you’re doing and not so judgmental, and then gently nudge them in a few places where they’re not being as safe as they could be. Or when you’re handing the secrets to someone else, maybe judge them harshly and come down like the wrath of whoever. But I think that’s the only way it’s going to work. Stuart, what do you think of my potentially controversial idea?
Stuart Ashman (26:00):
I think there’s a risk we conflate governance with a document or a policy, and I don’t necessarily see it that way. So, earlier on, I sort of mentioned this idea that you should better make a decision within a minute, and that decision could be, I don’t know enough; I need to ask somebody; I want to use AI in this circumstance, or I want to deploy an agent that’s going to do these things that’s going to touch a customer’s account in these ways. And we don’t know what the scope or the reach of that really is, then we should be figuring that out. What I’m trying to say is that the governance is really helping people make appropriate decisions at any point in time. So, it’s not like you must do this always or in these circumstances you can do this other thing, because that’s too detailed and too much for someone to reference at any point in time.
(26:44):
You might have that documentation as well, but the point is that the common sense in the moment piece that we need to be sharing with the managers, the leaders, and the individuals is how they make a good choice and, to a certain extent, what their accountability is, what they own. If they are deploying something that, given the example I just gave, touches a client account in some way, then they own the outcomes of that. And they need to understand what that means, what they’re responsible for. And if they’re not comfortable with that, to check that with somebody. So, back to your mentor idea, that could be the mentor is then going to help them make good choices around that.
Mike Hrycyk (27:16):
It strikes me as being similar to the idea of the mission statement and how it’s supposed to be used and isn’t almost everywhere, right? You come up with a mission statement for your company that says, “We really want to do this.” And then whenever you come up with a question of, “I want to move forward in this direction,” you’re supposed to go back and say, “Does this meet our mission? Is this on brand?” And so, your governance document becomes, “Is this on our AI path, our AI guidance, our AI mission?” And I think that’s a good way of looking at it, only I hope we adhere to it better than people do to the idea of a mission statement.
(27:50):
Okay. Once the guardrails are in place, how do you know that they’re working? Doesn’t feel like you can measure this. Stuart, let’s start with you.
Stuart Ashman (27:58):
So again, I think unsurprisingly, I’m going to go from the testing aspect and say, you know, the guardrails are working by trying to break them. If you put some guardrail in place, you test it. If you haven’t tested it, it’s just an assumption. I can give you some real topical examples if you like, but it’s that same sort of tester’s instinct: we say a piece of software is meant to do a thing under certain circumstances, we test that it does that, and we try to break it to see when it won’t, for example. When it doesn’t honour that. It’s the same thing with the guardrails in AI.
Mike Hrycyk (28:27):
That sounds dangerous, Stuart. You want to have someone put our data out there in AI just to see if it’ll come back, but I don’t think that’s what you actually mean.
Stuart Ashman (28:34):
No, I mean as we’re building with AI, so we’re putting guardrails in place. We instruct an AI agent to do something, and we might use terms like never do this or always do that or before doing this, do this other thing, check with us or ask us or something. If we put that guardrail in place with AI, we need to make sure that guardrail’s actually going to be honoured by the AI. And back to the earlier conversation, as the underlying circumstances change–that could be model change, that could be tool change, that could be something else in the variable space changes– you need to retest that to make sure it’s still going to follow and honour that guardrail.
Mike Hrycyk (29:10):
I like that. I hadn’t thought about this. And I mean, I’ve reviewed your draft guardrails, but it’s been quite a while. And so, I don’t know if this is in there; maybe it should be. Part of writing your governance document should be talking about using appropriate guardrails when you’re developing using AI, which means putting a limit that says never do this and talking about that a little. So, I think that’s a useful thing to have in that document, but then you’re right, that’s something very specifically that you have to test, that those guardrails are working, and that’s something else you put in the document. Build these guardrails, also test what you just built.
Stuart Ashman (29:42):
And know that if you wrote that in the document, you’re probably changing it quite often because it’s specific enough that actually that notion of how AI either uses or doesn’t use guardrails is changing all the time as well, and the different models will apply them differently.
Mike Hrycyk (29:58):
I mean, in the agent I’ve been working on, the guardrails I write are less about safety and more about, I want you to do this first and then that. It can get really difficult to get AI to believe what you’re saying is what it should do, right? So sometimes you have to get very prescriptive.
Stuart Ashman (30:13):
Yeah. I don’t know if you want me to quote news items, but we’ve heard lots of instances where the AI is very outcome-focused. It’s trying to achieve what it thinks you want it to achieve or what you’ve told it you want to achieve. And it will often effectively overwrite or overrule itself in terms of how it achieves that thing because it sees that as the highest priority and not necessarily following the constraints that were put on it.
Mike Hrycyk (30:35):
Will, anything to add?
Will McIntosh (30:36):
It’s one thing to build an agent and test the agent and try and break the barriers you put in place. It’s another thing when that agent gets shipped into the world among your own team, how are you monitoring its outputs in their production environments? It’s a real challenge. How do you keep an eye on it?
Mike Hrycyk (30:55):
I don’t think that’s something that we’ve nailed the answer to. I don’t mean us specifically. I don’t think that tech has nailed the answer to that question. How do you monitor agents when they’re out in the world? And that’s a scary thing that will keep us all up at night, I am sure.
(31:07):
Okay, time to wrap up. We’ve had a really good conversation so far. A little piece of advice from you guys, as someone who is starting their AI governance journey, they have nothing today, they want to get started. How do they get started? What do they do in the next 90 days? Will?
Will McIntosh (31:23):
Yeah, so day 1, you’re looking to understand what exists, inventory, how people are using AI, what they’re using it for and what technologies they’re using. Understanding what you’re trying to accomplish. Are you using AI just because it’s a shiny tool that seems really helpful, or is there a real reason for your business or your organization to be using AI? Understanding your goals, understanding your comfort and the risk you’re willing to assume. And then don’t wait to have all the answers. And as I’ve said a few times now, it’s just that you never will. AI is changing so quickly, and it’s one of those things where you just have to prepare yourself enough that you can start experimenting and making mistakes safely, but having the flexibility to adapt and make changes to your strategy as you go along.
Mike Hrycyk (32:11):
Cool. Alright, Stuart.
Stuart Ashman (32:13):
I guess, in brief, as Will’s already mentioned, find out what’s already in use. Name an owner, so make someone own the problem space if you like, and then pick an AI use that matters or matters most, and then decide what good and wrong look like, and test it and understand what rules it has and try to break them. And obviously do that more than once; do that several times over so you have a good understanding. Essentially, what we’re really saying is that the policy follows the evidence, not the other way around. So, you learn what people are using, you learn what good and bad looks like for what you are currently using AI for, and you figure out how to test it and how to challenge the guardrails. And then you write policy that says, this is what we like about that and what we should keep doing versus what we’re not so worried about and therefore don’t need to do or in what levels you associate those things.
Mike Hrycyk (33:03):
Great. I mean, I think it’s really important not to take 90 days. People out there are using it, people out there that you never even thought of. So, you might do your investigation, but there will be some other people that you didn’t talk to who are using it, possibly for incredibly stupid things or possibly for incredibly great things. So, get something out there so that people have some guidance. Then I think the corollary, or the flip side of that, is that having a company send the guardrails out is a way of letting everyone know that AI is acceptable at your company. It’s a useful tool. Use it within these safety nets. And then the people who were scared figure out ways to use it. And then who knows what it’s going to do, what it’s going to do for your company.
(33:44):
Alright, well, thank you, guys. Thank you for joining us for a really good discussion about AI governance and how to get started and where you’re going. I hope our listeners can use some of this advice to get their own journey underway or to correct their path or to come back and tell us when we’re all wrong. And here’s a better way to do it, because we would love to hear that too, because we always like to have better ways.
(34:04):
If you had anything you’d like to add to our conversation, we’d love to hear your feedback, comments, and questions. You can find us at @PLATOTesting on LinkedIn and Facebook or on our website. You can find links to all of our social media and website in the episode description. If anyone out there wants to join in on one of our podcast chats or has a topic they’d like us to address, please reach out.
(34:23):
If you’re enjoying listening to our technology-focused podcast, we’d love it if you could rate and review PLATO Panel Talks on whatever platform you’re listening on. Thank you again for listening, and we’ll talk to you again next time.