PLATO Panel Talks: Quality in DevOps

Where does QA fit in DevOps? This month on PLATO Panel Talks, our host Mike Hrycyk talks with experienced DevOps experts Jonathan Duncan and Terri McAllister about what quality and testing mean in DevOps and dive into a discussion aimed at answering that question. The three also debate about how best to rename the concept of DevOps to make it inclusive of developers, testers, and everyone involved in a successful software development life cycle.

Mike Hrycyk (00:01):
Hello folks. Welcome to another episode of PQA Panel Talks. I’m your host Mike Hrycyk today. I’m really excited. We’re going to have a great panel and we’re going to have a good discussion about DevOps and what quality and testing mean in DevOps because that’s a question that a lot of testers out there have, and I’m sure it’s a question that non-testers have as well – where to test your fit in. So I have a great panel with me today. We’re going, we’re going to have a great and potentially lively discussion because we don’t agree about everything. So it might be an interesting one. So, without further ado, I would like to introduce my panel, who is going to introduce herself now because I think she’ll do better than I will. Go ahead. Sorry.

Terri McAllister (00:39):
Great. Thanks, Mike. Hi everyone. My name is Terri McAllister. I come from a consulting background where I’ve been doing DevOps and operations support for about the last 10 years. Specifically, I work a lot with AWS, so I’m a certified AWS solutions architect and SYS ops administrator.

Mike Hrycyk (00:57):
Awesome. Thanks, Terri. Welcome. And, Jonathan, tell us about yourself too, please.

Jonathan Duncan (01:03):
Much longer than a decade. So nearing the 25-year mark. So a quarter of a century, I guess we’ll call it now and really been involved in all facets, starting in testing, and then thinking that development was the better path. Only to come back to testing and really the premise of the bulk of my career is just around quality throughout all the different roles, whether they were business analysts or development, or, um, back in the testing world. So happy to be here.

Mike Hrycyk (01:31):
Let’s kick this off. Let’s set a baseline. What do we define DevOps as? There are lots of definitions out there and it’ll be better if we can start from a commonplace. Terri, why don’t you tell us what DevOps is to you?

Terri McAllister (01:47):
I think DevOps has a couple of different connotations around it and one is very specific to the role that somebody plays in a company or on a team and that’s the DevOps role. And then the differentiation that I like to make as well as is the DevOps methodology, which is, you know, really aligned with the agile methodology. And it’s really about the practices that a company or a team put in place to achieve the goals of a DevOps person or process just to make companies more efficient, more agile, and bring a level of automation and code reviews into every aspect of the life cycle,

Mike Hrycyk (02:26):
I guess. And let’s just speak and maybe we’ll talk about this more later, but that gives you a good separation because you can have someone on your team, who’s got a DevOps role without your organization really embracing a DevOps methodology.

Terri McAllister (02:38):
Yeah, absolutely. And that’s why I think it’s always a very good differentiation to make between the role and the actual methodology and practice.

Mike Hrycyk (02:47):
All right, Jon, what do you think DevOps is?

Jonathan Duncan (02:50):
So I’d agree with all of that. And the piece that Terri talked about making more efficient – I think that’s the piece where testers often struggle with, well, how do I fit in there? Right. Historically testings, that piece that gets jammed up. So if we’re trying to speed up deployments to production, how does that kind of work? And we all know we still need quality, but where do we fit? So interested to hear the ideas and thoughts we’re going to have around on where we think it fits, where Terri in the real world has seen it fit and sort of how we’re going to help promote that going forward.

Mike Hrycyk (03:25):
Yeah. I think I agree. I don’t think I have a problem with the definition. So we’re kind of in the same starting point, which is really good. So still continuing with the baseline a little bit, where does DevOps come from? What was the motivating factor for businesses to step forward into DevOps? Because we know that businesses don’t make changes unless they have to, let’s start with you this time, Jon, where does it start?

Jonathan Duncan (03:49):
I could say I started way back when I was too lazy to do deployments and I decided I just write some code to create it, but no, I didn’t create it, but I do think it started with, as the world has become more enabled and we don’t all have to be in offices and we need to get information out to customers or other people within our organization there was a need to say, okay, well you fix that, but I can’t wait six months to get it back out there. Right? So it was around, let’s move a little bit faster. Let’s get the change, let’s move it forward. Let’s build it and deploy it. I think that’s where it started not knowing exactly, but happy to hear that I’m wrong too.

Mike Hrycyk (04:29):
But I think from my perspective that you’re right, it was about speed, speed of delivery and getting things done, but it wasn’t, it wasn’t the big picture about speed. I don’t think for me, I think the first big step was a bunch of devs sitting around a room complaining about how operations, the guys who provisioned the machines and installed this stuff on them and made them available for the devs, always had their own schedule and always had their own set of priorities. And the devs were forever complaining about how they were going to have to deal with that. How, how are we going to get this there? And the ops guys similarly had complaints. And so someone just said, well, help us. And some manager was like, “Oh yeah, we could, we could do this.” And so they got together and decided, well, what things that the ops guys are doing, could the devs learned to do that’s for me, that’s where the definition came out of. Terri?

Terri McAllister (05:23):
I think that is spoken like a true developer. And I think, you know, obviously, I have a development background as well as an operations background. And I think you’re right. There are so many challenges between a traditional development team and traditional operations team. You know, operations folks would be equally frustrated because the developers would develop an attempt to deliver something that wouldn’t work on the infrastructure that they had available, or they wouldn’t take into consideration the limitations and even process right? Around how to deploy what built. And I even see that in a struggle now where teams may have adopted or think they have adopted a DevOps methodology and approach, but the developers are still very much kind of developing code with blinders on and no respect to the infrastructure, what it takes to create it, maintain it, deploy it, and what those underlying guts of the infrastructure to support their code is. And so I agree with you, right? It was built upon a lot of frustrations on both sides of the fence and a way to try to increase transparency on both sides, right? From an operational perspective, learning what the developers are doing so that they could be better prepared to support them in a deployment. But then also bringing some of the development practices into operations and things like code reviews and doing infrastructure as code and taking scripting of deployments to another level. And instead of it being, you know, manually copying the code onto the server to deploy it, having a script to do that, and then having a script that does it automatically.

Mike Hrycyk (07:04):
And just to clarify for any of our readers who aren’t familiar with the term infrastructure as code, that’s a concept where you are using some sort of automated script of deployment script, where you write a script to make sure the same thing happens each and every time and the configuration is part of it, it’s all data-driven and so it is something that is code that you can review and move forward with.

Terri McAllister (07:25):
Yeah. And I think even another level to that, Mike, um, is actually defining your infrastructure as code. So it’s not just scripting the deployment, but literally defining if you need a VM, you described that VM how much memory it needs, what CPU capacity it needs, all of the networking, the connections, everything that is in that infrastructure as code and not just scripting the deployment of the software code.

Mike Hrycyk (07:52):
Oh yeah. Good point. I think that, that another extension of this, or another way that this ties in and it comes back to Jonathan’s description is that agile really supported this and supported it in a couple of ways because agile, a big part of agile was smoothing for a more rapid transition to production of your code. One of the other legs that agile stands on is the concept that it’s a whole team approach. And that while you have expertise in the team, you don’t have silos and every single person on the team helps where they are needed when they can. So a dev who’s done their work maybe helps with testing or maybe helps with some requirements and the product manager or the product owner will help with testing and so on. And so it really makes sense that if you’re hitting a bottleneck at all, anywhere in your development process, if it’s at the deployment end, the people inside the team should be willing and trying to help with that. And agile really supports the concept of experts but does not support the concept of silos. So broadening out your knowledge base really makes sense from the agile concept. I think that feeds into DevOps.

Mike Hrycyk (08:57):
So we’ve already answered this a little bit, but if you are a person I have, maybe I just answered it. If you’re a person in DevOps, does that mean you have all of the skills? If you’re a DevOps person, does that mean you’re equally as capable as a developer and a tester and as operations, Jon?

Jonathan Duncan (09:16):
No. I think even in that, where you talked about agile and everybody needs to pitch in where they can, right. Even in those teams, there are still folks that are very specialized in specific items. So I think it’s unfair to say that, “Oh yeah, we have a DevOps team.” And while the team may know everything, any one single member of that team is a specialist in test automation, automation of infrastructure development, right? They do know enough, but they still need people to support them that are truly specialized in those areas. In my view.

Terri McAllister (09:52):
Yeah. I wholeheartedly agree with that.

Mike Hrycyk (09:56):
Alright. I’m going to ask the question about testing and DevOps. Now, since that’s kind of what we’re here for, I’m going to ask first, what have you seen the expectations of test and DevOps being? So not, not the rosie eyed future where everything’s working just the way we want it to be. What has been your experience today or in the recent past of what DevOps teams are expecting of their testers? Terri, let’s start with you.

Terri McAllister (10:20):
I think it varies. And perhaps I have a different view on things because I’m generally focused on a consulting perspective. So I’ve seen it where, you know, some companies are still very much doing functional testing, so they have a CICB pipeline that does the continuous integration and the continuous deployment to a non-production environment. But then testing is still your very traditional functional testing. But then I’ve also seen the other end of the spectrum where everything is automated. You push the code to an environment and automated test suite runs and that test suite passes. It gets pushed to a staging or a UAT environment and gets the final sign-off and pushed automatically to production. So I think it really depends on the maturity of a company, and how much focus they want to spend doing automated testing, because that’s obviously a huge gatekeeper to having a true DevOps methodology that includes quality.

Mike Hrycyk (11:21):
So in the first end of the spectrum where testers are doing manual testing would it be fair to say, we’re kind of saying that testing is outside of DevOps, that they’re just this silo off to the side that you talk to, or do you consider them actually part of DevOps, even in that equation?

Terri McAllister (11:39):
I think they’re included as part of a project team, but they’re not really included in the DevOps methodology,

Mike Hrycyk (11:48):
Cool. Jon?

Jonathan Duncan (11:48):
I think in that path they can be part of that team, right? If you haven’t gone and committed to test automation early, that you can’t get it all in there overnight. So when you’re dealing with it in that manual functional capacity, well, two things are gonna happen. One there’s now this manual step that we’ve got to go and do before we can deploy, right. If the tester was to do everything that they really needed to do, we’re probably not gaining any of the efficiencies that the goals were of the DevOps methodology. So, what happens likely in the real world is that that tester has to do a risk analysis of, okay, well, here’s what was changed if I don’t test that, what’s the risk of it, either showing up. And then if it does show up, what’s the priority of what it could have caused if it did pop out there. So I think maturity is one of the things that Terri mentioned that I a hundred percent agree with. I think the other is a commitment from the organization to say, this is how we’re going to approach this and we’re not just going to wait until we get closer to production. We’re going to start building out that test automation so that it can be part of the process the whole way through so that when you do hit that, now we’re going to go live and it’s in production you’ve got a suite of tests that the tests are that’s there because there is still going to be a need for some manual testing before you get that next piece of automation there, is going to be able to quickly say, okay, here’s the piece that didn’t get tested through automation. I’m going to go check that real quick before we hit the button.

Terri McAllister (13:19):
Yeah. You don’t get to be a Netflix or an Amazon of the world doing thousands of deployments per day with manual. You don’t get there overnight and you don’t get there with manual steps or gates that have to be actioned by a person.

Mike Hrycyk (13:36):
True. One anecdote I have, I guess, because I worked with one company where the idea of the CI/CD pipeline was associated with automation. So it’s automated deployments or automated builds, or somehow the word automate or automation got attached to it. And this attitude came up and it wasn’t really necessarily an attitude at the director level or the maybe one or two have this problem, but because the word automation was part of it and because the word automation at that organization meant testing because of the testers who wrote their automation, that suddenly the entire CI/CD pipeline, including provisioning servers, in the minds of a lot of the developers were like, well, that’s a testing responsibility, the testers have to do that. Which doesn’t really fit with the DevOps idea. And it doesn’t really fit with the agile idea, but it was an interesting silo, especially when you consider that so much of the experience and skills in building up a lot of parts of the CI/CD pipeline are traditionally developer skills. So I think part of it was, they just weren’t interested in that work and if they could find someone else to shove it off onto, then that would just make them happier. This is work I don’t have to do because it’s somebody else’s job, which is very much not dev or agile in my perspective. And I fought against that while I was there. And it wasn’t that hard to win back again. It’s like, well, everyone’s supposed to do everything. So why would you do that? But it was interesting to me. Then there’s the other perspective that is you have a whole lot of testers out there that are afraid of scripting and are afraid of getting knee-deep in the real technical stuff. And so they also try and avoid it, but unfortunately for those people, they’ve got to drag themselves into the 21st century. Alright, so we’ve touched on this a bit already because we talked about, well, what have we seen? What do we think the rules should be for tests in DevOps? Jon, let’s start with you.

Jonathan Duncan (15:35):
Yeah. I’m just going to follow on, I think the role of testing in DevOps, again, starts way back at the beginning of a, I’m going to say a project because you’ve got to start somewhere, right? If, you’re somebody listening to this and you’re new to this, right, you’re going to need to pick a project in order to start pushing forward to this sort of new way of the world. You needed to find right away at the beginning when this goes up into that operational perspective, where is testing going to fit in? Are we going to have just manual testers, be checking things and slow down our deployments, or are we going to commit early and upfront to we’re going to start automating things? And it doesn’t mean automating everything, but it does mean automating significant portions of it and starting to run that automation well before you even get into that recurring deployments to production. So yeah, starting early and then their role is fairly similar, right? As production issues come in, let’s figure out what it is, let’s triage it. And I think tester even on that perspective is the best to triage it. Triage it, figure out what’s going on, get it fixed, but let’s build the automation right then and there so that it’s ready for the next time so that’s another item that we don’t have to test.

Mike Hrycyk (16:51):
Cool. Terri?

Terri McAllister (16:53):
I don’t know if I have too much to add. I just think, you know, that continuous quality needs to be a focus, right? And, and to what Jon was saying, it’s not even just in a non-production environment, but also in a production environment. So I come from a background of managed services and, you know, the worst feeling in the world is to have an end-user or client report something before your team knows about it. And so I think having quality or quality resources keep in mind, what are the things that we need to monitor when this thing goes live, to be able to detect the degradation in services or something not working quite right. And it’s not even about the site being completely down. I mean, if you have a search on your site and it suddenly stops returning the right results, right? You want to know about that. And you want to know about that quickly so that you can get it addressed. And I think that’s often a piece that gets left out from a QA perspective.

Jonathan Duncan (17:52):
I just want to add just cause it just popped into my head, right. If we go back and think about, and it’s sort of an idea that I’ve been poking around with lately is test automation in production, right? So not invasive testing, but testing that’s out there confirming certain aspects of the application and giving feedback to that operations team so that they can actually sleep well at night and feel comfortable that they’re not going to get that phone call at three o’clock in the morning. So I think that’s another spot if thought about properly can sort of help give that whole team information and ultimately improve the quality of applications that people are deploying all around the world.

Terri McAllister (18:28):
And I think the other evolution that I see for quality is around cloud specifically. And coming back to that infrastructure as code and in a cloud-based infrastructure, which is obviously where I spend most of my time, but you can deploy a load-balanced auto-scaled environment that is going to be resilient and you know, it will add web servers as traffic increases or maybe the server will auto-heal and if something happens and that server goes away, it’ll automatically be recreated and be able to take traffic. But I find a lot of the time that is the stuff that does not get tested, right? If you write the infrastructure as code, it deploys, everyone assumes that it’s working until it doesn’t

Mike Hrycyk (19:13):
And I think that’s important when we’re thinking about, well, how testing can help, when we start talking shift, right. One of the ways that we can help in the monitoring sphere is not just to have monitoring, watching a log for errors and watching a bunch of metrics for levels. It’s the ability to run small smoke tests in specific areas that give very specific feedback that apply confidence to, yes, this is working, it’s working in a way that it should. It’s, it’s not a deep, comprehensive test because you do that outside of production because that will impact your production. But you do it before that. I do want to take this opportunity to throw in a plug. We have a prior podcast that’s up on our sites on the shifting left and shifting right where we dig a lot more in-depth into those things.

Mike Hrycyk (20:01):
I’d like to hold back a little maybe and change our perspective just slightly. The thing that I think is important, so everything that you’ve talked about is true. I really do believe that, but I think that the other big role that testing needs to have in DevOps and in agile is that of owning testing and owning the idea of quality, which means that as you’re transitioning from a place where testing was a manual function, and it became its own little bottleneck. And as you’re trying to move into your CI/CD pipeline world and have everything be streamlined and scripted and moving forward is that testing needs to hold a conceptual ownership level in there, which says, okay, we’re going to do this. And we’re going to run this test here and this test here, and then we’re going to have this automated gate, and then it’s going to go. And in that specific area, what they should be doing is they should be the people who are bringing up in conversation and thinking about themselves and advocating for the notion that we’re going to do this, where are we changing our risk profile? So there was a certain risk that came with certain testing and missing certain things when you’re doing it manually. We’re going to shift that over and we’re going to do it in an automated fashion. There’s still going to be risks, but it’s not going to be the same risk. And so understanding what that risk profile looks like, understanding what you can do to mitigate that and having those discussions. I think making sure those discussions and those thoughts are happening, I think that really becomes a necessity. And I think that’s one of the core things that your test is going to do inside a DevOps methodology environment. Is there going to be that test advocate? And a fair warning I might talk more about this because next week I’m building a presentation to do some training on quality advocacy. So that’s kind of my focus right now.

Mike Hrycyk (21:47):
So are we on the same page with what testing and DevOps are? I don’t think we’re really in disagreement.

Terri McAllister (21:54):
No, I don’t think so.

Mike Hrycyk (21:56):
Awesome. Alright. Now we’re going to get into the part where we had interesting pre discussions. So one of the things about the term DevOps is it’s not very inclusive. It’s got the word Dev and it’s got the word Ops or operations and what it doesn’t have is the word test. And I’m thankful that CI and CD, continuous integration and continuous delivery, don’t really exclude testing. But one of the things that kicked off our conversation, which helped us decide that we wanted to have this conversation. How do we make it a little more inclusive? What kind of terms should we embrace or is dev ops just good enough? So, why don’t we kick this off with you, Jon?

Jonathan Duncan (22:41):
So, no, I think you’re exactly right. I’ve sat on the development side and while I never wrote a piece of code that I didn’t believe worked, I also know that I didn’t know enough about testing in order to test it appropriately, and I had something else on my plate to move to next. So I think you’re a hundred percent right. That, that word while I think everybody in there cares about quality it definitely doesn’t put quality in the forefront of everybody’s mind. And again, CI/CD the capabilities there to include testing and include quality, but it, in the purest form, it’s not necessarily in the forefront. So a little bit back, Terri mentioned continuous quality, right? So that’s the one we can start getting that flowing around the world and having people think of that as maybe stuck in the middle of CI/CD I think that would be a good thing. If only for a reminder, those doing it right have already got it in there, but I think it sometimes goes forgotten along the way.

Mike Hrycyk (23:42):
Terri?

Terri McAllister (23:42):
I agree with Jon, you know, there is a missing component and it’s very easy to say that you do CI/CD, but you’ve completely forgotten about quality and testing in that. And whether or not it belongs in the middle or at the end, as we’ve talked about, you know, having that production level of test automation as well. But I do think continuous quality needs to be part of it. I think, you know, the other thing with DevOps, when you talk about methodology, a lot of people think automation, right? Automating your environment, automating your deployments and automated testing has been around for a long time as well. So I think DevOps has a catchy phrase, whereas maybe automated testing doesn’t and I think a lot of people are doing it and it’s really about a kind of integrating it into that pipeline, which is usually where the CI/CD comes in.

Mike Hrycyk (24:36):
I had referenced at the start that there might be some contentious discussions in here. And unfortunately, I might’ve just talked myself out of a contentious discussion, but I’ll frame it so that we can at least talk about it. The thing that I was going to talk about, and we were going to help decide is whether we liked CQ or there is another term out there that we found when we looked around and we kind of heard a little bit which was called QualiOps, or as I like to say QualiOps, I think that sounds better, but maybe it’s just because it sounds more like koala anyways. So I liked the idea. I liked the sound of QualiOps. I like the idea that it really brings quality into it, but in support of the idea of diversity and inclusion, it goes too far the other way. It takes, Oh, we have a phrase DevOps that, that doesn’t include quality. So let’s throw away the word dev and put quality back into it. And whereas from the testing perspective, I would like if we did everything from the quality first perspective, I mean, that’s a very test centric idea. And I think it’s a great idea, but, I think that there’s power in words. Now I sound a little bit like Michael Bolton. There’s power in the terms that you use. And if we decided that we’re really going to advocate for and push for the term QualiOps, what you would get is you get a bunch of testers who sort of rallied around it and supported it, but it’s not actually moving the whole team forward. It’s not actually pushing the concept forward. So, I think I’m coming around to the thing that you guys are pushing for, which we had decided was, was CQ let’s figure out how to make CQ a term that starts being used. And then at least the Q, there’s not a lot of other Q out there. So, people who don’t even know it are going to probably guess that it’s quality. I guess I argued myself out of this argument. You guys have any comments to make about that?

Terri McAllister (26:25):
That was really easy. That is my first comment. I didn’t even have to do anything, but I think, you know, the other thing around continuous quality that makes me lean towards that term, as opposed to QualiOps is you can automate anything you want, right? You can automate your infrastructure. You can automate your test suite. You can have automation everywhere, but if you never use it, it’s useless. This is why I think the continuous quality, is… like, even if you provision an infrastructure automate it and then you go and make manual changes, you’re defeating the purpose, right. And that’s where the CI/CD pipeline with the gates and controls puts that automation in place and it gets used. And that’s where I think quality and the automated test suite really need to be used on a regular basis for it to be useful. So just another plug for the CQ train.

Jonathan Duncan (27:20):
Yeah. And we talked about it all kinds internally. I’m sure there are things out on our blog posts that talk about developers can fix things easier if they can get the information quicker. Right? So that just flows again to let’s go build the automation early and let’s run it and run it all the time so they can have the information right away. Because if you tell them the next morning that, Oh yeah, something went wrong with this functionality. They’ll remember what the train of thought was. Whereas if you tell them three or four months later, it’s hard to remember what was going on at the time that made you write a piece of code the way you did and then potentially breaking something else that you didn’t even think of.

Mike Hrycyk (27:58):
Oh yeah. Fail fast, fail often. We talk about that somewhere. I think another thing that struck me as interesting as I was talking myself out of being my own advocate was that we talk about CI and we talk about CD and we talk about CI/CD and the interesting thing for me, it’s not an or. There’s absolutely no capability to have an or between having CI or CD. It’s a spectrum, it’s a road. You can’t get to CD without starting with CI. You need to start building a continuous integration pipeline that starts with building some sort of automated build process and then moving forward. So we’ve combined them CI/CD because that encompasses everyone on that journey. Right? It encompasses all the people and kind of what came out of it a little bit for me is – Where do we fit CQ in that? Is it CQ slash CICD? Is it CQ at the end of it? And it really isn’t that continuous quality needs to be throughout that spectrum. And you need to test stuff before you build it with your unit testing and you need to test it as soon as you build it, to make sure that you’re delivering something to test that is acceptable. You need to test it as you deploy it, and then you need testing around your deployment. And then that’s not just the code. It’s also the infrastructure as code. So kind of in my mind, and I’m not a graphic designer, and I will never produce what I’m thinking about is, is a logo that somehow has CI/CD in the middle of this, the way we’ve always seen it. And then somehow surrounding it CQ because some people may not want to hear this, but to me, CI/CD is just a small part of CQ. It’s a part of having continuous quality, having a quality mindset and producing a good product each and every time. Either of you want to volunteer to build that logo?

Terri McAllister (29:33):
Absolutely not!

Jonathan Duncan (29:33):
No! Haha.

Mike Hrycyk (29:37):
So, okay. So we got to the argument without really having an argument. So I apologize to my audience if that’s really what you were signing in for. Real quick, what kind of toolkit is essential and what should the tester’s involvement with the toolkit be Jon?

Jonathan Duncan (29:54):
Well. And you’ve heard me talk before, so this won’t be any big surprise to you. I actually think that tools are often less important than the commitment to integrating quality in every step throughout, right? Because I can’t say use this tool, and this is the only tool, and I don’t think there’s any tool vendor out there that would say, no, we’re going to, we want to be the only tool you ever use. So I think it’s making sure that you find tools that your team is familiar with, that they feel comfortable with and that they can easily use, right? I think all kinds of projects have started with the best intentions and things start to fall apart when you put processes in place, or tools in place that the team has a hard time using. And they fall back to what they know how to do because they’ve still got a deadline to meet. So, I think looking at it more holistically is the right path, but at minimum, right, you need a tool to automate, right? So that could be a commercial tool like Tosca. It could be open source like Selenium, there are some cases where it might be, Nope, let me go build this custom Python script because that’s what this particular set of tests needs. And then you need a place that you’re going to be able to run those tests, right? So you need some sort of tool at the backend that you can say, okay, I’m going to run these tests and here’s how I’m going to schedule them right? Maybe you don’t run every single test on every check-in. Maybe there’s a suite that runs overnight and then maybe there’s another sweep that’s larger that runs on the weekends, but testing needs to be in control of the here’s what tests are gonna run snd here’s the reasoning and logic behind why I selected those tests to run at these particular intervals, whether time-based or event triggered.

Mike Hrycyk (31:37):
All right, cool. Terri?

Terri McAllister (31:40):
Yeah, I actually really agree with Jon. It doesn’t really matter what tools you’re going to use. It’s about adopting a methodology and a process, and really being honest with yourselves and your team as to what you can accomplish, what they’re open and comfortable with. Try not to boil the ocean all at once. It can very much be baby steps towards the end goal. It doesn’t matter if you’re going to do it in AWS, or you’re going to use Jenkins or Azure DevOps, or even on-premise, right. It’s really about the process, the methodology that’s going to work for your team to achieve your goals and, laying out a plan on how to get there.

Mike Hrycyk (32:19):
I agree. I think I agree for the most part. I think though one additional concept though, is that I think that there needs to be common fluency in some of the tools that you use. So if your chosen CI tool is Jenkins, you should, you should have that fluency within every member of your team so that testers can get their automation into Jenkins. They can get results out and understand them. Developers can get their build stuff in and they can understand the results. And then, in the same way, I think that a team is more successful if the same is true around your automation set and the teams that I’ve seen most successful around automation are the teams where the developers and the testers all help with maintaining the script. So developers write code that breaks an automation script if it shouldn’t have, then that means they know right away that there’s code that needs fixing. And if the scripts themselves need changing, if it’s a simple thing, the developers should be doing it. If it’s a bigger thing, they should be having conversations with the automators. So it becomes a fluency thing. That everyone on the team has a certain amount of fluency with all of the tools that you’re using. And that extends for testers back to the coding and looking at the code is you don’t have to be the person who is maintaining and editing the code, but the more you understand the more helpful. So, it’s a little bit of the same thing as it’s not so much that you’re tool-agnostic, which is a term that PQA loves to use. It’s that you’re ready, willing, and able to embrace the tools and use the tools. Don’t say, I’m not going to touch that because that’s not a tester’s job. And developers, don’t say, I’m not going to touch that because that is a test job.

Mike Hrycyk (33:47):
Alright. So we’re just about out of time. I would just like one or two sentences of advice from each of you for an organization or a tester just starting out our journey into DevOps or CQ. What ideas should you embrace to make it a better time for yourself? Start with you, Terri.

Terri McAllister (34:05):
I think I’ll come back to a piece of advice that I actually just mentioned, which is don’t try to boil the ocean. Especially if you’re just starting out and maybe you are completely focused on functional testing right now. You know, it is a journey, towards true DevOps or true CI/CD/CQ figure out the low hanging fruit of things that you can learn, training that you can take or ideas that you can escalate within your team and focus on those baby steps that will get you to where you want to be or where you want to take the team.

Jonathan Duncan (34:42):
Hmm. Well said, I think my biggest piece of advice Mike would be to start early, right? Think about it early. And like Terri says, take on little pieces, but make it a habit. Have it be a habit for the team that that’s just what they know because things always go sideways at some point in a project. And once it becomes a habit it will stick. And that’s how you’ll do everything going forward. And starting early is going to have quality ingrained in everybody’s mind, um, the whole way through to production and for the life of the product.

Mike Hrycyk (35:15):
Awesome. Thanks, guys. I think the additional piece I would give is to make sure you remember that you’re the quality advocate and you’re the person who’s bringing the tester’s perspective through this. The CI/CD journey is going to happen at your organization. Eventually, it is going to happen. And if only the developers drive it, then it’s going to become a little lopsided. The best way to get it in so that it’s still producing a quality product is that you’re there embracing it and helping it be a part of the process.

Mike Hrycyk (35:47):
Alright. So our time is up. I would like to thank my guest, Jonathan Duncan and Terri McAllister for joining us today. It was a great conversation. Maybe we’ll search out some other topics that can be more contentious so we can have a true argument next time. Out there in listener land, if you have such a topic, please send it in and we’ll consider it. I would like to thank our listeners. If you want to continue this conversation or ask questions, please touch base with us through one of PQA’s social channels. We have Twitter and Facebook and LinkedIn. And if you could, it would go to, if you could boost this podcast to your friends in your networks and leave comments, and reviews on one of the podcast channels that are out there. So have a great day!