Use technology to make society better for all.

Alistair Croll: Hi, and welcome to another FWD50 conversation with one of our partners helping us put on the 2021 conference. Coral Kennett is the education lead for Amazon Web Services here in Canada. She’s got a storied background in innovation and she’s got some amazing stories to share about how Amazon itself innovates. And we’re going to spend some time looking at how those can be applied within government. So please give a very warm FWD50 welcome to Coral Kennett. Hi Coral.

Coral Kennett: Hi Allister. Nice to be here. Thanks for having me. 

Alistair Croll: Nice to have you here. Where are you at today? 

Coral Kennett: I am in Tsawwassen, which is just south of Vancouver here in British Columbia.

Alistair Croll: Excellent, I’m all the way over in Montreal. So nice cross continental conversation. What are you doing at Amazon? 

Coral Kennett: I am the education leader for Amazon Web Services in Canada. And I also manage our cloud innovation center at the university of British Columbia. 

Alistair Croll: What does the innovation center do?

Coral Kennett: So the innovation center, it’s a, it’s a program right at AWS, it’s collaboration between an educational institution in this case, UBC and AWS, we have 12 innovation centers around the world. So, this is our Canadian cloud innovation center. And really the idea of the program is collaboration. So how can we bring together all of the amazing things from the institution, faculty, staff, students, which are a huge part of our program – they are builders. AWS, our expertise, our innovation methodology, our staff, and then our challenge sponsors. So, we invite organizations across Canada to bring us their problems or their opportunities and we work together to build innovative solutions for the benefit of everyone. 

Alistair Croll: Well, one of the things that I found fascinating about the whole cloud computing shift and some of us are old enough to remember having to go into data centers full of server rooms. You know, if you want it to do something, you had to order a machine and then procure it and stand it up and get it configured and so on. It really feels like the advent of cloud computing has significantly shortened the whole cycle time of innovation, not just of the technology stack. 

Coral Kennett: It absolutely has. And I mean, in the past, as you said, it, there was a pretty high cost to innovation. If you’ve wanted to provision some new servers, you’d have to write a business case. You almost need to know the answer before you even start, because you’d have to kind of, you know, build the team and really put a justification together for why it is that you want the resources that you want. Which is a little bit backwards when you’re thinking about innovation, but with cloud computing it really lowers the cost of innovation. It lowers the cost of failure and you can really experiment a lot faster. I can actually give you an example from the cloud innovation center, where we were working with the provincial health services, authority and Vancouver coastal health. And they had an idea that by using some natural language processing and machine learning services we could innovate on the MRI waitlist times and actually lower the times to get your MRI appointment, but in order to get the resources to actually experiment with this they, they didn’t exactly know what they were even asking for to start with. So, we were able to bring them into the cloud innovation center to use our Amazon’s innovation process called working backwards, where we really dive into. What is the problem that we’re trying to solve and who was the customer we’re trying to solve it for? We were very quickly able to put together a proposal of what we thought we could do. And then in the course of about two to three months, our students, our co-op students who in this case happened to be third- and fourth-year computing, science and electrical engineering students. They actually built a prototype that demonstrated this solution and what we did is we took the MRI intake forms. We put them through our natural language processing service called Amazon comprehend medical. We then use that data to create a rules-based algorithm that would help the intake clerks assign the priority to the forms. And so, all of that work was done in a matter of months. And then the team at Vancouver, coastal health had everything that they needed to take that forward and really turn it into a product that will be launching across the province soon. So really lowering that cost to try and experiment and lowering the cost of failure. 

Alistair Croll: Can you tell me a little more about the working backwards methodology you mentioned? 

Coral Kennett: Yes, absolutely. So, you know, a lot of times what happens is development teams get excited about a new idea or a new technology, and they want to try to find ways to use it. And, you know, at Amazon we have a lot of technology. So, when we’re thinking about a new innovation, we really want to make sure that we are solving for the right problem and for the right customer. And so, what we do is something called working backwards. And the first thing that we do when we are thinking about a new solution is we write the press release. And so that might sound a little backwards to some people, because if you think about traditional development, you know, you, you develop your product. You, build a marketing campaign and then when you launch it, you do the press release. But what we find at Amazon is by keeping us really focused on that end goal of what it is that we’re trying to accomplish. It then helps the development process move a bit faster. So, we do this every day at the cloud innovation center. And when organizations come to us, we begin by asking them five questions that are key to working backwards. The first question is who is the customer. And that’s actually a more difficult question than you might think in a lot of cases. I work a lot and the cloud innovation center works a lot in healthcare. And when we ask who is the customer. Usually, the answer is patients. And that’s because I think the people working in healthcare are still focused on patient outcomes, which is a great thing. But in reality, usually the patient is the beneficiary of what we’re trying to do. They’re not actually the customer or the person who’s going to use this new solution. So, we really have to dive a little bit deeper to figure out who is the actual customer. And then we have to figure out what is the actual problem that we’re solving for. And a lot of times when different projects come to us. The problem itself inherently is stating a solution. Like we need a machine learning algorithm to do this, but it’s really, what is the problem you’re trying to solve with that? We then talk about what is the most important benefit, and this is really key to Amazon’s development process as well. It’s not about trying to get every benefit all upfront. It’s what’s the first thing we want to do. What’s the main thing we need to accomplish to be successful? At Amazon we call it the MLP minimum lovable product. What is the thing we need to do to have our minimum lovable product and be successful? And then we’ll iterate from there. The fourth question is how do you know what your customers need or want? And this is not necessarily just about asking your customers what they want. It’s about truly understanding them and innovating on their behalf and using data to support that. And then what is the customer experience look like? Really visualizing what we’re going to do before we bring any technology. And we find that that works really well for us because it gives us that the long-term vision of where we’re trying to go. And then we can be very flexible in how we get there. And so, we do this every day. Another example of where we did this was with the e-comm 9 1 1 in here in British Columbia, where they provide 9 1 1 services. They were having. The problem that they had when they originally came to us. It’s just, they wanted to help with call times they wanted to help the call takers. And so, we, we really dug into that to figure out how can we help with this issue to benefit people who are calling into 9 1 1. And so, we focused on the customer of, of new call-takers and their specific problem was around. The fact that there are so many standard operating procedures that can be deployed; thousands that can be deployed. How can we help them to sort through so that they can spend their time focusing on the caller, the ultimate beneficiary? So, what we did is we built them a virtual assistant that helps as, as the caller is talking, it’s pulling out keywords using some of our natural language process processing services and then recommending different standard operating procedures that we think be deployed for that specific case. And so, using this process, we were really able to get to the key of what the issue was. And they’re currently experimenting with that to see if it’s something we can deploy in the province.

Alistair Croll: So, when you build something like that and you build a sort of prototype or proof of concept, does it generally then get built as an Amazon product or does the government build it or like. How does that find its way into a 9 1 1 call center in production?

Coral Kennett: Yeah. So, we’re, we’re doing this on behalf of our customers. So, it’s something that they can build in with, with our help. We have partners that can help them, but we also a key tenant to the cloud innovation center is building open-source solutions. So, anything that we build, we publish the code. We publish deployment guides. We publish everything that you would need if you wanted to replicate this solution. And we have lots of different ways that we can help with that. In the case of MRI, it’s something that they do want to deploy across the province. In the case of 9 1 it was more of an art of the possible what might be possible for us to do. And that might be something that ends up being used in the future, but it was really so that their teams could experience this form of innovation process and really experience what they might be able to do with it.

Alistair Croll: I love the precedent that sets a sort of augmented worker I know post COVID, when COVID. So many government employees were in Canada were repurposed to become call center operators for CERB to answer questions, because there was such a huge rush to claim CERB and other COVID support benefits that many people who’d never worked call center stuff now found themselves at home, basically being call center employees. And it seems like the ability to spin up an agent to help that person ride the learning curve more quickly would be incredibly valued. 

Coral Kennett: Yeah. And that’s why, when we were looking at this, we were really trying to figure out like, what was the problem when we started, it was a really big problem of, you know, just really improving the overall experience. But by getting specific, you know, with this process of working backwards we’re not looking to solve everything that’s happening in the process. We’re looking to solve one specific thing, solve it fast, iterate on that solution and then move on to the next part of it. So, you know, I have a visualization that sometimes we use when we’re starting, these workshops have a really big Boulder, and I say, this is like the scope of the problem. We don’t want to minimize the problem. Cause especially when we’re dealing in government and healthcare and education, these are really big issues. And, and it’s not about minimizing the issue. It’s about breaking it apart so that the counter image to the Boulder is just a big pile of pebbles. And I say, what we’re going to do is try and break this up and figure out which pebble we want to address first and what we’re solving for first, and then we’re going to keep going from there. And that’s very similar to how Amazon innovates and, and I don’t know if anyone’s ever seen the first Kindle that was ever developed. And if you haven’t seen it, just Google it it’s, it’s hilarious. It’s huge. It’s bulky. It’s not like maybe what you would picture as a Kindle. I’ve only ever seen it under glass in our Day 1 building in Seattle, but that was our MLP and we were solving for one specific problem. And that was how can we get thousands of books into anyone’s hand in seconds. And we achieved that MLP. That was the benefit we were going for. And then we iterated. And of course, now it’s lighter and better battery and better screen and all of these kinds of things. 

Alistair Croll: It does seem that there’s a certain audacity of making. Like if there isn’t something audacious about your claims and you know, here’s mine, you made it a lot smaller. It doesn’t seem. If there isn’t something audacious about your claim, then you’re not going to be inspired to work on it. Like how do we get books to people in a day or so that just doesn’t get people excited. Yeah. What do you think is, but at the same time if you say something daunting? That sounds impossible. People are going to give up before they even start. So, what have you found is the right level of sort of ambition to get the troops excited and thrilled, but still make it, make them feel like they have some fluency in achieving that thing?

Coral Kennett: Yeah. You know, that is a great question. And we’re still experimenting with that every day at the cloud innovation center, there’s always this tension between, you know, what can we do fast? What can we do? What can we really push ourselves on? When we’re doing our product development, at the cloud innovation center, we take these problems and we have things that are, like I say, the minimum amount that we can get done that we know we have to get done in order to achieve that key benefit. And those are sort of table stakes of things that we need to get done. And then we also can add things on that we consider perhaps stretch goals that we’re trying to, to really reach for something that is, you know, truly above and beyond. And I think that by sort of separating it out this way, we we’re ensuring that we’re really focused on that long-term vision of what our key benefit is. And then we’re still iterating. And sometimes those things get done in scope in this really short development cycle. And sometimes they move to the next cycle and, and because we’re a collaboration program we, we just bring in more and more resources. So, we’ve brought in UBC faculty. If we have a really sort of gnarly problem that we need to solve. We’ve brought in expertise from AWS. So, it’s really about experimenting, but because the cloud, like I said earlier, has lowered the cost of experimentation. Our students who are just absolutely incredible. They’re their undergraduate and graduate co-op students from UBC. They have access to really almost limitless technology right at their fingertips. And they have a sandbox to sort of play in and try out all of these different things. And if it doesn’t work, we just turn it off. And there wasn’t really very much cost to that. 

Alistair Croll: Yeah, that idea of like the cost of deleting something is zero. You know, in the past, there was always this case of, well, you bought the server, you better use it, the sort of path, what economists called path dependency. You already made the investment. You got to use the thing, you got to get that ROI. You know, I remember Randy Bias, who is a fairly well-known person in the cloud world from years and years ago. Had this line about, we used to have servers and we gave them names and treated them like pets and when they got sick, we made them better. And now we have instances and when they fail, we take them out and shoot them. It’s a little inhumane, but the idea is I just delete it, start again. And there’s a certain amount of, like, it removes the sort of guilt of bad innovation and frees you to try again in a way that I don’t think was possible in the analog world. But it’s very digital. But I don’t, I still, it seems like government has many other cultural issues that make it hard to say, okay, forget that. It was a bad idea. Let’s move on. Let’s try again. It’s almost like the public sector needs a different set of expectations about experimentation to get to a minimum lovable product. 

Coral Kennett: Yeah. And I think there’s a lot of like, very understandable reasons for that. I mean, these are public funds that we’re working with, so there’s a certain amount of responsibility there. I think, you know, the word failure in, in public sector kind of has a very high connotation, but it’s, it’s about creating safe experimentation. And, and we’ve really been able to do that at the cloud innovation center, but I’ve seen a lot of other organizations build this same kind of mechanism into their organizations. We have a concept at Amazon called a two-pizza team. Which of course we love to debate how big the pizzas are, but the idea of it is to create small autonomous teams who have everything they need. All the resources they need to be able to move really quickly and experiment fast, but in a safe way, that’s not causing a lot of inter interdependencies amongst the organization. And Amazon itself has organized in this way where everyone is free to innovate without causing a ripple effect across the organization. 

Alistair Croll: It seems like that that idea isn’t just like how the tasks work, but also breaking the product up into components. I remember in the early, early days of Amazon, the first service was S3, which was just scalable storage on the internet. You give it an object. It gives you a URL. You have a URL; it gives you an object. It’s really simple, but like that was the first thing you needed. You needed a way to store something and then you could build a compute instance that booted from that store thing. And last time I looked, there were hundreds of services on AWS, like CloudFront and quick view and there’s just so many different services right now. Kinesis and Redshift. It’s just it’s. You know, but back then it was like, you got S3 and you got EC2 have fun. It seems like because Amazon web services product offerings are composed that it’s easy to organize your teams in a composed way because it fits like that, that mentality. Can’t just be how you organize your org chart. It also has to align with how you organize the services you build. 

Coral Kennett: It absolutely is true. So, the architecture of AWS itself does lend itself to this kind of experimentation. You know, we moved from being sort of monolithic architecture to breaking it up into these, these two pizza teams. But I take your point. I mean, we do have over 200 services now. It can sometimes get difficult to even keep track of like what all of our services are. You know, but another thing that we’ve really been able to demonstrate at the cloud innovation center is that like, you don’t need to know all 200 services. Like there, there aren’t very many people who do. And like I said, we have students working for us. So, we have students who come, who have never worked on the cloud before they go through a little bit of training on the specific services that they need to be using to innovate. We give them a safe place to innovate and then they go build an MRI scheduler. That’s going to be deployed across the province in two months. So, I think that’s a perfect representation. We had another, example where we worked with Trillium Health Partners in Ontario. Their Institute for Better Health, which has their data science team. They came to us and said, we’re interested in your data and analytics tools. That’s what they’re interested in. You know, they don’t necessarily need to know about networking and, and they’re not interested in a lot of the other things that we have. They want our data and analytics tools and they wanted a safe place to try to figure out how they could use their data for real-time prediction of resourcing with rising and falling levels of COVID. So, they were really trying to get at more actionable data for their internal customer who is in operations, trying to schedule resources. And so, you know, we worked with them and helped build a data platform and they got very, very proficient on these data and analytics services in a very short period of time and in a safe environment to do so. So, we do have so many examples of where teams in public sector, in government, in healthcare have been able to do this kind of very fast iteration and innovation in a, in a safe way. 

Alistair Croll: So, you’ve said a lot of really interesting words that have kind of become part of the vernacular of innovation these days. And people might not realize that many of them came from Amazon. You mentioned Day 1. And, and the idea of like, now there’s a place you can go to see first, Day 1 inventions. The two pizza teams. I know there’s this idea of you should be able to summarize what you’re doing in one page of text, not a bunch of PowerPoint slides. I talked to Jesse Robbins, who’s actually spoken at FWD50 in the past about resiliency and he set up the whole game day idea of breaking something, which Netflix turned into chaos monkeys. So, it seems like a lot of these ideas of game day and day and two pizza teams and one-page briefs. Are all things that came out of Amazon’s culture that you’re kind of sharing with the world. Can you talk a little bit about the one-page brief and, and how that initial idea gets framed and why that’s more effective than interminable PowerPoint slides? 

Coral Kennett: Yes. Well, PowerPoint is not something we do internally at Amazon. And actually, the press release that I was talking about earlier can never be longer than a page. That’s it? That’s the limit. And, and we also have the concept of a six pager. So, you know, AWS itself was developed from a six pager and there’s really nothing ever that’s more than six pages, but to your point, it’s about saying it succinctly and, and really using a lot of data to back up what it is that you’re trying to do and having clarity of vision. And what we’ll do at Amazon is if it’s, if it’s a one-page paper we will all get together in a room. Well, before COVID. Now in a virtual room we will schedule a one-hour meeting. Whoever has written this paper will distribute it at that time. We will spend about 15 minutes reading and commenting and editing, and then we’ll discuss and we’ll give feedback. We will make some decisions right there.

Alistair Croll: So, you actually read in the meeting, you don’t ask people to read beforehand. You sit down and read together? 

Coral Kennett: We sit down and read together and it can be very interesting because when you are actually in person, it’s a paper document. So, it’s not that you’re on your computer it’s paper document and you can hear pencils and pens just scraping away on the, on the paper. And sometimes these papers can go through multiple revisions. There’s no sort of pride of ownership. They’re intended to be co-written documents, but it really forces you when you have one page or for something big, like AWS six pages is the absolute maximum. It really forces you to have clarity of vision in terms of what you’re doing. Who are you doing it for? Why are you doing? And what’s the data to support that. And every word is precious. So, you have to be very clear in, in your speaking and you can’t hide things in a PowerPoint presentation. 

Alistair Croll: I can see that working in two really interesting ways. One is I’m forced to do my research so that if I go 74% of people are going to do this, it’s, I’ve got, you know, tons of documentation behind that, if someone questions it, it’s almost like a hyperlink to this other stuff. You don’t need to know. You just need to know that the 74% was there, but at the same time, if I have to summarize things in a document, this one page. That implies, I have a lot of trust for the people who are going to take that, that sentence and turn it into a technical spec. Right. It implies, it almost embodies delegation. And I think this is something a lot of people forget is that the goal of leadership is to create a shared vision. And then the goal of management is to make sure that people can execute on that shared vision. Right. It’s not to do it yourself. And I think that that’s something that, that we often forget when we make these very long. Make sure you got to really. It’s almost like if I said it in the slide. Then I’ve covered my butt and no one can accuse me of having overlooked it, as opposed to, I’m not going to say to this slide deck or this one-page paper, I’m trusting other people to do the work, to think their part through 

Coral Kennett: Yeah. And somebody asked Jeff Bezos at one point, like how does Amazon continue to innovate? And his answer was that we are stubborn on vision, but flexible on details. And I think that sort of embodies that. It’s like having that clarity of vision in that one pager of where we’re trying to go, but then the trust of your teams, and especially when you’re dealing with technology, especially when you’re dealing with AWS, there can be many different ways to achieve something technically, but if you have that long-term clarity of who your customer is, what the problem is and what your main benefit is that you’re solving for, then allowing the teams to sort of experiment and fail fast. Like we said, to get there it, it works for us, 

Alistair Croll: So, we’re almost out of time, but I really want to get your thoughts on the weird sort of Zen, like duality of innovation. On the one hand. If you’re a startup, or if you’re, you know, a pilot project, you need to believe that something is possible. Despite all evidence to the contrary, you’re trying to do something new. You’re trying to prove to somebody else that this thing can exist. And by definition, you’re trying to convince people to something that’s not obvious because if it was obvious everybody would have already done it. At the same time. You have to have this incredible humility to be flexible and bend. As you now learn things because you will always learn things you didn’t know at the outset. And people can be very defensive about that. If there’s, if there’s a public sector kind of oversight, then people say, oh, wait, you’re changing. You’re moving the goalposts. You’ve changed things. So how do you set an expectation up front that when we’re in the field of innovation, we are going to, as you said, be very, very stubborn on vision, but very flexible on implementation. How do you get that kind of Zen like yin and yang balance of stubbornness and flexibility and, and let other people who are more accustomed to traditional sort of formal waterfall approaches, understand that that’s the norm for what you’re doing at the time? 

Coral Kennett: Yeah, it’s, it’s, it’s tricky. And we do have a few different ways that we do this. We have, we have a lot of different mechanisms that we use at Amazon to ensure that we sort of have this common behavior and common language. And one of the things we have our leadership principles, which are very integral to everything that we do. And two leadership principles that I particularly resonate with are bias for action and dive deep, but those are inherently contradictory, you know, how are you having bias for action if you’re diving deep? And so, one mental model that we use to make sure that we’ve got the right balance here is the concept of a one-way or a two-way door. And so, if we’re working within our two pizza teams and we want to try to, to experiment or innovate, we can ask ourselves, you know, should we have bias for action. Should we just go and try this? Or should we, should we dive deep and really, really figure out what it is that we need to do? And the one-way and two-way doors. How we do this, if it’s a one-way door and we can’t come back from it, we’re going to make a decision and it’s. Going to have a very substantial impact on our business, on our customers. Then you know, we’re going to have to dive really deep, maybe write a one paper, get all the data and, and really make sure that we’re doing the right thing. Uh, alternatively, is this a two-way door? Can I try this? Can I experiment? Can I fail fast? Can I take some risks? Can I do some things? And if it doesn’t work out, then I just turn it off and come back and, you know, think about it again and go and try again. And so that really helps us to keep that balance. And you know, really that sort of long-term vision on what we’re doing, but like the flexibility to get there and to know when we need to spend more time. And when we just need to go and try things. 

Alistair Croll: And I guess when you miss identified the type of door, that’s when management has to step in and go, whoa, we made a mistake because you’re assuming that the team has the ability to tell the one-way door from the two-way door.

Coral Kennett: Well, I’ll give you an example really quickly that, um, you know, it’s not like we never failed. Like we definitely fail. And in the, but in the 2015 shareholder letter Jeff Bezos said that, you know, failure and invention are inseparable twins. To invent you have to experiment if you know, in advance, it’s not going to work. It’s not an experiment. So, we’re failing a lot, but it’s not about just failing for the sake of failing. It’s also, then that introspection on what failed and I mean, one great example of that is the fire phone, the fire phone, was $170 million write off for us. So that was a one-way door that we walked through that ended up being a failure. But we then did an analysis of what had happened there. It was one of our first times that we’d worked in hardware. And we really looked at what we had done, what worked, what didn’t work and took all of that analysis. And that team went on to go work on the echo device, which ended up being, you know, a great success for Amazon, but it was a lot of learnings we took out of the fire phone, went on to the success of the echo. So yes, it’s not always something that works, but we have ways of taking that failure and, and moving forward. 

Alistair Croll: Yeah. One of the most popular lines in analytics is if it won’t change your behavior, it’s a bad metric. And it seems like, you know, if you’re going to fail, you should at least know ahead of time what that failure is going to do to your organization and how you’re going to adjust the processes that got you there. It’s amazing to me, how many different, I’ve just been taking some notes here, but like one page or two pizza, game day, dive deep, one-way doors. Like there’s so many great aphorisms here are sort of metaphors that, that it seems like bleeders, whether they’re in the public or private sector could take to hearts. When we start thinking like a digital organization, and that really does seem like the biggest challenge. So, I’ll end with this. If you’re speaking to a leader in the public sector, who’s trying to make government more responsive, more adaptive. Maybe it’s not minimum lovable product, maybe it’s minimum lovable government service. What do you think they need to change about the way they frame their job, like how do they need to change their job description and circulate that to allow them to have this sort of minimum lovable services attitude? 

Coral Kennett: So, I think one of the things that government organizations could think about is trying to figure out how can they integrate maybe some of these concepts, like a two-pizza team into some of the things that they’re trying to do. And again, these are sometimes very big problems, but is there a way to break that up and to give teams a little bit of autonomy, even on pieces of a bigger issue to push forward and to try some things and it may show some. Some ways that they can iterate and, and progress a little bit faster rather than, you know, bigger teams who all have a lot of interdependencies. So, in specific instances, that could be a very real way that they could show some of these concepts and see what works for them. These are the things that work for us. We don’t claim that they’re perfect, but they work really well for us. And maybe that’s a way that some of them can start to get integrated into some of the new initiatives that government’s working on. And we’ve seen a lot of that, especially in response to COVID. Government has been highly responsive and so any way that we can support that and share the things we’re doing around working backwards and our methodologies, we’re happy to do. 

Alistair Croll: Awesome. And what’s the best way for someone to find, I will put this in the description, but if someone wants to find out more about the projects you are working on the innovation center and so on. What’s the best way to find that out? 

Coral Kennett: Yeah, UBC has a website, our cloud innovation website that I can share with you. And we also have websites about working backwards and Amazon’s culture of innovation and our processes that I can share as well. And we’re happy, like I said, to facilitate these workshops and work with different organizations as they’re experimenting.

Alistair Croll: Fantastic. Well, thank you so much. This is, the time has really flown by and I didn’t expect to be getting quite so philosophical about the future of work and uncertainty and risk. But it’s been a real, real pleasure covering these things with you. 

Coral Kennett: Thank you. I’ve enjoyed talking to you. Thanks so much.