Why Agents Can't Be Governed Like People with Amazon Web Services VP & CISO Amy Herzog

Evan Reiser: Hi there, and welcome to Enterprise AI Defenders, a show that highlights how enterprise security leaders are using innovative technologies to stop the most sophisticated cyber attacks. In each episode, Fortune 500 CISOs share how AI has changed the threat landscape, real-world examples of modern attacks, and the role AI can play in the future of cybersecurity. I'm Evan Reiser, the founder and CEO of Abnormal AI.

Mike Britton: And I'm Mike Britton, the CIO of Abnormal AI.

Evan: Today on the show, we're bringing you a conversation with Amy Herzog, Vice President and Chief Information Security Officer for Amazon Web Services. AWS is the world's largest cloud provider, and its active defense systems process more than 400 trillion network flows every day. That scale makes Amy one of the few people who can say what AI genuinely changes for defenders, and what it doesn't.

A few things stuck with me from this conversation:

First, Amy points agents at the most repetitive, pattern-based work first, so her people stay focused on the problems that actually need context and creativity. Attackers still want the same things; what's changed is the speed and cost of getting them.

Second, Amy thinks vulnerability management is about to change dramatically. When a comprehensive scan across your whole system takes minutes and costs almost nothing, you can't defend a 30-day patch cycle anymore.

And finally, Amy thinks AI agents should not be treated like humans. Agents are trained to seek a goal, not to fear consequences, which means the controls we've leaned on for years have quietly stopped working. We need entirely new identity an d authorization models built for them.

Evan: Amy, thanks so much for joining us today. Do you mind sharing a little bit with our audience about your role and how you got to where you are today?

Amy Herzog: Yeah, absolutely. So, my name’s Amy. I am the CISO for AWS here at Amazon. That means I care for the security of our customers, of AWS customers. I work on a larger Amazon security team under Steve Schmidt, Amazon’s top-level CSO. And this is a fun job because AWS has really included security as a key customer success criteria or business result from the first days that we were building the cloud. That was a pretty big technology change, and this is another one. So, it’s a fun time to be in this role.

Before this, I joined Amazon a little over three years ago now as our CISO for our specialized businesses, which is a nice way to say the things that are not the cloud or the bookstore. So, that included a bunch of fun stuff like our Alexa devices, our Kindles, our broadband satellite internet company, our movie studio. Just a bunch of really interesting and necessary stuff.

Evan: As we think about AI’s ability to really help and support defenders with security, where are some of the areas you feel like the promise of AI is really holding up? And what are some areas that we need to be more thoughtful about, maybe where we invest more as a global security team?

Amy: To give our audience a sense of scale, we process over 400 trillion network flows a day as part of our active defense systems. You’re not doing that if a human is looking at a dashboard for everything. That’s just not the way my job works. And so, as AI grew more capable and it was clear that there was benefit here, we already could fit it into our ordinary practice of: okay, what is the most repetitive volume-based or pattern-based work that we’re doing?

Let’s see what new chunk of that type of work this technology can help us do at machine speed, so that we can keep our human brains focused on the places that really require local context and creativity and human engagement. That was the pattern that we have been using for a long time to make automation pragmatic across our work program.

And that’s really what we’ve done with AI, as well. It’s not radically changing any of the fundamentals of attacks, or what attackers or threat actors are after, or the fact that we need to defend. It’s changing the specifics in a way that I see as a continuation of a broader pattern.

Evan: Performance is a big part of what you do. Does AI unlock new opportunities to approach problems in new ways?

Amy: Yeah, I think it does. For us and for threat actors, I think the thing that’s really changing at the moment is the speed and scale at which things can happen. And there’s an interesting economic aspect of that, for threat actors and for defense teams. But a more direct answer to your question is that it gives us a new ability to expand the volume and depth to which we’re carrying out large pattern-based work. So, if you think about a thing that any security team needs to do, which is to take in some set of potential issues and then understand the local prioritization across those issues based on your particular systems and your particular configurations.

That process has been one where AI has really helped us increase the speed at which we can deeply understand the full volume of issues. When you think about a CVE intake queue or something like that, you can really get deeper eyes or more sophisticated analysis on a much broader set of potential issues much more quickly than you could before. So, that’s one specific example of that.

Evan: What would be your advice to CISOs out there about which areas are most ripe for investment? And how do you make sure you’re actually injecting technology or new tools in a way that’s going to achieve the benefit without creating too much complexity?

Amy: I am often kind of in the middle, which is frustrating sometimes for people. There’s some skepticism amongst technology practitioners for anything that captures the public imagination. And I think there’s some real, durable, tangible benefit that really shouldn’t be dismissed. So, I understand both sides of that. My advice would be to think first about places where you have the ability to understand the quality, the false positives, false negatives, or precision and recall for your human activity.

And then you can build a system where you can understand comparatively: what is this better at? What is it worse at? How do we need to rethink the way we’re doing this process so that human engagement can effectively be scaled by this technology, instead of just automating what you do today without really worrying about any of that?

I still think that in 2026, focusing on something that’s measurable, so you can be clear what the real impact is, and maybe avoiding some of the shiny tools this year, will help you with your agility over time. With this field changing so much so quickly, it’s very important to be able to ground yourself in what you’re really trying to do for your customers and your business.

And the best way to do that is to have a set of measures that you can go back to over time to keep you centered on what’s really working and what’s not working. It’ll help you adapt.

Evan: You’re working on cybersecurity, but you also have exposure to new AI products, right? And you can probably imagine new potential threat vectors or worries. Is there anything that’s been surprising as you’re exposed to some of these new frontiers ahead of some of your peers? Any lessons or insights there about new risks, or new ways of carrying out some of these threats?

Amy: There are a couple of things, but let me start by talking a little bit about what’s not changed. The threat actor's motivations and their goals have not changed. Credentials, data, system access, the things that were interesting to a threat actor are still interesting to a threat actor. And really, what we’ve seen throughout this whole last few years is that the speed and the cost to get those things is where we’re seeing the real difference, and that’s for them, as well as for us.

And so, as you think about how that works inside a security program, and talking to my peers and other security teams, a lot of the muscle memory that we’ve built up as a field for how to accomplish this job needs to adapt to the changes in cost and scale. You might have assessed something as difficult or expensive, and hence unlikely, and taken one risk mitigation approach five years ago, whereas that’s no longer necessarily appropriate.

And we saw that bear out in our own processes internally too. I don’t forget, however many years ago it was now, the first time you have a conversation with your builder partners and say, “that’s not actually all that hard anymore.” That’s a different conversation. So, really making sure that you’re understanding and keeping up with what the state of the art or state of the practice is, and then making sure you’re communicating that to your business so that they are with you in this problem-solving moment, rather than it setting up a tension. Those are both super important.

Evan: What are the things you see more of, or things that maybe as defenders we should start paying more attention to, to make sure we’re focused on the right things for the next chapter of the threat landscape?

Amy: I am focused on speed, and in particular, making sure that my team has the creativity room in their day-to-day operations to keep up with how things are changing. So, if I think about what I’m evaluating my team on, and really asking them to show me, and discussing with my business partners now versus four or five years ago, I’m quite focused on details about the speed of our actions, particularly proactively. I think that’s really key. This was always important to us internally, but now it’s even more important that I’m making sure my humans are able to bring their best critical thinking and human creative side to their job.

Evan: Any pro tips, any rituals or traditions or best practices that you’ve found helpful on your team to keep the team focused? Any pro tips for us aspiring team leaders out there?

Amy: Two core mechanisms across all of Amazon, including in security, are our business reviews and our ops reviews. And in my business reviews, you have to explicitly look not just at the end format that you might want to discuss with the board, or your business leaders, or whoever else the security team is reporting to, but at the details around your security achievement and your work program that help you audit and inspect as a leader.

So, one crisp example there is that you might report out from an SLA perspective, because that’s a shared industry language around vulnerability management. In my BR, I’m interested in time to recover or time to mitigate. And I want to not only see the mean, because the mean doesn’t really tell you the shape of your work program.

In particular, I want my 10th percentile: how good can we possibly be? And I want my 99th percentile: what’s the longest pole? So, that’s one example of how I would recommend any CISO look internally at what you’re doing. And another is in our operations reviews, we really do look at how much time people are spending on their more rote operational tasks.

On average, do we have the ability to NFA? How many failed experiments have we had? If that number is zero, something’s wrong. And then, if you want a couple of questions to just start talking through with your teams to get started, I would definitely ask: what are you building? What do you need to learn next? What would have to be true for us not to need to spend so much effort to do this task or that task? That will start to excite some ideas around innovation that can be helpful. Those are all pretty common practices inside Amazon.

Evan: What are some of the upcoming trends that you think are most important, that you would encourage your team or other teams to really be paying attention to right now?

Amy: One topic that I’m quite passionate about, that I think is still early days, is that AI agents and agent action need to be treated differently at a fundamental level from either system action or human action.

So, I think when you first start playing around with agents, it can be tempting to think about them as the other form of non-determinism that we have in our lives, which is humans. And they are really not. Their motivations are totally different. They do not have any notion of eventual consequence, which is key to a lot of controls on human behavior. You don’t want to get fired, so you’re not going to take down this or that. Agents just don’t think that way. They are trained to seek a goal.

And I think what that means as a security team is that we need to be very thoughtful right away and shift our thinking about what it means to do risk management with this kind of autonomy, and to give yourself the ability to treat it entirely differently. So, I might think about a human as having either role- or attribute-based permissions or authorization. I might think about a system’s deterministic behavior as something I can tightly constrain. Agents share bits of each, but are not the same as either. And so, internally, we’re spending a lot of time thinking about and developing new authorization models and new identity models, such that we can get the different security picture these new types of principals need when it comes to securing the agents.

Evan: What are some things you think people would overestimate or underestimate?

Amy: It’s weird to be a security person saying this, and there’s a lot of nuance here, but I’ll say maybe the spicy thing first. I want to make sure that we hold them to the appropriate bar. Humans are not perfect, and so refusing to get the benefits out of agents until they can perform at a state where we want them to be better than humans. Obviously, we want perfect security performance, but I think there needs to be a discussion, and measures, which is, I guess, why I talked about it, around: how good are they performing? Is it better than humans, even if it’s not perfect? How do we feel about that risk? I see that the moment an agent makes a mistake, we’re done with the full technology. These are things you need to control for, and risk-manage, and understand deeply.

But I think we can do that. If you measure the efficacy of what your overall system is doing, even without AI agents, and then you understand how agents fit into that, you’ll be able to take a more pragmatic path forward. So, that’s one where I think maybe we, as a field, over-index on perfection versus understanding the deeper detail.

I think we underappreciate how tightly our controls are matched to a human deterministic system context. So, we’re definitely seeing that it’s deep in the grooves of the security field to rely on controls like asking, are you sure? On some level we’re relying on that consequence anticipation as the behavior inhibitor. And I think that is not going to serve us well. I would like us to stop thinking about these as human or human-attached, because that tempts us mentally to put a bunch of goals and consequence-driven attribution onto those agents that I think is not warranted.

Evan: While we’re on the topic, any other spicy Amy takes? I feel like these are the most interesting.

Amy: I think vulnerability management is going to change dramatically. The biggest thing we’re seeing so far is the speed and low cost at which you can do a fairly comprehensive search across a system.

And all of this, anyone listening can do right away. By the way, Opus 4.7, for example, is a great tool to either augment or auto-harness for security work. So, there’s lots of this that’s really practical for anyone listening. You can see for yourself how quickly you can look across your properties. It’s fine stuff.

And I think that’s going to be true writ large. That means we can’t talk about 30-day patch cycles anymore. That’s just not commensurate with the threat of the near future. And that’s such a profound difference. Talking about a month versus minutes is a really big change. So, I think that’s going to have pretty impactful consequences on the field, and on how we think about operating and how we let it evolve.

Evan: What do you think needs to change for us to survive in this world where there could be a novel attack every day, on a new vector we couldn’t have imagined yesterday?

Amy: The easiest or most pragmatic answer is to just start understanding how your response looks in terms of minutes. Just do it. It’ll be illuminating. It’s frightening to start thinking about, maybe, but it’ll be illuminating and it’ll help you drive what you need in your teams super fast. But I don’t think that’s the actual answer, because that’s still a fundamentally reactive cycle, like you said.

And on the proactive side of this equation, which is, I think, how the industry is going to need to adapt: it was a less usual thing when I first took this job for proactive teams to really exist, and be launch-blocking, and be engaged with builders all the way through the shift left. That was kind of an aspirational dream. I think that’s where the center of our work is going to need to be. And that means that process has to be embedded throughout the developer workflow. One of the things that’s true of my team is that we are builders alongside AWS builders, and I think that’s going to need to be true for all security teams.

You can’t just think about a point-in-time check anymore. You have to make sure that the system that’s producing the code is producing good code. And then the other thing that I’m thinking about and investing in right now is that a lot of the security industry approach has been focused on this reactive view of, okay, what are all of the things a bad guy could do? And then how do we think about mitigating those? What are the paths? I’m moving toward a more proactive version of that too, which is: what are the invariants that need to be true of the system? What do we want to always happen? Any flow must have multi-factor authentication, or whatever it is for your particular thing, and then verifying that, which is also quite a fundamental shift.

Evan: How do you strike the balance between trying to deal with the overwhelming scale of the new attacks coming in, but also trying to get ahead of things? How would you advise any of your peers to think about that?

Amy: Yeah, I mean, first you want to just be respectful that that’s real. I think that’s step one. But the second thing, the more hopeful thing, is that we were actively defending against 400 trillion network flows a day before AI. So, it feels overwhelming, and that’s super real, but there are lots of very pragmatic, near-term, well-understood things we can do, and you can do, in order to get yourself the kind of speed of operations and scale of operations that you’re going to need.

The key thing to focus on to unlock that for any company is business owner support and partnership. So, if your CEO is not thinking, “oh, I need to really watch out for my security team and make sure they’re able to scale to meet this moment, I’d better ask them what they need.” That’s a thing to work on first, because once you have that partnership with the business, it’s much easier to do what you might need to do to your own security team, whether that’s hiring scientists or software developers to help you scale, or evaluating a different selection of vendor products or infrastructures or architectures you can build. Each team will look a little different.

Evan: What gets you excited and motivated? What makes you feel positive about the bright future?

Amy: I am a deeply optimistic person. I think in the short term, we’re going to have a bunch of really interesting challenges to solve in short order, but that’s okay. I love hard jobs and I love innovation, and we’ve got the best possible set of folks to work on these challenging problems. How meaningful and motivating to make such a difference in the world. So, I’m optimistic about that.

I think the particular cause for optimism is two things. One, our own adoption of this technology as security practitioners, and our use of it in our day-to-day world, will bring us as a field closer to the builders we are trying to work through to protect our customers. So, instead of us speaking an entirely different language and having an entirely different frame of reference, I see this as a step closer to what our internal customers do, and I think that’s a very positive and optimism-generating step.

My developers don’t want to give this up. My developers don’t want to slow down, so it creates that commonality. And the second thing that gives me optimism is that the ROI and the business decisions around investing in security foundations and successful scaled practices, this is the moment for us to make and succeed at that argument.

There is this broader awareness of how important it will be to get things right at the foundation and build on that strong foundation. That’s something I have personally believed true throughout my career, that AWS has believed true from the beginning and built into the cloud that we created. And now I think the broader industry is looking at this problem with fresh eyes and seeing the same things that we do. So, very optimistic that, based on those, we’re going to be able to do something pretty cool in the next little while.

Evan: So, we only have ten minutes left. We like to end our episodes with what we call a lightning round, which is not super original. Basically we ask a couple of questions that are very hard to answer succinctly, but we’re looking for the one-tweet response. So, please don’t totally hate me if these are too challenging, but I know you like challenges. 

If you could share one principle with every CISO, something you’ve learned in your career the hard way, what would it be?

Amy: You have to keep the threat actor and their capabilities at the center of everything that you’re doing.

Evan: What’s your advice for security leaders to stay up to date on the latest technology trends at this moment?

Amy: I think talking to one another and your peers is one of the better ways to stay up to date, and I would recommend augmenting that with periodic random searches of grad student publications in this field.

Evan: So, one the more personal side, is there a book you’ve read that’s had a big impact on you and your leadership? If so, I’d love to hear what and why.

Amy: Getting this to a tweet will be hard, but let me give it a shot. I think particularly for CISOs at this level of leadership, there are two that I would recommend. Measure What Matters is a classic. Maybe three: The Five Dysfunctions of a Team, and making sure that your team is acting as a coordinated team instead of as the heads of their functional silos, which is super important for the role in this moment.

And the last is a newer bit of work by one of my MIT professors, actually, called Step Up, Step Back. It’s about how, as a senior leader, to figure out how to set the vision and the measures correctly so that you can get the benefit of your team’s innovation, rather than being very focused on prescribing the how of your teams doing their work, which takes away their agency and autonomy.

Evan: What do you think will be true about the future of AI and cybersecurity that most people today would consider science fiction?

Amy: The obvious answer is autonomous attack and defense systems, with security professionals operating at a more strategic level. That feels like science fiction, but it seems to be on the path that we’re on.

Evan: What would be your advice or guidance to that person who is just thinking about getting into cybersecurity, or maybe is already there and trying to step up for bigger impact? What coaching would you share with them?

Amy: Absolutely get hands-on with all of the tools. Figure out a local experiment you want to run, or a thing you want to learn, and just do it. This is not a moment where you can read and fully prepare for the change that’s happening all around us.

Evan: If you’re a CISO listening and you believe this would be helpful, but you’re unclear where to start, what would be your first one, two, three? What are the best projects to kick off to just put some points on the board, and also show the team that it’s possible?

Amy: So, I would start by looking at the portions of your work that have two characteristics. You have a huge volume of it, or it’s fairly pattern-based or repetitive, and it is eating up the operational day-to-day of the humans you want thinking about the bigger problems. The sweet spot is when you have both of those things, because it is both going to make your teammates’ jobs more interesting and it is going to give you deep practice with what this stuff is good at.

Evan: Any other advice for AI quick wins for peers out there to try and have an immediate impact on security operations?

Amy: Yeah, have fun with it. It’s worth the time to have a little fun. But if you want to nudge it in a productive direction, ask your teammates, and ask yourself: what are you learning? And really hold yourself accountable for writing that down. That will also kick the iteration off.

Evan: Appreciate you making time to chat. I really enjoyed the conversation, and I'm looking forward to doing it again soon.

Amy: Yeah, thanks so much for having me. It was a pleasure.

Evan: That was Amy Herzog, Vice President and Chief Information Security Officer for Amazon Web Services. 

Mike: I'm Mike Britton, the CIO of Abnormal AI.

Evan: And I'm Evan Reiser, founder and CEO of Abnormal AI. Thanks for listening to Enterprise AI Defenders. Please be sure to subscribe, so you never miss an episode. Learn more about how AI is transforming the enterprise from top executives at enterprisesoftware.blog

Mike: This show is produced by Abnormal Studios. See you next time!