Ep 68: AI Beyond Enterprise Speed with Marsh SVP & Chief Information and Operations Officer Paul Beswick

Evan Reiser: Hi there, and welcome to Enterprise AI Innovators, a show where top technology executives share how AI is transforming the enterprise. In each episode, guests uncover the real-world applications of AI, from improving products and optimizing operations to redefining the customer experience. I'm Evan Reiser, the founder and CEO of Abnormal AI.

Saam Motamedi: And I'm Saam Motamedi, a general partner at Greylock Partners.

Evan: Today Saam is talking with Paul Beswick, Senior Vice President and Chief Information and Operations Officer at Marsh.

Marsh is one of the world's largest professional services firms, with four major businesses spanning insurance brokerage, reinsurance, HR consulting, and strategy. Paul has spent nearly thirty years there, first as a consultant writing the tech strategy and now actually delivering it across the firm as CIOO.

A few things stuck with me from this conversation:

At Marsh, Paul's team is piloting a voice-based crowdsourcing tool. Instead of emails, surveys, or meetings, employees just dictate two or three minutes of their thinking, and AI cleans it up. When a big news event hits, they can ping their specialists and turn scattered expert opinion into a scalable asset the firm can actually act on.

Second, The first version of LenAI, their internal AI tool, took only a couple of days to build and was rolled out to the entire firm in just a few months. That early bet taught them tool routing before MCP and multi-step planning before agent harnesses existed. Their home-built version of an agentic AI tool now trails the commercial market by only a few months, and at a far lower cost.

And finally, Paul says the cost of the meeting to discuss what you're going to build is now often more than the cost of building the thing. LenAI itself was built over a weekend, before anyone debated whether to do it. When production-grade applications take days instead of months, his advice to other CIOs is to break their three big AI projects into twenty small ones and ship them sequentially.

Saam: Paul, thanks for joining us. I’ve been looking forward to this conversation. Just to start, can you give our audience an overview of yourself, your career, and your current role at Marsh?

Paul Beswick: So I’m quite surprised to find myself nearly 30 years at Marsh in various forms, although through a number of very different phases. I started as a summer job in the consulting business that we own, Oliver Wyman, and was there for about 23 years. I ended up running the global digital practice. 

Then, slightly to my surprise, I ended up deciding that I should go and deliver the tech strategy that I’d helped write as a consultant by becoming the global CIO. And then more recently, I picked up the global operations team as well. So I currently run our tech and ops teams globally for Marsh and all of its companies.

Saam: Amazing. And maybe it’d be helpful for our listeners… I think not everyone may appreciate the scope and impact that Marsh has. So can you give us an overview of Marsh: the company overall, the scope of operations, and then what falls under your purview and your role as CIO?

Paul: Yeah. The business is, I think, well known by people who use them, but largely B2B and so less well known to the world at large. We have four major businesses. Marsh, which is an insurance broker and risk advisor. Mercer, which does a variety of work around investments, pensions, benefits, and human resources consulting. Oliver Wyman, the firm I spent a long time at doing strategy consulting. And Guy Carpenter, which is a reinsurance broker and helps make sure that the insurance business is also on safe ground, as well as all of their clients.

Saam: Are there any specific examples you can talk about, either things you’ve done organizationally to enable the supercharging of this judgment concept specifically, or things you’ve observed employees do with the tools that you’ve given them that really stand out?

Paul: We have a project that we call Crowdsource at Marsh that we think is an interesting play. We’re testing it out in a few places, and the idea is pretty simple. At the end of the day, if I want to gather this stuff, one of the best ways to do it is to ask people. But the way we ask people today is we either ask them to type a long email, fill in a survey, or have a meeting.

These are all pretty inefficient approaches that require quite a lot of coordination. What if we just ask them by putting a question in front of them and recognizing it’s easier and faster to speak than it is to type? Just get them to dictate an answer. Bring it back into a central form. Get AI to clean it up and process it.

Now I’ve got an artifact that I can actually scale. Just as a sort of information capture engine, we’re seeing some interesting use cases for that approach. You take a client team and survey them every month to ask them what is happening in that client. What are the key issues that are on people’s minds? What are the hot topics that we need to know about? We know this stuff, but we don’t know it in a way you can act on systematically. Some people do that already. A lot of people don’t really systematize it in a way you can start feeding other things off it.

Another example would be some big piece of news hits. The Strait of Hormuz is blocked. What do our marine people think about this? Well, let’s just ask them. Make it a big deal. Let’s just ask them to give us two or three minutes of what they think. And the combination of those things, rather than just being some stuff that people said once, becomes the foundation of an asset that actually can scale and create some network effects.

So that’s a pretty interesting one that we’ve been finding a lot of use from, and I think is quite exciting.

Saam: Are there any other things you know are unique or interesting that you’re doing?

Paul: I think a lot about white collar jobs and companies as essentially factories that make decisions, where that is the output and the raw material is the information and the data. And then there is the decision process. So being able to effectively feed that data in ends up being really important. Moving from, say, dashboards and spreadsheets floating around into something where you just ask and it comes to the point of the decision on its own, I think, is an interesting play. 

But I stumbled upon something completely random about two days ago, just in response to a question someone was asking in one of their Teams rooms. And it’s not a big play, but it’s interesting. I’m sure people will recognize the spreadsheet that they get sent every week or every month, that they have to look at and do something with, that isn’t necessarily particularly user friendly, but at least always looks the same. A nice little use case for Claude Cowork to build me a local HTML file that doesn’t need a server, doesn’t need to be deployed anywhere. It sits on my desktop, but is designed to take that spreadsheet just by dragging it in and give me the dashboard and a breakdown of everything that I actually need to know in that dashboard, just mechanically and in half a second or so.

It’s quite a lot of that sort of stuff that sits around the place. Another interesting one. We have a tool called LenX, which is a browser extension for our LenAI tool. Think about data entry problems. Obviously, ideally, you engineer them away, but in the meantime, LenX knows what page it’s on. And so, writing a little custom module that says, “For this app that maybe we don’t control the source code for, whenever I’m on this page, copy all these values out and store them temporarily in some sort of cache, so that when I go to this page in the following app, I can produce a one-click button to paste it all in directly.” It doesn’t solve my main problem, but it does make it a hell of a lot more accurate and a lot faster. So there’s lots of little things like that that get interesting.

Saam: Those are both really good examples. And I think they illustrate something I often talk to leaders in our portfolio about, which is that some of the most consequential things are actually quite small. The HTML example you gave wouldn’t show up on a board-level slide, like, “Here’s what we’re going to do with AI this year,” but that could make somebody’s life way more productive and change the way they distill and glean insights from that spreadsheet. And there’s all sorts of knock-on effects that are hard to a priori estimate.

Paul: Oh, it’s one of the big problems with AI, right? It shows up in so many places, so many of which are only partly scalable and very, very difficult to monetize. The story of it is quite a difficult one to tell. It’s fine if you can find a $100 million problem and go after that. That’s an easy story to tell. But so much of it is the accumulation of hundreds or thousands of different things that people who do different jobs have figured out to make themselves more effective and put that time into something more valuable. So, a much, much harder story to tell than a lot of other types of technology.

Saam: I want to talk a little bit about some of the internal tooling you’ve built at Marsh. In doing research before the show, I did some reading on LenAI, which seemed really interesting. So maybe I have a two-part question. One is: tell us about LenAI as a kind of good concrete example. And then, tell us about the system underneath that allows you guys to build these types of things and the investments you’ve had to make.

Paul: Yeah. So we started very early on this. LenAI’s first version was created late spring ’23, and we had it in pilot by the beginning of the summer to a few hundred people. So that was probably a few weeks after we could get access to OpenAI models through Microsoft, which was sort of the big security unlock for us at the time, and we added it to the entire firm by the end of that summer.

It’s sort of grown over time. But the nice thing about it, especially early on, was it’s incredibly cheap. It cost us a small team, not very long to build. The first version actually took a couple of days to put into place and get into that pilot group, and now we’ve sort of built on that and developed the ecosystem around it. It’s been enormously helpful in terms of learning how this stuff works and having a really direct, hands-on experience of it. 

So we were building tool routing approaches well before MCP came along, because we needed to route to tools and figure out a structure for doing that. It is nice that we don’t have to do that anymore, and now we can just hand it off to MCP. But we learned a lot about the realities of how you do that effectively at that point in time. We were building multi-step planning things ourselves before harnesses came along for doing that kind of thing. We’ve sort of started to switch those out as the market has provided standards for those sorts of things, but it’s also helped clarify a lot of the stuff that allows us to make this accessible.

LenAI itself is fundamentally a colleague productivity tool, and it’s very general. It’s also a very low-cost experimentation space where anyone can try out their ideas. We can see what works and pick up the best and go and do something with them. But in doing that and listening fundamentally to what the organization wanted, it led us to build out document processing capabilities and translation and summarization at very early stages, and more recently, connectors to data of various kinds and connectors into the office suite. All these sorts of layers of new tools and capabilities that helped expand the number of use cases that we could take on, regardless of whether LenAI continues to exist.

By the way, this is Lenny right here over my shoulder. I don’t think many companies have a soft toy for their internal tool. Regardless of whether this thing sort of continues to live forever, all of the stuff we’ve built to connect into the rest of the organization, it’s all the scaffolding we need for the industrialized use cases, productivity and efficiency plays we’re running in the ops organization. If we buy a commercial tool, what we would plug into it to extend its capabilities into our environment. So it’s been a great learning experience, and it’s built us this sort of ecosystem, a kit of parts, and we need to go and do this at scale and in the places where we can really pin value to this very directly.

We continue to find that even if we lag by a few months as to what’s out there commercially in the market, we only lag by a few months, and it costs a lot less to do it this way. And so, for instance, our Claude Cowork equivalent is a home-built version that we have in pilot. We’ll start to roll it out to the organization pretty soon. It’s taught us a lot about writing great agent harnesses. And that issue is not just one that relates to a general-purpose productivity tool. It’s really important as we start to put agents into the heart of some of our operational processes as well.

Saam: Maybe to segue into one part of the business we haven’t talked about, which is sort of the insurance and risk management side of the business. What’s the role of AI there going to be? And, again, we like to tease out the contrarian in our guests. So, are there things you think about how those industries are going to reshape over the next three or four or five years that you think are maybe not well understood yet?

Paul: Yeah. So the insurance space in particular is a very document-heavy ecosystem. There’s a lot of interchange of information that happens via the medium of PDF, and there’s an enormous amount of time, cost, and delay that’s caused by the realities of that. So a big, big push on document ingestion and systematically capturing, parsing, filing, and then re-routing a lot of that information based on capabilities for reading that stuff that are dramatically better than they were even a couple of years ago.

So that is a big play. We see good gains from that. And I know lots of people are looking at this sort of space, but it’s very fundamental to the way that we operate today. So I expect faster, lower cost, more insight being gathered, because the cost of gathering the marginal piece of insight from the stuff that flows through is all going to help us be a stronger advisor to our clients, to have a deeper understanding of the market, and to serve them more effectively.

So on that side, that’s probably the biggest single play. But there are lots of ways in that business and in others that we’re transforming the way that our clients interact with us, the insight we have, and the data that we have, to make it just lower friction, easier to get the stuff at their fingertips, easier to do simple transactions in ways that are lower cost for them.

And then probably the third piece, focusing on that space specifically, would be arming up risk advisors with more insight into what’s happening in the marketplace and the ability to build more effective and more strategic coverage programs for our clients, because of the data that we’re able to bring to that conversation, because of the depth of insight into the market and all the things we talked about around knowledge management and data. It allows the corporate, enterprise-wide scale that we have to turn up on the front line much more systematically and consistently.

Saam: How do we measure and justify the ROI on AI? We intuitively can feel it, but how do you… I’m curious for you specifically, but maybe even just more generally: what’s the case that one makes to the CEO or the board around continuing investment in AI, given the amount that one can actually spend on these things?

Paul: If you keep it cheap, it’s a lot easier to show ROI, or not have it be a big issue. And one of the things that we’ve done with the strategy we followed is it is a dramatically lower-cost strategy, especially on the colleague productivity side, than buying into tools that come with very large upfront commitments and big bills, which almost force you to start pushing targets down to people to help cover the costs, because you can’t pin it to any one thing.

It is general productivity, and the only way you can really make that fly is by making the cost problem someone else’s. We haven’t had to do that, and I think that’s been enormously useful with this exciting new technology coming in with all the excitement and none of the fear. So that’s certainly one part of it. But there are different plays that we have. It’s that productivity is hard to pin down.

But be smart about the way you’re building, and you can do it dramatically more cheaply than you think. From a client transformation perspective, we see revenue growth in a number of our products as we’ve taken them through an AI transformation, just because the experience is so much better and more compelling and the value that the clients can get from that is much higher.

We have a product called Aida in our immersive business that takes a huge amount of insight about managing talent and dealing with rules and regulations in different countries around the world, based on a huge database of documents that we have from ourselves and from other sources. It was incredibly difficult to navigate. But now you can just ask and we will find you the right stuff and bring it back to you in a form that’s most useful to you.

It’s a massively better value proposition that’s helping drive revenue growth within my organization. On the operations side, I think that’s another point here, though. I saw… I don’t know how I can give credit for this, but I saw a LinkedIn post from someone that resonated with me. Increasingly, the cost of the meeting to discuss what you’re going to build is more than the cost of building the thing. And one of the things that’s an even higher cost in a lot of companies is the months of negotiation to get budget, to even start thinking about doing anything at all.

The way we think about ROI, the way we think about funding these things, is going to have to change. Because, you know, if you take the beginning of LenAI, there was no conversation about that. It was done over a weekend, and it was in pilot. And the conversation was, “Should we do this thing?” The thing was done.

There was zero cost to doing that. In reality, the conversation was: are we going to like how the cost grows as people use it? But the cost is proportionate to the use and to the value. So it’s a very different place to be if you can knock out production-grade applications in weeks and sometimes days.

And we have done the multi-month negotiation process. The CapEx is insane. And I think there are pretty fundamental changes obviously happening in how software is being built. The need to be commensurate, changes in how organizations think about where software should be built, by whom, under what constraints, how the money flows for doing this. And so a big topic for us is thinking about how we reorganize our tech team and match it up against a business in a fundamentally different way to reflect that.

Saam: Well, there’s a lot more I want to talk about, but I want to make sure we get to our lightning round before we run out of time. So we’ll jump into our lightning round. And we’re looking for sort of the one-tweet version of each question, which I admit is a little difficult on some of these. Does that sound good?

Paul: Give it a go.

Saam: Okay. Awesome. So to kick off: how do you think companies should measure the success of a CIO in today’s AI era?

Paul: That’s a pretty tough one to start off with. I think the most fundamental is the ability to out-innovate your competitors.

Saam: If you’re talking to a new CIO in another industry, so not a personal competitor, and he just started his new job, and they say, “Hey Paul, we can do three projects in our first year. What are the three AI projects you would recommend someone start with?”

Paul: Oh, I think there’s a problem right there, because I think that implies big projects. You need to do dozens of projects. So, I think my main advice would be to break those three into 20 each and do them sequentially, one at a time. We’ve got to break these big projects; they’re designed to fail. You can’t show momentum that way. It doesn’t sit with what people want to see. And they create a dynamic that accumulates risk and drag and increases the odds of it, and it adds cost and risk at the same time.

Saam: What’s a book you’ve read that’s had a big impact on you and why? It doesn’t have to be work related.

Paul: Astonishingly little time to read these days, and I mostly read fiction when I do. My favorite book is Titus Groan, because it’s so wonderful. The characters are so wonderfully well drawn. I do find myself thinking about that, but I don’t know what relevance it has to the rest of my life, other than it describes this sort of ancient, fantastical, rambling castle, which perhaps is quite like most people’s legacy IT systems.

Saam: What is the best way, as a CIO, that you can just stay relevant on the latest in AI in today’s enterprise AI trends?

Paul: I’m a big believer you have to do it. You have to try it yourself. Just having coders involved, that is a problem. I have always liked to have a few side projects going at any point in time, and I code probably weekly if not more. I like to try things out and feel it hands-on. And I think you’ve got to touch and feel it to really understand it and get a sense for its potential.

Saam: I completely agree. And maybe one final question: what do you believe will be true about AI’s impact on the world that most people listening would still consider science fiction?

Paul: That is a big question. I’m not sure how to answer that. There are so many possible futures here. I think what I’m reasonably sure of is that by the time my kids are my age, they’ll be doing jobs that don’t exist right now and we’ve never thought of. And we will find we have a capacity to innovate and come up with new things to do that will surprise us.

Saam: I completely agree, and that’s why I get quite frustrated by the conversations around there being no more work. I think many jobs we do today, including in some ways the job I do, if you had asked 100 years ago, “would this job exist?” The answer is no. And human ingenuity will never be defeated. So I think there could be a messy period where there’s work transformation happening, but I’m very optimistic. And I agree with you. I think the next generation will be doing things that will be hard for us to describe and imagine today.

Paul: Yeah, that notion is the most dangerous thing in the world at the moment.

Saam: Yeah, exactly right. Terrific. Well, thanks a lot for joining us. Really excellent conversation. I could have kept going for a few more hours. And I love some of the specific examples you shared. Really excited to get this out to our listeners.

Paul: Excellent. Thank you, Saam.

Evan: That was Paul Beswick, Senior Vice President and Chief Information and Operations Officer at Marsh.

Saam: Thanks for listening to Enterprise AI Innovators. I'm Saam Motamedi, a general partner at Greylock Partners.

Evan: And I'm Evan Reiser, the founder and CEO of Abnormal AI. Please be sure to subscribe so you never miss an episode. Learn more about enterprise AI transformation at enterprisesoftware.blog.

This show is produced by Abnormal Studios. We'll see you next time.