Ep 49: Breaking News in AI with The Washington Post Chief Technology Officer Vineet Khosla

‍Evan: Hi there, and welcome to Enterprise AI Innovators, a show where top technology executives share how AI is transforming the enterprise. In each episode, guests uncover the real-world applications of AI, from improving products and optimizing operations to redefining the customer experience.  I'm Evan Reiser, the founder and CEO of Abnormal Security.

‍Saam: And I'm Saam Motamedi, a general partner at Greylock Partners.

 Evan: Today on the show, we’re bringing you a conversation with Vineet Khosla, Chief Technology Officer at The Washington Post. The Washington Post is the third-largest newspaper in the United States, with 135,000 print subscribers and 2 and half million digital subscribers.

In this conversation, Vineet shares his thoughts on the mainstream integration of AI technology, the transformative impact of AI on journalism, and the future of personalized news delivery. 

Do you mind sharing a little bit about kind of your background and your role today and you know, how you got into it? 

Vineet: Yeah. I started out getting my master's in artificial intelligence way back in 2003-2005, where honestly, my dad was really worried. He was like, well, what type of job will you get after that? And I got into doing these loan mortgage risk systems. And if you remember that happened right at the cusp of the great financial crisis and the mortgage meltdown, and I was writing, uh, AI models to tell you somebody should what you know what type of loan risk they are.

Well, obviously that AI did not work too well as we know now, but fortunately for me, you know, that gave me an opportunity. They were a group of people in Silicon Valley who were developing Siri. So I ended up being the first AI engineer hire on Siri and started working on the world of language models and language technologies. And that was a amazing turn of event in life, which has led me to then 10 years at Apple developing Siri. I took a break from language and language models and went over to Uber and I wrote the algorithms for their routing engine.

And finally, full circle back, LLMs took off. My interest went back into the language technologies world. My, you know, uh, origin of tech career was in them, and when I saw what LLMs were doing, there was no way for me to sit on the sideline, an opportunity came to lead this talented group at Washington Post, and I think this is the critical part in this world of LLMs, original content and original data, fact based reporting, the writing style, this is what matters, uh, and who better than Washington Post with 150 plus years of history and all the news organizations we have, uh, to be, uh, participating in this language revolution that we are in.

Saam: Well, Vineet, we've been really excited to have you on for a number of reasons, and I was just thinking about like you helped build Siri and sort of Siri as a precursor in many ways to like the AI products we're all used to using now.

You know, we're sitting here in October, 2024, like where are we in the AI cycle? I mean, dating back to when you studied in school and were part of building these early products. Like, where are we now and what's your assessment of that?

Vineet: Yeah, this is a great question because honestly, I do ponder on it literally all the time. AI is one of those techs which will constantly under deliver, right? And that's been the problem with it since the 50s. The minute the machines do something smart, That becomes the baseline and we say like, well, uh, clippy, you know, or like spell checker. It's like, you know, I mean, they were interesting technologies and they were considered AI in their own time. So there is this problem of constantly shifting baseline, which is because we are constantly evaluating it against our intelligence, right? So, That's problem number one, and it will never go away, and I think that's an amazing thing because it means the innovation will never stop over here.

The question of like, is this real? Is this really going to change it? And I tell people, just look around you, right? If you want evidence AI is changing life, just look around you. Just look how people are interacting with chatGPT and Gemini and the llama models and we are putting some products out. Look how it has changed people's day to day life.

This is not crypto technology. Crypto was a really good tech, but it did not impact everybody in their day to day life. So this is where I feel the era where AI used to be specialized, like you had the best AI to play chess and you had the best AI to play go and you had the best AI that did credit card fraud monitoring.

I don't know about you guys, but I think it's been almost like five plus years. Since I've actually had to pay for a fraudulent credit card charge, right, like these systems are so good now, they just catch it, they just stop it, they don't even charge you, I don't even know, and I'm pretty sure in the background, there are thousands of transactions that are being dealt with, right?

So, AI has always been with us, but this generative AI brings it to language. And that really brings it to the forefront of our human experience. Like, we are the talking apes, right? So now it becomes so real, it changes how we communicate with each other, how we communicate with machines, how we put our work out, what things do we value now, right?

What is the role of creativity and ingenuity in these machines? So, Exciting time to be alive, and it is for real now. 

Saam: I think for many of our listeners, the big aha moment with AI was ChatGPT's, uh, release in November of 2022, and in particular, a lot of the writing use cases, right? And, um, I'd be curious, like, Maybe just starting really high level. What are some of the ways you're using AI at the Washington Post, um, you know, to impact your product and service?

Vineet: So one thing Washington Post has been doing AI for a long time, but it was showing up in our content recommendation algorithms and other subscriptions and stuff like that. But speaking specifically of GenAI, we have put four pillars to our company out. The four pillars where we want to use generative AI. 

So the first part we're doing is creator tools and creator tools means everybody in our newsroom, our journalist, uh, to the people who are writing an email newsletter, to a marketing copy, right? Anybody who's doing creative work. But let's focus on the journalism aspect. 

There are so many parts of putting a new story out, which are essential to getting the job done, but they're not essential to the skill, right? So you start with like researching a story, getting data for it, you write something, then you have to do SEO, then you have to do headlines, then you need to do summaries, you have to create social media assets, you have to figure out the timing of your story, when do you publish it, when does it go in somebody's personalization feed, to what ads get shown for it, to subscription, right?

And a lot of this work. gets piled on and be done either by journalist or someone else. So these are the type of tasks where we are saying, we want to move off from having a journalist responsible for all these other things and get them to focus on their core scale. In a way, I call it doing the sucky part of your job. You come tell me what's the sucky part of your job. And we will try and get AI to do it. So that's one big pillar where we're focusing on is the creator.

Then, of course, on the consumer side, the world has changed again. News has always been twofold, right? Like there is the surprise and delight. You open the app, you open your newspaper, you see something really interesting, you read about it, you think about it, right? So news has always been in the surprise and delight. There is a second stream. The second stream is curiosity. You have a question. I want to know something deeper. What about my 401k? Can somebody give me advice on like a personal thing? These are things that newspapers do, but they don't really follow the surprise and delight. This is the curiosity stream we have given up to Google.

So on the consumer side, we just launched Climate Answers, and our hope is to go well beyond climate. So this is a conversational way for you to come and ask Washington Post, uh, what do you think about carbon capture credits? Are they real? Or you can ask any of those things. And we want to expand it from the trusted Climate Answers journalism to everything that we write about. 

The third pillar that we are working on is ambitious journalism. Generative AI and LLM has actually unlocked a lot of possibilities. Now you take the use case of somebody having to go through like 500 hours of security video to figure out what happened on January 6th. This is months of work, right? Or you get a court case or you get some data dump. Now we are using generative AI. 

We internally launched a tool called Haystacker because video was the number one problem for our team, where they can upload huge amounts of video and then start questioning it, ask the system to say, Hey, highlight everywhere, Somebody with the red baseball cap is walking and so on. So that ambitious journalism is our big third pillar that is working on internally. 

And finally, the fourth pillar we are embracing in Washington Post is AI everywhere. We want our finance, we want our legal, we want our HR. We don't want any department inside of Washington Post to be left behind. So the tech team here is building products to enable AI for every employee of Washington Post too.

Evan: Do you mind sharing a little bit about the operational side, right? Where are the areas where you guys have seen the kind of, you know, started to see some of the potential for impact, right, with AI? 

And like, when you talked about the, the workflow, the journey of, of going from an idea for article to publishing, a lot of stuff probably is not top of mind for the average person, right? You don't think about figuring out the right SEO words or making the, You know, 130 character description, right? Or the timing, that's a lot of stuff that feels kind of outside of the core journalistic, you know, mission. It's more like required to, you know, for practicalities in the, in the, in the business and operations.

So like, is, has AI had a big impact there? Like, you know, where, where are you kind of starting to see like little examples, um, of kind of real wins that are emblematic maybe of like future opportunities? 

Vineet: The way I would probably say is there are more green shoots than real wins, right? Um, we have started work and we have implemented some, but I won't call them a win in this case right now.

On the long run, journalism is one part, but as I mentioned, we are doing AI everywhere. So maybe I can talk a bit more around the finance stuff. Uh, there is a unique thing that happens in media industry, it's ads, right? Ad operations. You book a lot of ads and the process for somebody calling us and saying like, Hey, I want to put an ad in your newspaper and these are my criteria as to all the way of us, uh, creating the material, putting the ad, tracking it, figuring out clicks, no clicks and paying them out. This is a long process, right? And that's one of the areas that we are attacking with AI everywhere is saying, Hey, this is a process where AI can do bunch of road tasks for you.

The other area we obviously have is customer service, right? We have a huge install base. We have a population that likes to call and talk in. People want changes to the product. They want to change the subscription. There is a mail stop. They're going on vacation. They don't want the newspapers to pile up. They're back from vacation. There are hundreds of scenarios. So those are the other areas operationally where we are seeing using more AI can help us, um, Make the customer happy faster, right? 

It's just, you know, I know a lot of people think about cost and people equate AI and computers with cost saving. What we are taking the line of is like, okay, if I can free 20 minutes of somebody's time from like a sucky task, you know, a road task, you have no idea with that 20 minutes, what level of creativity, what they will do for the company, right?

So my goal over here is like, free up your time, right? Then just let it be and say, go do something beautiful with your time. We just never know. 

Evan: Like, imagine you kind of, you know, had a coffee with one of your peers who is very bearish on AI. Right. And they're like, ah, you know, I just think like, it's all kind of hype. It feels like a bunch of toys. Like, I don't think it's really going to change, you know, how we work. I don't think it was like real impact there. Is there like, and I know that like my personal is like, you know, Biggest impact from AI is yet to come, obviously, right? 

Are there any examples for like, where you would respond back to her and say, okay, I know we're not there yet, but let me give you an example of like, hey, here's one thing we've done that's already had an impact today, right? Anything kind of like tangible there or any kind of anecdotes you can share where, and even if it's small, kind of like early wins that you've seen that have kind of either helped, you know, free up some of the time, as you mentioned, or, um, you know, create other, I guess, other kind of time savings or, you know, operational savings.

Vineet: I want to give you two examples, and that happened, I think, last week. The Nobel Prize for Physics went to Geoffrey Hinton. The Nobel Prize for chemistry went to Demis Hassabis, the deep mind, right? So this is the first time where we are seeing in this world, the brightest of the bright, the top prize in your field of physics and chemistry went to somebody who created an AI which solved a problem in that field, right? 

This is where I say, don't think that Jeffrey Hinton actually won a prize for physics because he's been a physics professor all his life. And the DeepMind guy didn't win it because he's the world's best, uh, chemistry person who understands protein folding like nobody else's business. No, that is actually not the case. It is the AI they invented that did such a commendable job that other people in that field were forced to recognize their achievement as being top notch. And I'm pretty sure that wasn't easy. 

I am very sure in those communities the debate about giving these guys the top prize in this field was a hot debate. But this, for me, is proof that this is not a fly by night type of technology. This is real. It is making a real world impact. It's making a real world impact on our daily lives. It's making a real world impact on science. And I don't know what further proof can one give here. 

Saam: Yeah, I completely agree. It was like a seminal moment. You know, I was reading some of the conversation afterwards, and some people were hypothesizing, like, hey, how many years will it be until an AI wins a Nobel Peace Prize? Uh, and it's, you know, on the one hand, it sounds crazy. On the other hand, it's not, if you think about the impact. I mean, it actually connects back to what you called ambitious journalism, right? But, uh, If you take that same concept and you apply it in biology or chemistry and you have sort of investigative AIs working and to push the frontiers forward, uh, it's great for it will likely be great for progress and also could could could mean that one of those AIs wins a Nobel Peace Prize at some point. 

What is your observation around how people who have been journalists maybe for decades are learning to like adopt AI into their workflows? And sort of the analogy I think about in Evan and I's world is you have a software engineering team and, and oddly, I found that sometimes software engineers who only have two or three years of experience, But have sort of the willingness to try these new AI tools are becoming much more productive, much more quickly, are you seeing any dispersion in how people are adopting these tools and any learnings from that? 

Vineet: Yeah. I think you will find that pattern almost everywhere.

Um, so once again, I think this goes to the baseline when early career people are coming out, AI is their baseline, right? So they have no problem saying no, I can add value on top of this. This is amazing. It freed up so much of my time. I can do interesting stuff with it, right? And in journalism, like one, It is a profession where you get paid to be a skeptic. You get paid to say, that was awesome, but I have a few questions, right?

Um, so when AI comes, and I think in this case, I had, we talked to the team, we talked to the company, I can understand the fear, right? Listen, creativity is the highest form of humans, writing, dancing, singing, music. We give awards to people, we put them on a pedestal, right? So when these AI models start to be creative, it is understandable everyone's afraid.

All right, let's put that as the baseline and say this is not wrong. It doesn't make anybody bad. But slowly and the way we're doing with creative tools is we want AI to do the part of your job that you shouldn't have been doing anyways, and you start to see a change in people's behavior and their hearts and mind. And of course, some people will move faster than other. But fundamentally, I don't think this is a case of People will not do it right when they see around them. And when they see the actual benefit, the skeptics come around and they use it to their power. But what you observe in engineering, I observe it in the engineering team to.

I observe it in myself, right? I, I, been a while since I've been written code for production, but I started doing it again recently. And I went on the GitHub copilot. I downloaded the plugin on my IntelliJ and I asked it a bunch of questions around like, how do I make a file stream? I'd forgotten that code. I don't need to go read Java talks for it. It just gives me a very nice sample and I'm done. So the curve will exist. 

Saam: This may be a hard question to answer, but like, have you been able to quantify productivity gains? Um, the coming from using these AI tools or were or how do you think about the impact it's having today and where that impact could go?

Vineet: Yeah, um, we have not tried to do that and fundamentally I would probably not even try to do that. I, I think this is a case of more of carrots, less on stick, right? You give people these tools, you let them be productive, you let them go on their journey, you encourage them. Right. Like you obviously give really good use cases. And I said, like when I was writing code recently, I got the AI to write me most of my unit test because as an engineer, I hate that. And I know they're super important. Like there is no way I will check in code without it, but I hate writing them. Right. 

And then the time gets free. So what do you do with your time? You just inherently believe that when we hired somebody to join the mission of Washington Post, there is a big mission part here. They're not just sitting around idly. They're doing something to make the life of our journalists better, the life of our readers and our consumers better. So that's the tack we're taking in terms of like getting productivity gains.

Saam: Makes sense. And yeah, I think, um, a lot of this is like, how do you challenge yourself to use tools where it's not like yet fully clear exactly how to use them and all the ways they can impact your life, but they have so much embedded potential. And I think sort of the companies and teams that like aggressively experiment are likely to see the biggest impact. 

Evan: What's your view on like what the future looks like. So it's five years from now. And, you know, I'm, I'm on the Washington Post website, um, like how's that readership experience different, right? How is kind of the, um, the content creation different? How is the consumption different? Love to kind of help you or love to have you love to hear you help us dream a little bit about what that future world looks like. 

Vineet: Yeah. Um, I think, uh, that future world goes back. It goes back to a statement I made. We want to meet the consumers where they are, right? If there is a universe of people who really appreciate the 8000 word piece we wrote on a deep investigation. That will never change. Those are our core subscribers. They write to us when they read something like that. Nobody else does it. Nobody else. When I say nobody else outside of news does it right? No casual observer can do it. 

There are people who love our opinions. The people who actually love our tech talk videos, for example, my wife was like, well, I don't need to read Washington Post because your Instagram stories are so good. I know it right.

It's special level of creativity, right? So the way I think about this is there is a news, right? And we have already always thought of it as words. But now we're moving into the world where a news can be a summary. I don't have time. Can you catch me up on financial news? Right? Or I'm going to office. I actually don't want to hear anything hard because guess what? Office is hard. So I just want you to give me 10 minutes of good news and I'll deal with the rest later. So we want to build our experiences where reading, listening, summarizing, watching a video are all parts of our core journalism. 

Uh, we are. Making these changes in our tools. We're making these changes in our consumer products. We're making these changes in our journalism and our teams. If you guys had followed, we recently said we're building a third newsroom. Maybe for the tech audience. This is new. Typically, in every news company, there are two newsroom. There is a newsroom. Where you have that editor in chief who's pushing stories out. And then you have opinions. These are always kept separate and the goal behind is one is more fact based and day to day, and the other one is more opinions. And that's it. In the entire history of news, there have always been two newsrooms that are independent. And we said, no, we have a third newsroom now.

This is the newsroom which is going to focus on everything I said earlier, right? Um, this is the newsroom where you go to when you want to be an adult. When you say, like, I am 40 years old. I have never invested in 401k. I am embarrassed to even ask anyone about it. But guess what? On our side, Michelle Singletary, she writes really great columns about personal finance. Now I'm going to go to my, ask the Washington Post AI and say, I don't want to talk to anybody else, but can you tell me what Michelle says I should do with my 401k? We might, you know, give her advice a little bit. 

So, The goal over here is stop being one size fits all type of a destination of surprise and delight. Just come to the app. I exactly know what you want, right? Like, how crazy is that? Like, I opened the app and I know what everybody in the world wants and I'm going to show them just the right story and they all want to read this one and that's it and nothing else. So we are moving away from that. So that's where I think in five years from now you will naturally see Washington Post meet you where you are and our goal is to be your daily good habit. 

Evan: So if you kind of go back, I don't know, 50 years ago, you know, people would, uh, journalists would write a piece of content.

It would end up in a newspaper. You know, probably at some point, like every copy of the newspaper is the same. The content that was in there, the sequence of the things that were in there was kind of artistically curated by an editor or some kind of expert at that thing. Now we're kind of in this, you know, website age where a lot of the, like everyone has access to the same content, right?

The front page of Washington Post is probably similar for most people, right? There's probably some personalization, the recommendation right, is kind of re-sequencing that content. Um, but like, you know, if you go to a, if I send Saam a link to an article, he's gonna read the same article I do, right? Same level of, you know, same, you know, length of content, maybe same tone. Do you, do you imagine like, you know, that's what the journal, that's where the journalist wrote, do you imagine a world in the future where there's kind of a separation between the content that was written by the human versus the presentation, you know, back to the reader, right.

Cause you could, you know, you can imagine, I, I, I can actually, were like. You write it once that gets kind of rendered into maybe different styles, different languages, maybe the easiest one, maybe different formats, right? It's written in text and it's read it, you know, it's generated audio or maybe generated video in the future.

And so like, to what extent, like, I guess in the spectrum of personalization, how far do you take it? Right. And where does it end? Right. Like, cause there's some world where people create a bunch of source content and then you have your personalizer AI that renders the, the sequence, the style, the tone, right? Maybe, you know, they make it a little bit more happy for me because I'm a grumpy person and like a little bit more, you know, sad for Saam because he's such an optimist about the world of AI. Like, where does, what's the, is there a limit on kind of personalization, right? Given that the technology constraints and doing some of this stuff is quickly being removed.

Vineet: Uh, you've hit, I think like the heart of the debate here, right? And my answer to this one is we as a society will have a discussion and we will figure it out. I don't think this is for one company to decide. 

There are easy things to do. Like we have all of our stories read by an AI voice now, which means when you're in the car or in my case, like when I'm cooking dinner, I just love to hit the play button and I let the machine take over. And I'm like, You know, this is not the time I'm really going to do hard news. So, uh, if the algorithm really doesn't meet my entire need, it's okay. It's a thing I'm doing in the background anyways. So those would be the easy low hanging fruits, right? We write summaries. So we have started getting some, some AI generated summaries in our articles, but we have the journalist look through it. We give the journalist a gate check and say, Hey, this is the summary of the article. Did it the capture it? If not edit it. Because at the end of the day, I don't care who wrote the summary, whether it was a AI or the journalist, if it captured the intent of the article, right? 

The tricky part becomes when you say, Well, that was a very polarizing article about one candidate. Can you put it in a different light for me? And you start putting in words that the author, whether it's journalist or anyone, did not intend. So I think that's the heart of the debate is like, when does the needle go all the way to the other side where you just feed it random content and say, create me a happy news story.

At Washington Post, the stance we are taking is, you know, there are a lot of things we can do with AI and there are a lot of problems. You know, as I said, the sucky part of your job, but creating impactful stories is not one of the problems we have. We are winning Pulitzer's, we are winning Peabody awards, which is a sign of the quality of journalism. Uh, we have a very healthy audience. We have an engaged audience. When we put a story out, we get an average Two to 3000 comments per story. So no, this is not people on Twitter talking about us. This is our own people on our own and operated platforms having a debate with each other, right? So if that is not the problem to solve, I don't want to touch it in the near term, right?

Evan: So we got, we got to shift to our last, our final section, which is, um, our lightning round. Unless, Saam, you have any last minute things, but I think we probably got to move to this. 

Okay. So this lightning round, um, as the name might suggest, looking for kind of, you know, shorter answers, right? I did the one tweets version or the one, I don't know if you still do tweets these days, or one, the one XR, the one, the one, uh, tweet kind of article.

So just got like a handful, handful of questions here. So let me, uh, let you kick it off first. 

Saam: Awesome. So maybe, maybe start, um, how do you think companies should measure the success of a CTO? 

Vineet: The velocity of products being made. 

Evan: Vineet, what's one piece of advice you wish someone told you when you first became a CTO?

Vineet: I don't have an advice for CTO, but I think there's just general management advice that I have that I wish somebody had given to me when I was first as a manager is be curious, don't be judgmental. 

Saam: So maybe to switch gears to the personal side, what's a book that you've recently read that had a big impact on you?

Vineet: Oh, I am the type who reads two or three books at the same time. So the first one I'm reading is, uh, Max Tegmark, our mathematical universe. So this is basically using mathematics to define our universe going from all the smallest to the biggest numbers. So he uses really good numbers, uh, big to small to show the vastness of the universe. And I really like it because at some point reading this book, you go, Well, maybe this is a simulation, right? Like this probably is a simulation and we are some eighth graders science project. 

Um, the other one that I'm reading is, uh, the Vital Question by Nick Lane. So over. In this book, he really talks about literally down to the level of electrons, like what happens when you eat food? How do you get energy? How does energy move? How does sunlight down to the level of electrons moving through our body?

So these two books are like a very interesting perspective down to the tiniest mechanism that makes life. And, the biggest number which shows, well there might not be life. 

Evan: What's an upcoming technology that you're personally most excited about?.

Vineet: I'm actually very excited about the AR and the VR promise.

Communications is really important for us as humans. You know, we are on zoom, we are doing podcasts. Uh, but I moved to this country in 99 and I remember I used to work really hard to buy a 10 phone card so I could call my parents for 10 minutes in India. And over my lifetime, I went to like, Freer's webcams, and now with the face time and these technologies, communication is free, right? I can pick up my phone anytime and I can talk to my parents or anyone in the world, as long as they have an iPhone. And I think AR / VR really changes communication at another level where you can feel you're in the same space with people when it's done right.

So I am really excited for AR/VR, not for the games aspect of it, but for human to human communication aspect of it. It really bridges oceans, in my opinion. 

Evan: Awesome. Well, Vineet, I know we're coming on time here. I just want to say I really appreciate you joining, uh, joining us today. Um, I wish you more time to talk about all the cool stuff you're doing over there because, um, you know, I think Washington Post obviously has a huge impact on the world, but just want to say thank you for joining and looking forward to chatting again soon.

Saam: Thanks a lot, Vineet. 

Vineet: Thank you.

Evan: That was Vineet Khosla, Chief Technology Officer at The Washington Post.

‍Saam: Thanks for listening to Enterprise AI Innovators, I’m Saam Motamedi, a general partner at Greylock Partners

‍Evan: And I'm Evan Reiser, the founder and CEO of Abnormal Security. Please be sure to subscribe, so you never miss an episode. You can find more great insights on enterprise AI transformation at enterprisesoftware.blog

‍Saam: This show is produced by Josh Meer. See you next time.