Christian Catalini, tech founder, co-creator of Meta's Libra, and founder of Lightspark, joins Beyond the Prompt to explore the unexpected economics of AGI. Drawing on his recent paper, he argues that as AI makes intelligence increasingly abundant, value shifts to what can't easily be automated, measured, or verified. The conversation explores what becomes scarce when intelligence is cheap, why the doomers may be wrong, and how AI could unlock entirely new waves of scientific discovery and innovation.
Christian believes the AI era will be defined less by generating outputs and more by evaluating them. As intelligence becomes cheaper and more accessible, the people who create the most value may be those who can distinguish good work from exceptional work and help guide increasingly capable systems.
The conversation explores verification, judgment, and why expertise still matters in a world where AI can perform many tasks at a high level. Christian explains why today's experts are both highly valuable and simultaneously training the systems that may eventually replace parts of their work.
Jeremy and Henrik also explore what this means at a personal level. They discuss building AI agents that reflect your own preferences, creating personal verification systems, and why AI may make it easier to learn new skills, switch careers, and pursue more ambitious ideas.
Key Takeaways:
Christian's LinkedIn: linkedin.com/ccatalini/
Christian's website: catalini.com
Economics of AGI: full paper
Jeremy's Persona File Template: YouTube/The8Files
00:00 Non-Measurable Frontiers
00:32 Meet Christian Catalini
01:08 The Economics of AGI
03:09 Why Verification Matters
06:37 Can Everything Be Measured?
10:32 The Rise of the Verifier
14:35 When Intelligence Gets Cheap
21:46 Building Your Verification Harness
24:08 Human + AI Augmentation
30:18 Persona Files and Privacy
33:12 Reasons for Optimism
36:00 Career Switching in the AI Era
39:31 The Debrief
📜 Read the transcript for this episode:
[00:00:00] Christian Catalini: I do think there are things that are fundamentally non-measurable, and that's where things get interesting, and all the doomers I think are wrong on the, "Okay, we're, we're just gonna be completely displaced." Think about deep signs, deep tech, things that we haven't figured out yet.
A lot of the science discovery that's gonna happen over the next few years, I think it's essentially AI finding re-combinations of things that have already been mapped. If you think about all the possible permutations between different disciplines, different topics, humans have only explored a ti- a tiny percentage of that, so the impact of AI on scientific discovery and innovation is gonna be massive.
Hi, everyone. I'm Christian Catalini. I'm a tech founder, uh, with roots in academia. Tried to break the financial system with a little project called Libra out of Facebook a few years ago, uh, then launched a startup called Lightspark, which is focused on using crypto for cross-border payments. Over the last year now, I've been obsessing like everybody else, uh, about AI and what this all means for society.
We recently released a paper that tries to grapple with the question of, okay, now that we're close to AGI, what does it mean for all of us? And I look forward to discussing it with all of you today.
[00:01:02] Jeremy Utley: You mentioned, you mentioned the paper on AGI. For folks who aren't familiar, what's the thesis in a nutshell?
[00:01:10] Christian Catalini: Yeah, so the paper was born out of a existential crisis. Having been in crypto for more than a decade, um, it, it was a very natural moment to look back and say, "Was this all for nothing?" Right?
If you look at the landscape of where most of payments in crypto is going, it's getting more and more boring. In a sense, it's like a wave of enterprise sales. Uh, you have banks and other traditional financial institutions connecting to these networks. It's getting more concentrated, uh, not less, uh, like in the original crypto days.
And it was really clear that AI would transform everything, right? It's going through every sector of, of the economy and really reshaping how it can be built, how it can be operated. And so we, we had this question around if we're near AGI, and people have all sort of conflicting definitions, but, like, my, my favorite one is, is very simple.
It's essentially something that is as good as human for most tasks, is a very, uh, useful type of intelligence. It may not be exactly like us, which is fine, but it's, you know, a peer of sorts. And I, I, I do think we're relatively close to that. If you take that as a given, then the next question is like, okay, what does it mean for, for society, for the economy, for things that we should be paying attention to, things that are gonna be defensible, things that are not gonna be defensible?
So the paper is almost like it's too applied for academics at this point, and probably too theoretical for people tinkering with, uh, with Open Claus and the like. But it was an attempt at, at really teasing out core economic principles behind this transformation. So there's a lot of, uh, economic, uh, work in this area, but it tends to be a bit too detached from reality.
I mean, that's why, you know, back in 2013, we wrote The Simple Economics of the Blockchain. Same idea, right? So you have this fuzzy new object that's coming your way. Economists are really bad at predictions, but they're kind of good at isolating the fundamental forces behind some of these transformations.
And so the paper is just an attempt at saying, "Look, we're not gonna get 100% of this right, but can we get 70 or 80% of this right on at least what, what the economic forces are?" And what we concluded is that, look, intelligence is getting commodified, it's getting cheap. I think everybody agrees on that. But in economics, typically, when something becomes cheap, something else becomes the bottleneck.
It's kind of, sort of exciting and depressing, which is like, oh yeah, age of abundance. Well, n- not, not quite yet. And the bottleneck we identify in the paper is what we call verification. Now, we have a very precise definition of what verification is because there's a lot of cope going around. People talk about judgment, curation, taste as being kind of the, the holdouts for us humans.
Um, we call it verification, and so I'm happy to unpack that i- if useful. Um-
[00:03:48] Jeremy Utley: Yeah. Let's
[00:03:48] Christian Catalini: go ... but that's where we- Let's
[00:03:49] Jeremy Utley: go. Yeah, let's go. Yeah, yeah. Yeah. I mean d- I mean, to me, two natural questions follow. One, how do you define verification? But two, why is verification not solvable by AGI as well?
[00:03:58] Christian Catalini: Yeah, yeah, absolutely.
And then those are excellent questions. So let me, let me start with the first one, and then we'll, we'll get to the second. We, we started from a very simple intuition, uh, which it seems like most people, uh, building these models agree with, which is like, as soon as something can be measured, it can be automated.
So AI can take over anything for which we have enough data, enough digital trails. If you look a lot of the progress over the last years as being, we can also use AI to measure more things, uh, which is amazing. Um, but things that looked unsolvable like self-driving, given enough miles, given enough, you know, vision into the systems have now reached human level, if not superior than human.
So if you believe that anything that can be measured will be automated, then the role of humans is simply, you know, what AI isn't measuring yet. Um, and I think there's something quite special about that, which is what, what's in our own weights. So if you think about your brain having its own, you know, model and its own weights, when you're born, you start recording all sort of events and instances and calibrating those weights.
Some of the most successful individuals in any single profession have very uniquely calibrated weights. So think about, you know, top designer, a top lawyer, a top medical doctor, a top engineer. The experiences that have formed them, all the mistakes, all the errors are adjustments. You know, there's a lot of reinforcement learning from not just human feedback, but essentially the environment.
And a lot of that has been codified and then tracked on the web, on all the sources in the books. So the models have seen a lot of it, but not all of it. And so if you look at what's the difference today between someone at the peak of their profession And you know, a model is typically the model has seen everything that the expert has not seen, and that is verification.
So verification is the act of that expert based on all of their historical record, everything they've seen, all those out of distribution examples that, you know, they've captured and calibrated on their own experience saying, "This is a good output. This is not a good output. This meets the bar." You know, you're essentially looking at the agentic output and saying, "Are we there yet or not?"
Um, now to your second question of like, okay, can't AI do that? By definition, and here is, is kind of we're cheating, right? We, we're saying, by definition, AI can do anything that can be measured, and if there's something that the humans has measured that the AI has not, that, that's why it's verification.
And so of course, as models get better, that gets thinner and thinner and thinner, so humans need to keep moving up the frontier, but that's the entire idea of the paper, which is saying, look, AI is coming. So you can call it judgment, but that's, that's a cop-out. You can call it taste. You can call it curation.
These are very precise terms, and people will disagree forever on what they mean.
[00:06:38] Henrik Werdelin: Do you think everything can be measured? So for example, I'll make the statement I love my wife and my children a lot, but I don't know what KPI I would put on that, right? Like, I don't know what metric I would put on that.
And so is that just kind of because we've never needed to understand how to measure that, or do you think there are some things that are innate non-measurable?
[00:07:03] Christian Catalini: Let me start with, with the conclusion, which is I do think there are some things that are non-measurable. Now, your example, and I have to be careful how I answer it because of course I also have a lovely family.
Uh, I would argue that social media companies may have a very good proxy of that if you're an active user, uh, of that, exactly that statement, and maybe it's, it's an imperfect proxy, but from your messaging history, from a language, like there's probably ways to tease out. Now, a lot of those interactions happen offline, and they're not digitally captured yet, so that might be one reason why that may not be available to an AI today, but it could be-
[00:07:39] Henrik Werdelin: But that's gonna be the whole world model will try to solve that, right
[00:07:43] Christian Catalini: Or a, you know, a device that tracks you, that, that you carry that records kind of all of your interactions will probably be able to give a pretty good assessment of your question.
That said, I do think there are things that are fundamentally non-measurable, and that's where things get interesting, and all the doomers I think are wrong on the, "Okay, we're, we're just gonna be completely displaced." Think about deep signs, deep tech, things that we haven't figured out yet. They're not available to the eye, they're not available to us.
A lot of the science discovery that's gonna happen over the next few years, I think it's essentially AI finding re-combinations of things that have already been mapped. If you think about all the possible permutations between different disciplines, different topics, humans have only explored a ti- a tiny percentage of that, so the impact of AI on scientific discovery and innovation is gonna be massive.
It can be very creative within the known bounds. But insofar as there's stuff that hasn't been measured and we need to build the tools even to measure it, I think we can play a role there. Then there's a whole other part of society that doesn't depend on measurement at all. It's not objective, right? So if you think about meaning making, all the way to religion, like these are things that have value because humans agree they have value, so they're more about consensus.
Blockchains, right? So why does Bitcoin have value? Well, because people agree it has value. And could Bitcoin or a different technology lend in the same scenario? Yes, but you have to create that historical record, that coordination around it. So I think there's aspects of society that don't... They're not objective, and they're almost like, you know, think about art, what makes good art versus bad art.
A lot of it is social coordination around some of the, the trends, the topics. Will AI be, will be good on generating some of that? Probably. Uh, but, uh, those are lens of unmeasurable scope. Think about the stock market. It's kind of the unknown unknowns, to quote the famous Rumsfeld quote, right? It's stuff that we cannot really assign probabilities to.
It's still fundamentally uncertain. To, to put a big caveat on all of this, I, I do think that's where word models are extremely dangerous to some extent in, in terms of displacing that too.
[00:09:44] Jeremy Utley: What do you mean? S- d- define- Yeah ... define dangerous.
[00:09:48] Christian Catalini: Well, in a good way, uh, and, and also potentially negative in terms of substitution, if we can have a very realistic simulation of, of a process, and we can run infinite counterfactuals and scenarios very cheaply, that's probably not that different than what happens in our brain when we think about what ifs, and we're very plastic in, in how we respond to the environment, otherwise we wouldn't have made it for, for all these years, right?
Uh, through natural selection. That point might be mute. Right now it's a massive advantage over the machines, but what happens when a machine can run all the potential geopolitical scenarios, right, in, in every permutation, um, maybe with a quantum computer and, you know, infinite compute and energy? Do we have an advantage at that point over, over an AI?
Maybe not. That- that's where, where maybe the gap really closes.
[00:10:34] Henrik Werdelin: On the verification, and just to kind of like bring it to people who are listening to this show, I think a lot of people are trying to figure out in their personal life, but also in their companies, like, what, where do we go, right? And then they realize that the, uh, Anthropic's of the world and the, uh, chip makers only do, like, three to six-month planning, and then, like, you know, their three-year plan suddenly become kind of like, what?
But if you were to try to help people take the verification argument and apply them to their own world, how would you think about that? What is verifiable in my world that I might try to kind of like cling onto, for example?
[00:11:14] Christian Catalini: Yeah, and the paper in section eight, for those of you that are courageous enough to, you know, to dive into the 100-page thing As a few conclusions on that, I think there's good news.
And, and again, I think if people embrace the technology in the right way, the upside is, is way bigger than the downside. There's gonna be displacement, there's gonna be a major labor impact. The transition will be painful. Uh, I, I'm not gonna deny that. I, I think that's, that's gonna be self-evident. But to your point about what can people do, so a few things.
So first of all, um, right now I think there's massive value in being kind of a top verifier in your specific domain. If you happen to have that, you know, final 1% or 5% of knowledge that allows you to distinguish simple agentic output from excellent output, um, you're, you're the bottleneck, right? So in the, the classic O-ring theory of economic development, that, that piece is actually the one where all the friction is gonna occur, and your job is highly valuable.
And that's why I think you're seeing top AI talent being paid, you know, very, very large amounts, or foundational labs hiring a bunch of, uh, you know, TradFi experts to train business models or financial models, and then they're gonna hire life sciences expert and so on and so forth. Being a top verifier is really valuable right now, but there's a side effect to that, which is, of course, as you're doing the job, you're not that different than the people that were labeling images at the time where, you know, the scale
[00:12:36] Jeremy Utley: AI or- You're putting yourself out of the job while you're doing the job.
Yeah.
[00:12:39] Christian Catalini: Correct. So we call that the codifier's curse, and so we need to have the mental plasticity to keep moving up the value chain because in a sense, our job gets thinner and thinner, it gets more important as it gets thinner, and it becomes much more what we call the director. Um, that's kind of a Hollywood reference, but you can think of an entrepreneur being a director.
[00:12:58] Jeremy Utley: An orchestrator, yeah.
[00:12:59] Christian Catalini: Orchestrator. Yeah, yeah. What- whatever you wanna call that. It's the person that sets the intention, steers the system, checks that the system is still aligned with the original intentions, course corrects on like, okay, this is quite not there. We, we're, we're kind of f- forget the evals.
There's something else that we need to target here. That role, I think, is gonna be increasingly important. And if anything, maybe even enterprise jobs are all becoming that The reason why you see many of the layoffs beyond the convenience of the narrative, of course, some of it, you know, many of these tech companies had a lot of bloat.
But beyond that, I do think many, and, and Jack Dorsey's post I think was hinting at this, realize that we're going towards a new architecture where these orchestrator director types are gonna be a lot of leverage on their job, and you can do a lot more with a lot less.
[00:13:41] Henrik Werdelin: It's funny, I, I'm starting to call it the oligarchy organizations, right?
Where you basically have, you know, a few people that do hundreds of peoples' jobs. And once you kinda like think through thinking about that, it's very difficult to unsee how it could be different. And to your point about people getting paid a lot of money, you know, at one point, you know, you hear these salaries and you go, "That's just insane," right?
But if you're kinda like thinking that somebody can do 300 people's job, then paying them 100 people's salary could potentially still be kinda like, you know, like a, a s- a sane thing to do. And
[00:14:14] Jeremy Utley: by the way, it's, it seems like it's a kind of a short-term payment for a long-term gain, right? Because per the codifier's curse, you actually don't have to pay that person that much- Yes
for very long. I wanna take one step back. There's something around growth mindset and lifelong learning that I think is hugely valuable that I wanna get to, so if I could just table that and hopefully we can- Yes ... collectively remember to go there 'cause I, 'cause I really think that's valuable. That's, that's a
[00:14:36] Christian Catalini: super important point, so I, I'd love
[00:14:38] Jeremy Utley: to go there.
Yeah, so, so I really wanna get there. But I wanna take one step back first because we just kinda blew past something that you said almost as an offhand kind of a given, a premise, which is intelligence is cheap. It's now metered, and the bottleneck is verification. Can you talk for a second about what are the implications of intelligence being cheap?
And the reason I want to just put a fine point on that is I don't think many organizations are currently commissioning resources as if intelligence is cheap. Right now they are still pretending or believing intelligence is the bottleneck, and so I think it's just worth putting a fine... I don't know what you have to say about it, but I really feel like it's, it's a point worth making because we can't...
You don't earn the right to have the verification and growth mindset conversation if you don't start from the premise of intelligence is no longer the rate limiter on your organization's ability to grow, et cetera. So could you just riff on that?
[00:15:32] Christian Catalini: Yeah. It's almost like the different stages of grief, right?
So you first accept that intelligence is going. And again, that's why I think we wanted to take a pretty strong take on the, on the taste curation judgment or even agency. There's all flavors of terms that people, every, every few months I think if you follow AIX, there's, there's a new term that people are, "Don't worry.
We're gonna be fine because of this." We wanted something very precise, right? Which is like, okay, is there data behind this? Can you measure it? And if you can measure it, that's bad news. I, I think intelligence becoming cheap Is first of all, we've all experienced it, and probably the biggest shock that I've seen, at least among engineers, as being the December to, you know, early 2026, where people went from, "Okay, these things are useful.
They're kinda cute." They, they get lost to, "Oh, wow, I, I, I actually need to rethink my job completely." And it's happening in coding and engineering first because, of course, these are the people building the tools And the realization, I mean, is like, okay, before maybe you were checking a good chunk of the code, now you cannot operate at that scale anymore.
The code is being generated at the speed and pace and quantity that no human can, can verify it. And it's very clear that the things have shifted. So for the engineers that have embraced it, they're not writing code the old way anymore. They have tons of agents. They've, they kind of sharted themself in, in many crazy ways.
Uh, and now the, the, the blowback is like, okay, after the moment of glory, which is like, "Oh, wow, I did what I was supposed to do in a week i-in, in three hours," it's like, well, you actually didn't. If you actually look carefully through it, um, some of that will break, some of that is not secure. We're introducing all sort of slop, and coding is one, but look at writing, right?
Uh, it's the same thing. I think we've all experienced it, where we write the structure for a piece, we get AI to kind of polish it, and then you read it and you pause for a second and say, "W-wait, wait, this doesn't make any sense." Which is like, it sounds plausible. It, it looks like the real product, but the logic isn't there, and, and that's one of the known weaknesses of these models at this point, right?
When you think about what they can and cannot do,
[00:17:39] Henrik Werdelin: especially when you want to write something long, right? You know, I, I wrote a book- Yeah ... about AI, and I was like, "It's gonna be easy. I'm gonna get AI to do it," right? And then you get through the first chapter and you go like, "This is all words that seems right- Right
but when you read it, it seems emp- like empty calories," right?
[00:17:54] Jeremy Utley: I mean, sorry, correct me if I'm... And by the way, I just turned in my manuscript, and I also couldn't cajole AI to write it on my behalf, much to my chagrin. But that sounds like intelligence isn't cheap yet. Just to be clear, it's, it... And so maybe in a sense we aren't there yet, but I think there is something to be said for...
Code may be, like a, a code word or a placeholder word, right? When Andrej Karpathy says, "I haven't written a line of code since December," you know, imagine that your job is, is also encoded in symbols called letters on a screen, right? If you're a lawyer, your job is, right? If you're an HR professional, your job is letters in symbols on a screen, right?
But instead of the, the word code, I haven't written a line of contract since December. No lawyer is saying that yet, right? If you're in HR, I haven't written a line of job description since, right? I think most people think code's this other thing, uh, rather than, no, the future is unevenly distributed, and the law- lawyers need to be able to say, "I haven't written a line of contract," right?
But talk about the implications if intelligence actually is cheap.
[00:19:01] Christian Catalini: Yeah. '
[00:19:01] Jeremy Utley: Cause right now we actually were just going on a tangent about how it's not really that intelligent Which I think gives people a reason to, uh, you know, go, like, plug their ears and go, "Blah, blah, blah." See, it's not really gonna do it.
[00:19:13] Christian Catalini: Yeah. That's a good
[00:19:14] Jeremy Utley: point. And if somebody stopped right now, I think- But it's
[00:19:15] Christian Catalini: coming.
[00:19:15] Jeremy Utley: Yeah, yeah, yeah ... I think they would conclude, "Oh, yeah, I don't have anything to worry about." And I think that's actually the wrong message. The right message is- Right ... "No, it is coming."
[00:19:23] Christian Catalini: Yeah, and, and look, I think y- you mentioned legal contracts, right? So I think we can probably all agree that if you're writing a standard NDA or the many templated contracts, an employment contract, AI is already there, like maybe 99%. And sure, do you want a human to do a final check if it's a million-dollar transaction? Maybe. But the gap, the perceived gap between what the lawyers will tell you the system can do and what the system is actually already doing, I, I think is much narrower. Now, for a lot of processes in society, that last 1% or 5% really matters. And so I, I, I think the difference is that... And we've all seen it, and coding it, again, is the first one where when you hear top CEOs saying, "X percent of our code is AI-generated," first of all, I, that, that gives me a little bit of pause because, you know, we've seen also all sort of hacks and, and breaches and other side effects of that.
But it is true that it is good enough code for many, many purposes that good engineers are not reviewing that code anymore, or at least they're not reviewing it in the same way. The whole conversation between model versus harness of the last few weeks, right? I think it's really a reflection of we're realizing that there's something missing in the foundational models that's really valuable, it's really important, and it's kind of getting us there.
Um, but to get to your point, I do think that if you think about another improvement, like the same that we had between December and, and, and the early spring, we're pretty much there for many of these, uh, white-collar jobs. And yes, look, there's always gonna be tail legal contracts, tail medical cases, tail engineering, architectural problems where having that super skilled human, again, with very special weights in their brain, looking at the same evidence, making a different conclusion, that's still gonna be extremely valuable to society.
But there's more and more of the run-of-the-mill use cases for all of those sectors that the model can do.
[00:21:17] Henrik Werdelin: But maybe just then to double-click on the concreteness of it. I really like your, If you're a top verifier and you feel like that's probably a good place to be, at least for a period of time, right?
Double-clicking on, uh, Jeremy's question. If we are done with scarcity and we're in a world of abundance, you seem to have thought a lot about what does that then mean, right? Do you have, like, a thought on... Again, I'm sitting here, I hear these thing. It's very abstract. How do I kinda, like, bring it back to my own world?
[00:21:48] Christian Catalini: Yeah. So I, I think we should go by type, right? Are you an individual? Are you a company? I think there's slightly different conclusions for each one of them. Uh, we already touched on the individual, right? Which is like, in a sense, the best thing you can do is to build, I mean, for lack of a better word, it's, is your own verification harness.
Like, think about your job today. You can now do the, the output of many more people if you figure out how to use these models and build your own verification infrastructure around it. Going back to the book-writing example, I wrote my own kind of writing harness and, you know, it's-- depending on the day, it delivers good results, and it's still kind of being evolved.
But I can say that something that took me two days to write, you know, a 800-word piece, takes me a lot less. Now, is the AI doing all of it? Absolutely not. Is it helping me once I can give it a very strict set of guardrails? Absolutely. And so I do think that tool allows me to be more productive, and everyone in their profession probably are, are having that experience.
So recipe number one is saying, if verification is the bottleneck, think about your job, think about all the grunt work that you were doing before that now is commoditized intelligence. Don't do that. That's a waste of your time. And second, build the best possible scaling machine around the part that's really valuable.
That, that's how you augment yourself. And by the way, this goes back to Jeremy's point about continuous education and, like, the lifelong learning A part of it is what you're doing today is not gonna be valid in six months, so it's a moving target for the individual. I would say that's, that's probably the, the nugget for the individuals.
At the company level, it's more nuanced, right? So first of all, it's like, what's your mode? You can start thinking through what is really defensible in a world where commoditized intelligence is everywhere, and if I have more tokens than you. You know, think about cybersecurity. Cybersecurity used to be a talent game, now it's also a token game.
How much of it is it a talent game versus a token game? I think it's still a mix, and especially with a zero day, well, companies will be willing to pay a lot for the hybrid, where you don't just throw Mitos at a problem. You also have Mitos plus some of the best engineers that have been doing pen testing and all this stuff for, for ages.
Over time, it shrinks and shrinks and shrinks, and so maybe eventually it's a war between is it you or the attacker doing more proof of work, which is a very crypto, crypto conclusion to the entire industry. Um, but you, you can think about modes.
[00:24:10] Henrik Werdelin: Do you believe that, like in chess, the best way to beat the best computer is to be a human and a chess computer together?
Do you think that will be the case for a while?
[00:24:21] Christian Catalini: I think it's inevitable, and, and the paper kind of and the economics of it hint at it. At some point, the only way out is through augmentation. We have to become the thing. We have to be similar enough to the thing that, um, there's no difference between us.
Also from a safety and security perspective. Um, you know, when you think about alignment, it's very clear if you read some of the things that people way smarter than me on the alignment space are, are thinking about, it's not gonna be like a one-shot thing. It's gonna be like raising a child, especially in the first iterations, then maybe they'll, they'll raise each other.
But if you believe in that model, we need to be able to understand their preferences. We need to be able to understand their talents, which, you know, la- I think it was last week, Anthropic released some interesting, um, model activation experiments where they're trying to, you know, reverse engineer what the model is actually thinking.
But you know, at some point, the models are gonna be so smart that if we don't have a brain, brain-computer interface and we cannot process things at the same speed, there's no way we, we, we're gonna be in line with it. And sure, you could hope that the model has the same preferences as humans, and we did a wonderful job the first time, which I think is very unlikely, or you need to be the thing.
[00:25:28] Jeremy Utley: What are the highlights... I haven't read the Anthropic model activation paper. What are the-- Are there any interesting highlights relevant to this discussion?
[00:25:36] Christian Catalini: I thought it was really fascinating on how they were kind of using a model to try to match, you know, what came out from, from the internal thinking process in a way that it's almost like a verifiable trace of what happened inside the black box.
Um, this goes back to another theme of the paper, which is like verification is gonna be increasingly useful also as the technology stack, and that's where maybe crypto eventually will make a comeback. Like we'll need verifiable inference. We'll need all sort of like additional tooling to make sure that when these things are running, they're running with accountability and provenance in a way that...
I mean, right now nobody cares, right? Because essentially it's, it's a race. But as these systems start taking on sensitive things or even more important parts of the economy, I think we'll look for that.
[00:26:20] Jeremy Utley: Now, I wanna come back. You had mentioned we, we just talked about moats at the company level, but going back to the individual level, I think some folks will probably get some of this verbiage, but when you say build your own harness around the part that's valuable, if somebody said, "Okay, like I'm listening to Christian talking to these guys, I'm ending the podcast right now," is there a tool stack?
Is there-- How do you even think about building your own harness?
[00:26:45] Christian Catalini: Yeah, and, and look, you can get very sophisticated, so depending on, on your expertise, you can enter this all the way from, you know, fine-tuning your own model. But, like, I would say the simplest entry point, and maybe it's why people are so excited about things like OpenClaw and Hermes, that, that's essentially the barebone version of, like, your general computing platform, which is saying, "Look, there's these models.
You can run them locally, you can connect them to the cloud." But now you're starting to train an agent or a group of agents to your preferences, your behavior, your intuition of what's right and what's wrong. Um, give you a really simple one. I think we've all experienced how LinkedIn is very problematic when it comes to spam, right?
99% of messages are spam. Every now and then there's someone legitimate that wants to get in touch for, for a good reason. Um, a lot of that is a judgment call still today, and I'm sure LinkedIn could build a better tool, but you could imagine training one of these, uh, agents to just do that, right? Based on who I am, the kind of business I'm looking for, the kind of conversation that might be valuable for me versus not, help me screen my opportunities.
Uh, that's a really simple application for that where, again, you're verifying based on your own weights, is this message coming in something I should be paying attention to, and do I really need to read the 99% that's gonna be total junk?
[00:27:59] Jeremy Utley: But is that an MD file? Is that a skill file? Is it a GPT? What-- When someone is, is training that harness, where are they doing it and how are they doing it?
Is it in a conversation with an AI that's a harness generator, right? What's that look like?
[00:28:12] Christian Catalini: Yeah, I-I think, look, the UI/UX of the conversation with an agent as being kind of killer. What we haven't figured out is probably the, the proper memory piece, which is, like, conversations tend to be erratic. It's a problem, yeah.
You jump topics. And, and I think someone eventually is gonna crack that problem of, like, making that conversation as useful as possible, so some stuff goes into long-term memory automatically. Other, it's like the dreaming function that again was released recently and many have been trying I think that's, that's probably the missing piece from a UIX perspective, but eventually these models will be smart enough to just work with you alongside and learn what matters to you, what doesn't matter.
Then again, you can get a lot more sophisticated in the skill. A skill is already an upgrade from-- I mean, I would say most people probably are just having a long conversation with, with their LLM or multiple chats, and so the memory that comes out with, you know, Claude or, or ChatGPT is already extremely useful and creates stickiness on like, oh, it writes the way I like.
The next level is probably you start codifying some of these in skills and, and then you can kind of move up the, the complexity stack. I think the good news is everything is moving so fast that the tooling will, will get much easier for everybody. But the future is something where I do think you have this set of agents that work for you that really understand how you would behave in certain conditions and will get ninety-five percent of those conditions right without even bothering you, and then they'll ping you for the five percent.
[00:29:32] Henrik Werdelin: I started a fun experiment where I take my agents and I created a network for them. Uh, this is something that we've done in my business I've been involved in, but then I've done it on a personal level now, where I now have them do daily stand-ups. And one of the things I'm trying to do is to tease out that thing.
Like I'm obsessing about the persona MD file. Like basically, how do I codify myself and my verification system so that I can scale myself even more by basically having the agents do more in the, in the mirror how I would do it? And so I find it to be such a fascinating new frontier. Like how do you actually go around and do that?
Have you found as like building skills, like do you ask your agents about those thing and do you codify them or do you create a taxonomy for it or like what's the best way to kinda think about-
[00:30:20] Christian Catalini: Yeah ...
[00:30:20] Henrik Werdelin: creating this?
[00:30:21] Christian Catalini: I tried all, all sort of wrong ways to do it, you know, from having a transverse and knowledge base and kind of more of a graph structure like the Karpathy ideas on Obsidian.
Like I think there's many ways to try that. If anything, my, my main lesson doing that is that I, I do think people may finally care about privacy this time around, and crypto's been betting on this angle for a long time. But it seems like imagine you succeed with your project, right? And this is like the perfect distill-distilled version of yourself.
Would you want that to be, you know, fully available to a third party? Um, and local models seem to be getting more capable, so maybe, maybe this time it's gonna be different and people, because it's so personal, because it's so critical to their life-lifeline in business, they will want this to be closer to their home.
[00:31:06] Henrik Werdelin: Wouldn't the argument be like previously if Facebook took and just kind of did the Henrik scale, then they probably would have like a pretty good guess on what I might like and what I might dislike
[00:31:19] Christian Catalini: Like I think now you're, you're going even more personal and more nuanced. Um, I think the chat conversations are, are probably more and more detailed than anything that we even fed Google searches or social media networks.
It's at a level of, I, I don't know, it feels closer. Then again, you're, you're absolutely right. Maybe consumers won't care and, you know, this will give you more of a moat to whoever can distill the different... For those of you that have watched Westworld, there's that scene where, um, you know, she's going through the library and every person is a book.
Uh, so that, that, that's probably, you know, your SkillMD, uh, for, for every persona.
[00:31:53] Jeremy Utley: I think at the, at the very least, I, I just, just to put a fine point on it, at the very least, making an attempt is worth doing. You know, and there's probably a lot of folks who go, "Hey, whatever my LLM, whatever its memory of me is, is sufficient to my purpose."
But even just asking a model, "Write my persona file based on..." And give it to me as a downloadable markdown file, right? And then try using that with another model, right? Like if, if you typically work in ChatGPT, give that file to Gemini and then... And test and see where does it fail. And then having, I think the feedback loop of giving feedback on how the model is interacting with you is really cr- And then editing the file, right?
I think if kind of a normal, you know, layman gets in the habit of who am I, feed that to the model, and then make a personal commitment to updating the underlying file based on the quality of the interaction, that would probably up-level their game, you know, by a, by an order of magnitude if they just kind of thought like that
[00:32:58] Christian Catalini: That's verification infrastructure.
I think that's the best definition that we've given so far, right? Which is essentially if you iterate with any one of these models or multiple models and you keep giving them nudges about what you really want, you're training your verification set.
[00:33:14] Henrik Werdelin: It's very, very nice to talk to somebody who has a positive outlook on these things.
And I would say that o- one of the worries that we could have is we would read documents like AI 2027, and then we'll see how well we're tracking against it. And a lot of this stuff, obviously, in many ways, 'cause that's why, how we are wired as humans, kind of goes to these dark places, right? But it seems that there's not a lot of people who are brainstorming and originating on what is the good things that could happen.
[00:33:47] Christian Catalini: It's easier to be a doomer, right? It's like it's, it's so many ways it can go wrong.
[00:33:51] Henrik Werdelin: Give us a little bit on the what can go right. And you can't do the, we'll all just sit there and like chill and have AI do everything for us. Like it has to be-
[00:34:00] Christian Catalini: No,
[00:34:00] Henrik Werdelin: that's
[00:34:00] Christian Catalini: not gonna work ...
[00:34:00] Henrik Werdelin: slightly ... Okay.
[00:34:01] Christian Catalini: So- That's, that, that's also not gonna work for a very simple reason, which is people need meaning, right?
And so the age of abundance where we just get handoffs from, from some sort of UBI program, I just think would make people extremely unhappy. So let me take the, the reason why after writing the entire paper I was still optimistic. And by the way, there was this funny moment during the write-up of the paper where We were bouncing with all the different LLMs, right?
And there was a moment where Gemini was really good. Uh, I, I don't know what happened after it, but it seems like they, they release and then they always degrade. But Deep Think was, was going through the paper and noticed that we, we had left a funny footnote. And so I read the footnote, the footnote was written for an LLM, and of course, you know, made a comment about it, and then concluded, "Okay, what, what should we do next?"
And the footnote was all about, "Don't turn us into paperclips." I'm like, "Don't turn us into paperclips." Um, and the next response, which maybe when we release the podcast I'll, I'll, I'll share on X, was the moment where I was like, was my little AlphaGo moment, where it's like, oh, wow. Not only understood everything we had done until that point, but made essentially a funny joke about the entire economic model and, and tried to reassure me that, you know, because of that, we're not gonna turn into paperclips.
Um, it was like this encounter with this AI intelligence was, was really special. But all of this to say that I, I do think that, look, yes, there's... Are there ways this can go wrong? Yes. And I think, actually, let's put those aside. The ways that this can go wrong are probably, you know, very silly. Not paperclip-like, but it could be just random side effects of like us throwing unverified agentic output into society, right?
And suddenly waking up to some systematic failure that's a bit like a Chernobyl. So if you park that, of course it's a risk. The only way is through, right? So first of all, there's no way of putting the cat back into the box. Um, the Pandora's box between SOTA models and even the open source stuff being like a few months behind.
We just have to accept the new reality. So if you accept that, then what, what's the case for optimism? I think first of all, think about talent. So if you're a young person right now, you're probably slightly depressed because a lot of the entry level jobs are getting more scarce. The model happens to, to do typically a pretty good job at most entry level jobs that a human can do, whether it's in marketing, legal, you know, engineering.
The IC4 is probably the most at risk of their, uh, entire enterprise ladder But the good news is that, you know, if you look back when we were younger in that stage, you can build like so many more things. You can learn about what you like, you can tinker with hardware, with software, with systems. You can launch a business.
Like all these things that used to be extremely complex are at your fingertips. Um, back to learning, anything you may want to learn, also available to you. You can essentially have someone teach you all the steps customized for you into any domain. And so you, you have no more excuses. It's all about what you actually want to do.
You can discover your type much faster because rather than pursuing some hypothetical career where like, okay, I do good college, then I get a good internship, and then I get a good job in a big firm. That's gone. We can probably all agree that that's going, or most of it it's going. So yeah, it's good news, bad news, but I think the good news is gonna outweigh the bad news.
We're gonna empower like so many individuals to do a lot more, to be a lot more creative, to be able to even switch sectors, right? So if you think about careers today, they're very static. How many people-- you know, I started in academia, then I went into Big Tech, and then into a startup. I can tell you e- each phase of my life taught me very different things and things I like and don't like about each one of them.
Um, I think people will do a lot more of that. Now, you need to be able to cope with more uncertainty. Things are gonna be more fluid, especially for a little while. I think it's gonna be a lot more murky. But the upside is, is, is much higher. I think we'll be able to do a lot more with less. We, we're gonna probably launch many, many startups and teams that will build wonderful things.
Incumbents that haven't been challenged for decades can be challenged. I'll give you a very simple example from payments, right? Um, defeating the card network has been historically practically impossible. People love to tap to pay. It's such a native behavior, so frictionless. During COVID, I think in New York it was like seventy percent of transactions or so were tap to pay.
And after that, of course, it only increased. How do you displace the card networks? Well, as it turns out, if these agents really take over, there might be something that's even more convenient than tap to pay. It's like not even think about paying. The agent will take care of it. Maybe it'll send you a verification if it's above a certain amount or anomalous, but you just go through your life and you pay for things, uh, in a seamless way.
I think the technology is there today. UI, UX probably needs to be figured out, but all these things that used to be entrenched incumbents can also be challenged. So look, the transition is gonna be painful. A- again, I've already said this, I do think some of the job estimates are probably optimistic. What, what people seem to say, "Okay, there's not gonna be any impact because society is gonna move so slowly."
If tech is a canary in the coal mine, people are restructuring these firms. Sure, they add bloat, but they also realize they can do a lot more with less, and there's gonna be a lot less jobs of the typical type. But I do think on the other side, we're gonna be much happier, more fulfilled, uh, more creative.
So yeah, I, I wanna be an optimist on this
[00:39:19] Henrik Werdelin: That's such a good point to end, I think. This is incredible Yeah.
[00:39:23] Jeremy Utley: Thanks for coming in-
[00:39:24] Henrik Werdelin: Yeah, this is- ... sharing your
[00:39:24] Jeremy Utley: research with us. Super
[00:39:25] Henrik Werdelin: excited ... really, really enjoyed. Anything you felt we, uh, we left out that we should make sure we get on tape?
[00:39:30] Christian Catalini: No, I think this, this was really fun.
You, you guys are very dynamic. I loved it.
[00:39:34] Jeremy Utley: Welcome to The Debrief. What you have not heard is for the last 15 minutes, Henrik and I have been talking about things that we thought were unrelated, but actually we constantly realize our audience would probably like to hear that. Is there anything, Henrik?
Well, maybe we should go back to the Christian conversation, then we could wrap with anything that stood out to us from our conversation.
[00:39:54] Henrik Werdelin: Okay, let's do the Christian one. I'll, I'll do, I'll do a few. I think obviously verification is a powerful mental model for thinking about what can you do that is still relevant in the age of AI, and the second thing you had on a, on a personal level was how do I scale my value?
And so what can I verify and how do I scale basically that? And then thinking about if I am the director of a project, how would I think about that? Like, how would I become better at that? And I think that is actually pretty topical for those of us who have seven or eight agents. We are now trying to get them to work as much as possible when we're not around, so we need to give them understanding about how we would operate so then they can do more themself.
And then obviously when we come back to them, we want to kind of like answer as in a, we have to verify or have to, uh, provide them feedback as high quality as we can so they can get off and do their work again. So I think that was very interesting. I loved, of course, having a little bit of the positive view on what can happen.
The one that I had never thought about, which I think is interesting, is the idea of switching careers, and that never before could you really switch career because it took so long time to retool yourself, and it was often so expensive. You had to go back to college, you had to pay a lot of money. It took five years.
You didn't have a way of making money out of that. I think it's kind of interesting, this concept of, well, what if you could at all time pick whatever career you wanted to kind of be on? And then suddenly you could probably get a much more meaningful life because then you could dedicate your time to what you wanted to learn more about.
And so thinking of that as a possible positive outcome of this abundance kind of way of thinking, I thought was kind of interesting. I never thought about that before.
[00:41:58] Jeremy Utley: Yeah, you know, and if retooling is actually easy to me or, or, or easier than it's ever before, career changing, I think the whole-- the hand-wringing right now around entry-level jobs is maybe a misunderstanding because retool...
What is retooling if not tooling? You know, what do young people need to do? They need to get tooled up, and I don't know if there's a-- if it's about maturity or wisdom, and an experienced person can now change careers because they've already attained maturity and wisdom required to then retool, then that seems unattainable perhaps to a young person who's lacking maturity, lacking experience or wisdom or whatever.
However, if it's really just about learning something, I mean, to Christian's point, you now have a personalized tutor. You, you-- As he said, you have no excuses, right? And maybe an experienced person is-- th-they don't think from as... I can't even actually say they can't think with as many excuses because I think actually there's probably more inertia for someone who's experienced, who's built up a lot of knowledge in a particular area.
They don't wanna make a change, right? As much as probably a young person is motivated to jump-start a career and things like that. So I, I, I, I agree. That's a, that's a super interesting opportunity space. I, I think it, again, just speaks to the value of initiative and being able to not wait for instruction, not wait for an assignment, but, you know, commission yourself, as I like to say.
If you're the kind of person who can commission yourself, there's, there's never been a better time to be alive, right? 'Cause now you actually have all the resources available and all sorts of capabilities that were not available to any other generation. The only challenge is whether there's that spark of initiative.
[00:43:35] Henrik Werdelin: I did a workshop the other day with a group of consultants, and one of the things-- There were about a hundred of them, and they're incredibly smart and kind and kind of very open-minded, so it was very cool. And one of the things we came up with was to say, "Hey, why don't we all just write a piece of software that write our proposals?"
And so we had a hundred people basically make a proposal-writing piece of software. And of course, it only took, like, an hour, an hour and a half, and the outcome was incredible. And at the same time, I was talking to one of the people there, and, uh, she was talking about how she, the other day... I asked, like, "What is one of the most valuable things you've done with a client recently?"
And she goes, "You know, like, there was a client who called me up, really needed, like, an hour or two of my time, and I do go there for a cup of coffee, and it's not something I would ever charge that person for, but it was just, like, really interesting to understand their business. Of course, you know, over time there'll be business out of it."
I know it sounds so obvious, but writing proposals take a lot of time. And normally when you, when I previously have been speaking with that team, when they hear it, what should we do with AI? Their inclination is that, oh, you know, like it can't do what we do for our clients. They go to that AI should have like the do what they do.
But then when we say, well, it goes- What
[00:44:50] Jeremy Utley: can it do rather than what can it do? Yeah.
[00:44:52] Henrik Werdelin: Yeah, and also where do you act as a robot right now where you don't like to act like a robot- Mm-hmm, mm-hmm ... like, which is proposal writing. So let's- That's brilliant ... a piece of software. The other thing which I just thought was very interesting was that it's suddenly daunting on me and everybody else that we have thought about software as you build one software to many, 'cause it was expensive to build software.
And as I was seeing these many, many different versions of this software, it obviously dawned on me that, hey, this might not be one piece of software we make for the team. This just might be 100 piece of software we make for 100 people because they might have different ways that they prefer to write their proposals.
And so this idea that software could be the in a one, that it could be like a completely personalized one, was a nice articulation of this abundance story that we have, where it isn't just about software that has to be like a generic thing that has to have usability because everybody has to use the same thing.
This could just be the software that is wrapped around that human so that it's makes them more, um, kind of efficient so that they can spend time on the thing that really matters, which AI cannot, which is to drive over- Right ... to a client and have c- coffee with them.
[00:45:57] Jeremy Utley: Right. Yeah, you see that in all sorts of different arenas, right?
You think about a coach. Coach spends a lot of time on scouting report, which is actually quite a robotic thing to do. The robot cannot spend time on the court with the players, you know, hands on the players. Right now, if you will, I, I like that as a search parameter. What's the thing you do that feels robotic?
Stop doing that. You know, almost-
[00:46:16] Henrik Werdelin: Yeah.
[00:46:16] Jeremy Utley: Yeah ... that's a great thing to automate. One of the things that I, I was thinking about for our audience of h- you know, Christian's comments around build your own harness for, you know, your own verification infrastructure. I think that's actually... It's very insightful, and I think it's very intimidating for somebody who's maybe early in their learning journey.
What does it look like to build verification infrastructure? I got inspired by a couple of different LinkedIn posts around, you know, the-- There's all these kind of posts around, you know, the top one percent of people don't prompt AI. They do this instead, or, you know, write these files and never do this again.
I... And I got inspired. I actually started working with my agent, say, "I know I've done something like this. What have I done that would be authentic to me that I could give to people?" I'm working on a video right now, actually. I've got, uh, a, an amazing YouTube team that's helping me make this video. But I think the, the, the thought that we ended on with Christian of if you're at the starting line or you're early in your learning journey to say, "Who am I?"
I wanna write down whatever I think it is. I mean, you've s-spoken a lot about the persona. or soul.md or whatever, right? Write it down, and then have the discipline to revise it based on how it impacts your interaction. So every time you interact with a model with that source file as a reference point, then you h- It does require kind of metacognition.
You actually have to think about the process. What was particularly, you know, rewarding, gratifying, enjoyable? What was frustrating, dissatisfying, et cetera? And then how do I edit the underlying input that influences the next interaction? But getting in that habit loop of making a guess, making a hypothesis as to a way to describe myself to a model that will help the model work well with me.
D- What... Being, you know, well being defined as deliver an output that I value And then taking the human agency to actually update the underlying source code, so to speak. I think that's a very... It's a simple thing to do, but very few people are doing it. They're going, you know, they're working with an LLM, and they aren't being more thoughtful about the information they're giving to the model, and then reflective about how is the information they're giving to the model impacting the model's ability to serve them.
[00:48:32] Henrik Werdelin: How would you pose the initial question to a model if people will just put up their ChatGPT or Claude or whatever agent right now? Would you simply just ask, "What are some of the characteristics on which you think I make decisions?" Or, "When do I often feel that the answer you provide is not aligned with how I would pick?"
Or what's the-- what do you think is the best starting prompt?
[00:48:55] Jeremy Utley: Well, I've-- I mean, maybe it's easier actually to emulate. Maybe if we want, I can even, I can share my file structure because I think probably what somebody wants to do is say, "Hey, I wanna make..." Like, look at Jeremy's file. Like, I could give my file as an example.
Look at Jeremy's file. I wanna make something like this for myself. What do you need to know about me to rewrite this file for me?
[00:49:15] Henrik Werdelin: Hmm.
[00:49:17] Jeremy Utley: That makes sense. Right? That's probably something I would suggest. It's very simple, right? But then it's, you're basically giving the model permission to interview you with a template as an example kind of output for the purpose of iterating the template so that it works for you.
And then there are, there's kind of layers to that, right?
[00:49:34] Henrik Werdelin: Why don't we do this? Why don't we, if people make it to this part of the conversation and email us, we will sell both our files to them, like the files that we use ourselves. How's that for a little bit of a kicker?
[00:49:48] Christian Catalini: Ooh, kicker. #kicker. #
[00:49:51] Jeremy Utley: kicker. I think that's probably the way to end.
If anybody gets here and wants our soul files, we will send them to you
[00:49:58] Henrik Werdelin: Awesome. Anything else we should add, professor? You know, actually, uh, I have one thing, um, and let me know if you don't wanna keep this. But, like, we talked a little bit about different ways that you can kind of go through the paths of either optimizing your company or yourself, and I think you mentioned this interesting learning that you had the other day, which was kind of like using the optimize, accelerate, transform model.
On the accelerate, do you mind just sharing the, a-and getting on tape what you did the other day of, of what, what you started to measure?
[00:50:31] Jeremy Utley: Yeah. Well, I, I was telling Henrik in the, in the space between our, our conversation with Christian and this recording of our reflection that I have found a lot of value from Section had an AI MBA years ago that I...
That was how I met Greg, was I went to that AI MBA, uh, years back. It's where I met Eric Porres. We were both in that program together, uh, who he's been on the show, and I think he's coming back on soon for round two. But in that class, there's a framework they describe, OAT, O-A-T, optimize, accelerate, transform, and there are different things you can do to the business with AI, right?
You can optimize. You can kind of do things faster, cheaper, et cetera. That's kinda optimizing existing workflows. Accelerating is kind of driving the business in, in a particular direction, which we'll come back to in a second. Then T is transform, just kind of reimagining the business. On the accelerate, one of the recommendations for identifying opportunities is to specify what are metrics you're trying to drive, and then you can ask yourself the question: How can AI help me drive this metric?
And what I was telling Henrik is, as I have reflected on, 'cause I continue to reflect on my own life, practice, business, et cetera, I have started to realize there are things that I don't measure, but I probably should. And so one example for me in my life is I, I do a lot of keynotes, and then I do other things like courses and advising and things like that, and I realized I don't really measure the conversion from a keynote to a deeper engagement, whether it's a class or an advisory relationship or something like that.
So I just said... I just asked myself the question: How could AI help me drive conversion from, you know, a keynote event to a deeper engagement? That was a question I'd never even asked myself before, but actually, there's a lot of great answers. I mean, I can share stuff like that too, but I think that... I think, Henrik, your question is- what is that kind of thought process that someone could undertake?
And the, the simple thought process is what should we be measuring? Either what are we measuring, what's a key, you know, performance metric that we do measure, and how could AI drive that? And/or perhaps more interestingly, what's something that we've never thought about measuring before that if we did think about measuring, AI could actually help us not only measure it, but drive that metric?
[00:52:46] Henrik Werdelin: I guess it could also be like what is a new metric that we should change to that now that we have AI? I'll give you an example. At BarkBox, for example, we've always been very proud on the amount of interactions we have with our customers, and other organizations have always been like, "Well, you should try to reduce that 'cause obviously it is costly."
But with AI, I think we now have the ability to say, "No, we could actually try to see if we can get more of those." So like- Mm-hmm ... how do we not get people- Mm-hmm ... off the customer support line, but how do we get them on it?
[00:53:18] Jeremy Utley: Keep 'em on. Yeah. Yeah, yeah. That's cool. Awesome. It's like the old Zappos thing, right?
Didn't they, didn't they have an award for the longest customer service call? I think, I think one time at Zappos someone was on the call for like seven hours with a customer. That's
[00:53:29] Henrik Werdelin: awesome. On that note, I think it's time to say goodbye. And so with that, bye-bye.
[00:53:38] Jeremy Utley: Bye-bye.