In this episode, we talk with Phoebe Yao, founder and CEO of Pareto.ai, about fair and ethical data labelling platform aimed at advancing support for AI researchers. The episode addresses data labelers' challenges and stresses the need for higher standards and worker empowerment. We talk about the critical role of domain experts in AI training, the impact of ethical practices on data quality, and the broader implications for the future of work in AI. This episode offers valuable insights into how Pareto.ai supports workers and ensures quality, highlighting often-overlooked ethical concerns in the AI industry. Disclosure: Jeremy Utley is an angel investor in Pareto.ai
Website: Pareto: Premium AI & LLM Training Data Labeled by Elite Teams
00:00 Introduction to Pareto and Ethical Data Labelling
00:31 Spotlight on Pareto's Mission
01:53 Understanding Ethical Data Labelling
02:16 Challenges in the Data Labelling Industry
06:00 Impact of Ethical Practices on AI Training
08:03 The Role of Domain Experts in AI Training
13:28 Adapting to AI's Disruption in the Labor Market
20:56 Ethical Consumption of AI Products
25:44 Practical Uses of AI in Business
29:18 Closing Thoughts and Future of Ethical AI
Disclosure: Jeremy is an investor in Phoebe Yao’s company.
📜 Read the transcript for this episode: Transcript of A sobering talk about the need for ethical data for AI models with Phoebe Yao |
[00:00:00] Phoebe Yao: I'm Phoebe, the founder and CEO of Pareto. ai, and we're building a fair and ethical data labeling platform to help AI researchers collect the human data that they need to train their AI models.
[00:00:14] Jeremy Utley: Phoebe, welcome to the show. Thanks so much for joining us today
[00:00:18] Phoebe Yao: Thanks so much for having me, Jeremy and Henrik.
[00:00:21] Jeremy Utley: Now, as I've been kind of following your journey on LinkedIn and through your updates and things like that it's really exciting what you're doing.
And we're happy to shine a spotlight on it.
Tell, tell us just, you know, two sentences about Pareto.
[00:00:35] Phoebe Yao: Yeah Pareto is building a fair and ethical human data collection platform. Hopefully the first actual platform that's. Ethical working standards for its data workers. Right now we're working with crowd work researchers at Stanford and UPenn to build incentive systems that can give workers more agency on how they work on projects and as a result, motivate them to deliver more creative higher quality results. Our primary customers are AI research labs, really, like the top research teams that are building out LLM models and thinking through , how do we train ever more intelligent and aligned AI models.
We're really working with many of them right now to, build out these expert teams for their data labeling projects
[00:01:36] Jeremy Utley: for somebody who's just totally new to the space. Why is ethical such a foreground in consideration? What's the kind of give us the lay of the land of what's the status quo and why is ethical a new?
Feature so to speak that, you're entering the market with
[00:01:53] Phoebe Yao: have you guys heard of gail AI.
[00:01:57] Jeremy Utley: Yeah. Yeah.
[00:01:58] Phoebe Yao: Yeah. So scale AI has a subsidiary called remote tasks. That is basically their data labeling platform where they recruit workers and they name it a different name in a way because It has such a bad reputation among the worker community for all sorts of things.
One big example of that came up literally a few weeks ago, while we were in Nairobi, we discovered that remote tasks had suddenly suspended access for workers across Kenya, Rwanda, South Africa, literally countries and thousands of data workers that depend on. This work for their livelihood and their families depend on this work.
They just suddenly lost access to the remote tasks interface and it's been widely denounced as a really like move on skills front for obvious reason. And we like coincidentally got to meet many of these workers. who are really veterans in the labeling field a few weeks ago at our first in person worker meetup in Nairobi.
[00:03:22] Jeremy Utley: So what's that like to meet these folks for the first time? And what are you learning about the impact of your mission on their lives?
[00:03:30] Phoebe Yao: It's crazy how they tell us that they've never actually met people from these platforms in person and It's crazy to them to see us in person, to see the people behind these avatars on screens and realize that we're genuine and we're real and we're here to create systems that can support them and accessing better economic opportunities.
And like, we've heard from some workers that, they drove eight hours overnight to join this. Workshop with our team just to make sure that, we're legit and we're not here to scam them or anything, cause that's what they're used to. They're used to the typical labeling platforms having all this power and not actually supporting them in giving feedback on their work, removing canceling payouts because of. Quote unquote, low quality tasks without explanation when many times this is due to gaps and communication gaps and understanding past guidelines. But what ends up happening is these workers spent hours of their time. Working on projects, and then because they're not. Supported and they're.
Questions or giving feedback or really because there's no real channel communication their work is seen as erroneous, oftentimes, and then rejected and then they don't get, paid for that work. And that's very, very common across platforms, across the common data collection platform requesters of that, of the labeling projects end up having unprecedented power over the collaboration, while the contractors, the workers, have really no say in whether their work is approved or not, even if they believe they've followed the guidelines.
So it's just a huge gap in Working standards that exist right now in the labeling as well as wider gig work industry.
[00:05:44] Henrik Werdelin: So just to, so I understand maybe to simplify it is the labeling is really when, in order to train these big models, the models need to understand what something is.
And so is this as, an example that you show a picture of a cat and then a human people tells. The, basically the LMS of this is a cat.
[00:06:06] Phoebe Yao: Yes so we believe that labeling has fundamentally changed maybe 10 years ago, you would have these really simple tasks, like label hat or a dog picture.
And, put a bounding box around the cat and say that it is a cat. But the models of today are way more mature and the tasks that they're being trained to do are also a lot more complicated. So like one example is that perhaps you want a model to identify when it is making up a fact And we call that like model , hallucination, like hallucination training involves giving the model feedback when it's given you a fact that isn't true and through like thousands and thousands of different model responses and feedback to that model, you're able to train a model to, make up stuff less on, on maybe a particular area of I guess, knowledge work.
[00:07:09] Henrik Werdelin: We talked to uh, either Malik on a previous episode and he was talking about how gullible LM models are. Obviously, ethics and morality seems to be, such a core of what you're pioneering. To what extent is the morality and the ethics not just considered how we treat workers that are helping training these models?
But increasingly These workers will also be, in many ways, the people who will help create the underlying data on what the LM models will believe to be ethical. Is that true?
[00:07:46] Phoebe Yao: Totally. You want to have a diverse and really representative group of people training models. And you really can't.
model knowledge or expertise, unless you have people who have spent years of their lives studying that domain of knowledge and who are qualified experts. And so a huge part of the work that we're supporting nowadays is actually finding the most talented domain experts or subject matter experts.
For instance, software engineers or doctors or lawyers and helping them actually first be open to supporting AI model training, because oftentimes they see this as like lower quality work or, all these, you know, for the historic, ethical issues that have played the labeling industry.
And so first we have to get them to believe that this is like good work, that it's well paying, that it's not a scam, that they will be compensated fairly. And then actually organize them, coordinate them within large teams, sometimes hundreds of people to be creative and to be supported in creating the most unique and diverse tasks.
So there's a lot of different ways that models can be trained on.
[00:09:14] Jeremy Utley: The impact to your customer? So take the anthropics of the world, the open a eyes of the world. The folks who are buying or sponsoring this data labeling process, what is what have they seen to be the differential impact of treating workers differently working with you.
[00:09:39] Phoebe Yao: A very obvious difference between low effort work and high effort work and you have to really dig into the data to know that. And not only dig into the data, but ask your labelers how they're doing the work. For instance, we discovered recently that in 1 project, a labeler was submitting submissions.
Of this task every millisecond, every few milliseconds. And we were like, that's impossible. Why are they how are they submitting tasks so quickly? And we. Immediately wanted to suspend their account. But we held back because we realized, hey, this. Might not actually be our area of expertise. We're not doing this task every day.
Let's, try to figure out what's going on. So we get in touch with this worker and he explains that he's actually been experiencing the serious lag in the interface and he creates dozens of tabs and submits across dozens of tabs in order to get past this this lag in the model's response to their submission.
And we realized that, Oh,, there's actually a problem in the interface itself and not a problem in the worker's integrity, because when we actually look at the data and we dig in his data looks great, it's just this initial misunderstanding that could have led us to banning a really great worker and then losing that trust forever.
So it's one example of how it's actually really hard to be ethical because oftentimes we think on the side of I research side. We think we know what's right. But when it comes to coordinating. And organizing hundreds of people on a single and very specific task, those people are oftentimes better experts at doing that work than we are. And we have to respect that expertise in order to actually unlock the most creative and highest quality results for training AI models.
[00:11:56] Henrik Werdelin: I listened to this, I think New York times podcast a while ago that really resonated or really hit me and it was This woman that was working, I think, for a, in a factory that I think Ford had planned, and they were moving all the machinery , offshore, I think, to Mexico.
And so, her new Mexican colleagues had come up to learn how to operate these machines. And she was this incredible woman that was talking with immense amount of integrity about how much she liked, her colleagues and how she realized that basically she was training the people that was, it's going to take her job.
But for all sorts of reasons, uh, she felt that she really wanted to give her best effort doing that. And it was very remarkable. Now, when we talk. And it's training models. Um, obviously similar type of story kind of like comes to the forefront of the mind. And so where are you in thinking about using people to train models that could do stuff that the people that have trained them, uh, were doing?
[00:13:06] Phoebe Yao: It's such a great and important question. So when we started Pareto, we built the company to upscale work at home moms and give them work opportunities. And we actually didn't pivot into labeling for AI training until eight months ago. And the reason we didn't pivot was because we didn't believe that the work was Meaningful for the workers at the time.
It wasn't. It was as we mentioned all these really simple tasks that wasn't actually building skill sets and it didn't actually require a deep level of thought and effort. And we wanted to build a system to upskill people and help them reach higher and higher pay and complexity work.
Um, so the reason why we decided to pivot eight months ago was really because we got in touch with a group of expert workers. Who were earning full time wage from the AI training work and not because they were doing any specific one task, but because they had learned to be really adaptable and pivot super quickly using their really generalist knowledge and understanding of the world to go from different types of complex tasks, uh, Across different projects for AI training.
. So first off AI training can provide full time income, but at the same time, it's demanding a totally new skill set of being able to work really adaptably across different tasks.
[00:15:03] Jeremy Utley: You know, you're raising a great point, right? These part, just maybe to just read back to you what you're saying, because I, because it's a really nuanced point that I hadn't considered because of the general. nature of these models and the broad kinds of things they can do.
The data labeling requirements are much broader too. And the opportunities for complex tasks and nuanced tasks and adaptation is far greater. And I had, I never really thought about it, but because the models can do so many things, they actually require people to train them who can do so many things. And to be able to shift between radically, unlike, you know, say a assembly line , in a factory, like Henrik just mentioned, where a worker may be doing the same thing all the time here.
And I've kind of pictured label you know, training data as a similar kind of repetitive task. What you're saying is it's actually not repetitive. It's hugely varied. It's perhaps as varied as the functions that the model can perform.
[00:16:10] Phoebe Yao: Yes. Exactly. And don't want to claim that AI models are not taking away current set of jobs, because it is, I think, inevitable that as technology advances, it will completely disrupt that existing labor market.
And we've seen that time and time again with. Every innovation that has come to society. So I is unique in that the breath of disruption is going to be huge. But at the same time, the need to build those models into each unique domain. To support experts in those domains and take on work in those domains that will demand. So much out of not just domain experts, but folks who have any really different combinations of skill sets that it's about creating systems that actually enable people to use their unique skill sets and apply it across different types of projects really quickly.
Which is really fundamentally the goal of the systems we're building.
[00:17:32] Henrik Werdelin: What are some of the other, I mean like this is, I think for somebody like Jeremy and I who are generally very excitable, but also very excited about AI and how fast it's moving and all the stuff you can use it for. And most of our conversation is is the hoo ha of you know, like the, the coming of this new technology.
Obviously in the underbelly of the beast, there are conversations like these where we have to figure out like how people treat it as we develop the What do you think of some of the other themes of ethics that are not Discussed as much as they should in the AI space.
[00:18:12] Phoebe Yao: This is a hard one.
[00:18:14] Henrik Werdelin: You can pass on it too.
[00:18:16] Phoebe Yao: Want to go back to this point around work and there is so much fear from workers when I talk to them about their jobs being replaced.
[00:18:28] Jeremy Utley: You mean, you mean from data labeling workers that they have fear that their job will be displaced?
[00:18:33] Phoebe Yao: Well, really any workers, but including data workers. So like traditional jobs. Are going to evolve. But we get this fear all the time from data workers from experts and traditional domains who we're hoping to recruit for data projects. And the question is, like, why should I help train models when it's just It's going to replace my work.
And I think the fundamental and difficult truth is that no matter what we do, the genie's out the bottle and AI is here and it's being developed no matter if you participate or not. And this level of disruption when ever we have new technology is unavoidable. So I believe that. It is much better to face that wave and to participate in its making such that you can prepare yourself for what's to come and also make sure that technology is aligned with your values and principles then to try to avoid it and to avoid that inevitability. So I think that the perspective that we're coming from to the labeling industry is , we want to help people face that crazy, tumultuous transition period that AI is bringing with grace and help them also learn and build new skill sets that will prepare them to face the future tumultuous. Labor market. Um, and those skills are adaptability and the ability to learn new tasks really quickly. And to also have the ability to switch gears really quickly because as the technology is being built quickly. Old problems will be solved quickly and new problems will arise. Every week and every month. So you have to be able to be prepared to confront that new reality for the labor market.
[00:20:40] Jeremy Utley: Phoebe, I'd love to hear your thoughts On the consumption side, what does it look like for those of us who are consumers of AI products, technologies? What does it look like to be an ethical consumer or a conscientious consumer? We've seen, with ESG movement, there's been all sorts of ways to participate in ways to be mindful.
Do you anticipate? Mechanisms like those that have emerged in ESG to emerge in AI. And what does it look like to be an early proponent of ethical practices in AI? Just practically from a consumer perspective,
[00:21:16] Phoebe Yao: I believe people are going to be a lot more thoughtful about how the models that they're using are being built.
And that includes , uh, where the researchers are sourcing the data. We already have, you know, fair work practices for many existing industries, but AI has not quite, it's still so nascent. So labor standards are, really low and are non existent. And our goal here is to set new standards for the fairness and ethics of data labeling for AI training, such that workers that participate in AI training are respected and well compensated and have really high quality working environments that allow them to grow and upskill themselves. And I also want to add that right now there are some mechanisms by which bad actors are being called out. For instance, what I shared about the remote tasks incident in Africa, that was called out by a journalist, Karen Howe, at the time. Wall Street Journal.
And she's been doing incredible work and and investigating fairness and ethics and the AI training labeling industry. And I think because of her, a lot more people are becoming more aware of, uh, the challenges in this space. And we're actually talking to her later this week to try to.
Shine light on this other side of the labeling industry which is let's make this better. , I hope that, like, more and more people, especially AI researchers and anyone who consumes AI products will become a lot more thoughtful about how that model has been developed.
[00:23:25] Henrik Werdelin: In many ways, it's hard not to imagine that the work that you're doing with people overseas is in many way a process that we'll have to do domestically at one point also as the capabilities of AI becomes better and Anybody from university professors to people who start companies you know, their work can be done with AI.
And so it seems like the morality and the ethics that you're applying to this is could almost be a precursor to a conversation that we'll end up having., here in the U S soon.
[00:24:02] Phoebe Yao: I think we're already having it in the U S It's just super nascent. So we're already hiring many U.
S. based data workers, everyone from law students at Stanford to Harvard business students to Chicago medicine students really folks who are, like, incredibly talented in the U. S. Who were bringing in to help train models. And so I'd say that like labeling and AI training and data collection has already come to the U S labor market.
These jobs are already very actively recruiting for talent. But it's not something that I think people really regard as work that they want to be doing. Many, mostly because of the current stereotypes around labeling work. Um, so as more and more people get invited to, to do this, I think we're going to start seeing a lot more discussions and conversations about ethics. This of labeling in the U. S. And certainly hopefully greater and our team can be at the forefront of leading those conversations.
[00:25:28] Jeremy Utley: Brilliant. You're amazing. Thank you for joining us late at night .
in Asia. I hope you have a wonderful rest of your trip.
[00:25:35] Phoebe Yao: Yeah. Thank you guys for having me. Let me know if there's anything else that would be helpful. , I really appreciate, , Y'all covering the story, covering our, our journey.
[00:25:49] Jeremy Utley: Be good. Have a great evening
[00:25:50] Phoebe Yao: Henryk. Bye
[00:25:52] Jeremy Utley: She's awesome.
[00:25:54] Henrik Werdelin: I mean, like, obviously, like an interesting conversation, right? Because it does seem to be, it's easier to talk about people who are getting displaced. In Africa, but it's the same problem that a lot of people are gonna face. Right. How do you deal with people interfacing with ai?
, it is this conversation that started with gig workers and Uber, right. And TaskRabbits. Mm-Hmm. . Mm-Hmm. . That's just kind of like moving itself up the stack.
[00:26:20] Jeremy Utley: No, totally. That's actually what came to my mind is like a metaphor is just imagine an Uber driver. I mean, there's something about when it's far away, it's easy to objectify and, or not even really think about it's abstract, right?
But you imagine riding with an Uber driver and they do a good job and they take you to your destination. And then after they drop you off. They don't get paid because Uber says they didn't do it right or something, you know, it's to me I would because there's a human face to that experience. I go.
There's no you gotta pay She dropped me off. He dropped me off, right? but I mean just to hear about data labelers who are And so, you know, spending hours working on tasks and then getting no feedback and just not getting paid, you know, it's because of the kind of faceless nature of it. And because of the way perhaps that we have conceived of data labeling activity and work that's, it's there's no human cost to that.
So it was very sobering to hear about.
[00:27:17] Henrik Werdelin: Yeah. And I mean, like I have this story, which is kind of like maybe a little bit off topic, but it's. It reminded me of it. When we started Bark we had a warehouse out in the Brooklyn Navy Yachts, and we had a bunch of people helping us pack our boxes. And most of us had never been out to see the operation.
And so, we were, my co founder and I were talking one day and we were like, we should go out there and on a Wednesday and, Offer pizza and chicken wings and just hang out and meet like, cause these folks are kind of part of our crew. And so we went out there, obviously ignorant as I was, I didn't realize that most of them were Spanish speaking and Spanish being only, and I don't speak Spanish.
And so the first, there was this whole, You know, very well intended, but kind of slightly awkward moment of just like standing there, a bunch of people looking at each other, but what was nice about it, we then offered all the people that was working on the assembly line, we were offering them bar boxes.
And I think one of the things that happened, I'm not sure this is exactly how it played out, but a few months later, somebody on the assembly line pointed out that the vacuum in some of the treats had gone bad. And so air kind of coming in. If you really look carefully, some of these products were spoiling.
Now a spoiled product is super dangerous for our dog. . I'm pretty sure that the reason why we were fortunate enough that we ended up not sending all those products out was because somebody there told us now a good people to start with.
So probably not that, but I also think it might be because that they felt a little bit invested in kind of like the overall success of the company, because, They met us and, you know, we've shared like our love for dogs in the weird way you can when you speak two different languages. And I think when Phoebe was talking about AI labelers and you suddenly realize that here are people who are basically the jury.
of something being right or wrong. Like, is this fact, is this a fact, yes or no? And so somebody sits and say, yes, I believe this to be a fact. . You're basically taking a very important part of what we are all going to be using for a long time.
And that kind of whole data layer is being made by people that we might not be treating very well,
[00:29:44] Jeremy Utley: uh,
[00:29:44] Henrik Werdelin: And maybe then in return, they might not care, or you would self select yourself into people, desperate or whatever it is that kind of makes them not , choose in the same way as is ethically right.
[00:29:58] Jeremy Utley: Yeah, I loved her passion for building skill sets and , helping folks reach higher pay. I think in an environment where the assumption is that it's a race to the bottom and it's how can we compress costs.
And inevitably that, that involves compressing pay. I think for someone like Phoebe to be saying, no, how do we give someone more and more complex work? How do we build their skill sets? She's honoring the expertise of folks that I think largely gets overlooked. And I wish her enormous success in that.
And , my hope is that we live in a world. of ethical AI, where everyone who contributes to the knowledge that we're all drawing from is being treated ethically. And it's, it almost, you think it goes without saying, but it doesn't. And thanks to folks like Phoebe and the Prado team, we have agents on the front lines seeking to make sure that's a reality.
So hats off to Phoebe and the team.
[00:30:52] Henrik Werdelin: And so I think a little bit of a sobering episode from Beyond the Prompt. Thank you so much for listening and obviously as always, if you enjoyed the conversation and you want to hear more, please go into your favorite podcast platform and like and subscribe. And if somebody, is interesting AI and how it can be used in the work environment to make it better, then, uh, yeah, please send them a note.