Revolutionizing AI Data Labeling at Sapien with Ahmed Rashad

Sheikh Shuvo: Hi, everyone. Welcome to Humans of AI, where we meet the people building the technology that's changing the world. I'm Sheikh. Today's guest is Ahmed Rashad, CEO of Sapien, where he focuses on innovations in how training data gets labeled. Ahmed, thank you so much for joining us. Thank you, Sheikh. Uh, so Ahmed, the very first question I ask all of my guests is, how would you describe your work to a five-year-old?

Ahmed Rashad: Yeah. And, uh, so my work actually starts with the AI. So, if you think about AI, AI is able to figure out what's going on in the world or do something that we currently do as humans and for it to be able to do that, a human needs to go in and tell AI over and over again that, "Hey, this is a person, this is a car. This is the right way of doing this. This is not the right way of doing this." And you need to do that over and over again. And this is human feedback into the data that goes into AI. Yeah. So we specialize in the human feedback that goes into AI and specifically around the high-quality, high-complexity feedback.

Sheikh Shuvo: Sounds great. Nice and clear. I think my kids could follow that as well. Cool. Well, you know, you've had such an interesting... Um, I think you started as a drilling engineer, then tackled different parts of the tech world. Tell us just about your career journey overall. And, uh, you know, what were some of the inflection points along the way that led to where you are now?

Ahmed Rashad: Absolutely. So, uh, actually even, even from before that, I always had a knack for solving problems. I generally don't care about the context or environment, I just want to solve interesting problems, right? And I want to do it working with interesting people. So, um, I started my career in mechanical engineering by training and started my career as an offshore oil driller, uh, mostly in the Mediterranean and the North Sea.

Beautiful, beautiful scenery in the North Sea. You have to check it out. Uh, it's, uh, quite fun. And we had this problem where we were doing this specialized job where we needed a lot of equipment and, uh, we needed a lot of new tools and spare parts and so on and so forth.

And, uh, I didn't have enough space. So I had to figure out how to put everything in place and how to make it fit. And it wasn't working. So I figured out there could be a solution. I could build software that actually tells me what I needed and when I needed it because I also had a lot of stuff that I didn't need.

So, uh, I went to learn how to code, built that solution and, uh, it solved our problems, actually worked. And, um, that's when I first transitioned into tech. I was telling someone randomly at a party, like I was on offshore, you know, and someone randomly at a party. And he's probably my age now.

So I was like, "Hey kid, what do you do?" And I started telling him and it was surprising because I didn't know this was my side project. I had no clue. He was like, "This is interesting. Can you show me?" And I showed him and he's like, "Do you want to do this full time?" And I was like, "What is this? I actually don't know what this is." It's like just doing what you're exactly...

Sheikh Shuvo: Oh, that's so cool.

Ahmed Rashad: Yeah, that's how I transitioned to tech. He was a VP at Oracle. He was transitioning off to start his own startup, and I joined them for a few years. Then I joined Oracle, worked on product there as well. And then I started feeling that itch again, the itch where I love what I'm doing, but there was this itch where I felt my hands were tied because I couldn't solve all my clients' problems. I was limited by the tools I had.

I specialized in operations at the time, and in operations, you need tools, technology, humans, organization, and strategy to bind it all together. I was very tied to the technology part, actually a subset of it. So I thought, who are the people who actually are able to look at this end-to-end? Well, first of all, you have CEOs, COOs, and so on. And the other option is actually consulting at big consulting companies. I thought, okay, so who are the big consulting companies? I started doing more research and realized McKinsey was great in this area, covering everything in extensive detail.

So I decided to join McKinsey. I took a little break, went to grad school, went to MIT, studied operations research and supply chain, and then transitioned to McKinsey. Four years later, I had tried almost every industry and every use case I could get my hands on. It was a unique opportunity; no one would normally trust you with no prior knowledge of the industry or customer, but they trusted me to figure it out. This was beautiful in so many ways, and I wanted to get as much of that experience as possible.

This led me to a subset of McKinsey called Recovery and Transformation, which is about turning around companies not doing well. We wouldn't get paid until the results were in the bank. It was a very different mode of engagement. All of this, by the way, was random. I was talking to people at Amazon, and they said, "We have this business that isn't doing well. Do you want to come do this RTX, SMM, whatever stuff, and try to turn it around?" I said, "Sure, let's do it." I did that with one business and then with another.

And then, as I was doing all of this, I got contacted by Scale AI. They were early on, at series A, and they had trouble scaling. The thesis was solid, but they needed someone who could scale it. I joined the fantastic team, frankly, one of the inflection points of my career, where I was in entirely new territory, doing things that hadn't been tried before.

There was no roadmap, no guidance, nothing because no one had done it before and we were just figuring it out. And we did, and fast forward, the company is now valued at seven and a half billion dollars. The amount of work done, for example, when I joined, we were doing about 14,000 hours of work a week. Well, by the time I left, we were north of two and a half million hours of work a week. It's incredible.

I recruited a million people to do data labeling over my first two years. It's just ridiculous numbers. Right. I wouldn't believe it myself if I weren't there.

Sheikh Shuvo: There.

Ahmed Rashad: Yeah. And then, I met my co-founder Trevor and started talking, and that's how we launched this debut.

Sheikh Shuvo: Along that journey, was there a specific...?

Ahmed Rashad: Yeah. So, it's incredible because it was something that was building up and then there was the aha moment. It was building up while I was at Scale. I always reflected back on what we did and how we did it. At the time, it was the best option, the best outcome, the best we could have done knowing what we knew. But I always reflected, thinking, "Okay, if I do this again, this is how I'm going to do it." Frankly, I didn't go anywhere with it until I started talking to Trevor. He started asking all these "Why?" questions. He kind of knew the answer; he just wanted to see how it's going. We started having this conversation, and then it hit me. It's like, "Oh, it's actually incredible. There's this huge chunk of the market that's completely unaddressed."

I knew it in the back of my head, but it never really crystallized in front of me. It was like, "Oh, this is it. The market's actually way bigger than I thought, or that I chose to believe it was." I was living in this area of what the market is for Scale, but there's this whole slew of other markets that's probably bigger than what Scale and all the other incumbents are working on.

"Let's go do it. Oh, and by the way, this time, that was the aha moment." And then the second aha moment was, "Hold on, we're not just going to go do it. We're actually going to go slow to go fast. We're actually going to do this the way, knowing all I know now. We're going to do it the perfect way based on what I know now."

And because the market is so dynamic and the world in general is so dynamic, we're actually going to put in a lot of effort into making this flexible so that we're not locked into a mode of operation as we grow. So, that was the aha moment, a bunch of whys.

Sheikh Shuvo: Awesome. Well, as an ex-Amazon guy, at this stage of where Sapien is right now, what Amazon Leadership Principle (LP) do you think is most important for you right now?

Ahmed Rashad: Funny story about that. When I first joined, I was asked by one of the VPs at Amazon, "What do you think about our LPs?" And I think he laughed, but I think I almost got fired too, because I said, "I think they're wrong." I was like, "Who's this new guy coming in two months in, telling us our LPs that we just worked on are wrong?"

Sheikh Shuvo: Nice ballsy move.

Ahmed Rashad: I didn't plan it. It just happened, like everything else. And it's like, okay, so here's the thing. Customer obsession is a table stake. If we have to mention it, then it's not the right group. But I understand, as a company grows, you need to iterate it again and again to set the culture. But in general, beyond that, there are two things that matter: one, deliver results, and two, hire and develop the best. Everything else is a means to get to those two objectives. So in Amazon speak, those are the outputs, and the inputs are everything else like Dive Deep, Right a Lot, and so on.

So, to answer your question, my favorite LP is Are Right a Lot. Because it's not just about luck or intuition. The way you build intuition is by doing the gritty things, getting your hands dirty and going and doing it over and over again. That's how intuition builds its own magic.

Sheikh Shuvo: Absolutely. It's a type of compounded interest right there. Interesting. Well, one of the fascinating approaches you're taking with Sapien is, it seems to be very much a game mechanics-based experience to really engage the community and lead to higher quality data labeling. In designing the experience of the people doing the actual tagging, what was your approach to getting customer feedback and really understanding that engagement loop?

Ahmed Rashad: Yeah. So, it actually starts a little bit before that. When we thought about it, I understand the expression "data labeling." I don't like it as much. I prefer "human feedback into AI datasets" or human feedback in general. The reason I like that is because it emphasizes the most critical component of this process, which is the human. We said, humans are the most critical component, so we're going to start with the human and work our way backwards. We're going to figure out everything that's problematic with humans today.

I talked to a lot of people and we went and talked to thousands of others, like laborers, people who do data labeling either casually or professionally, and literally built a product roadmap around them, around what are their priorities, what matters to them. That's the first part. And by the way, we also involved a lot of them in the product development because I don't want to develop these things in our ivory towers. I need actual people who are going to use it to tell me what it is. That's the first part of the customer. The second customer for us is the researcher, or the scientist, or the engineer, or the company using the data. We captured their feedback in traditional methods. They don't care as much about the game mechanics or any of that. They just care about high-quality data with good cost, good time, and whether it can scale or not.

For the labeler, it's a whole slew of things. They're involved in the design. We talk to labelers daily. We try new things regularly and get their feedback. We incentivize them to give us feedback. They're part of building this, and this is for them. I want them to build it for themselves. I'm just providing the engineering and all the stuff in the middle, but it's their design.

Sheikh Shuvo: Yeah. Huh. Well, as you've been going through this rigorous testing and development process, getting tons of feedback from both sides of the market, have there been any major assumptions that you had going into this that have been turned around?

Ahmed Rashad: Yes. The whole hypothesis was revised a few times, actually. We started spinning our wheels on, "How do we do feedback? Let's build these complex algorithms and these complex AI and ML methods, and let's do all of these things and supervised and unsupervised," and all of that. And after a while, we just completely trashed that hypothesis after realizing that the biggest problem today, blocking paying people almost instantly, reviewing quality on the spot, evaluating quality on the spot, giving people feedback immediately, and the scarcity of resources—why resources are scarce because tasks are getting more complicated—it's all about the complexity of the task. So, if we can simplify the task to very, very tiny components, all of a sudden, everything else becomes easier. It was a massive realization. If we break down the task, instead of having to teach you 50 things, I can teach you one thing. And it's even better if you already know that thing. I actually don't need to train you on it. If the task has 50 components and five of them require super high skills, why have the one person with the super high skills do 50? Have them do the five and spread out the rest to everyone else to increase quality while reducing costs.

Sheikh Shuvo: In choosing to work with Sapien, what are some of the ways that you motivate people to be a part of your community versus others? What are some ways you are innovating the worker experience?

Ahmed Rashad: So, the core hypothesis is that people are good and want to do good. All we have to do is show them what good looks like and empower them to do it. From that hypothesis, the mechanics are evolving, but for now, we can pay people more because we've removed a lot of the management infrastructure and overhead costs. We're having people do parts of the work that are cost-effective, so we can afford to pay more. That's one of the biggest motivations.

The second motivation is steps. So, leaderboards, fanfare when someone consistently does a good job, not just on their own screen but actually in their community. Third, make the work easier, more fulfilling, and more engaging. That's where we borrowed a lot from gaming. How do you keep a repetitive game interesting? We're building that into our mechanics.

Lastly, genuinely care for the people. Talk to them, care about their wellbeing, not just that you're trying to extract work from them. And it works. But if you fake it, it doesn't work. You can't fake it; people sense it. But if you genuinely care, it will work. They will know that you care.

Sheikh Shuvo: With the gaming focus, I'm crossing my fingers for some type of Fortnite integration down the road.

Ahmed Rashad: We're not going to be developing a triple-A game anytime soon. I'm not promising, but the way I think about it, it's less of a game. It's more of a gamified experience. The whole idea isn't to create a game, but to make difficult work that requires a lot of focus and concentration, and can create fatigue, more manageable. The breakdown actually makes it more fun without sacrificing efficiency. We still need efficiency, but it makes it more engaging. We also developed a lot of skins for the labeling interface, and people can choose whatever skins they want. There's a casino interface, a star graph interface, a nature interface, and so on. This makes the screens, pointers, and shortcuts less exhausting and non-monotonous because you can always change it. Things like where your score multiplier shows up, where your points are showing up, change in the screen.

Another part of gamification is incentives. The more good work you do consistently, you start earning a multiplier that goes higher, like a score streak. For every task, you earn base points for the task times the multiplier. So, you can start earning more per task, and you see it live in front of you, and you don't want that multiplier to go down. You have to be careful and pay attention to keep the multiplier going.

Sheikh Shuvo: Got it, got it. Huh, interesting. Has working on labeling complex data types been limited by the power of the cell phones and computers people use? I'm wondering how you're distilling complex data down into bite-sized chunks.

Ahmed Rashad: So far, it hasn't been a problem because it's manageable. You simplify the task, break it down, truncate it, crop the images, and so forth. You don't need high fidelity for everything, so you can compress them quite a bit. However, there are a couple of cases in the medical field where you need high fidelity and the entire image. Hardware has been a bit challenging for that. We're working on options to minimize the bandwidth required and the hardware needed, which can be restrictive. In one case, a doctor evaluating needs to look at anywhere between five and 12 images, each about two gigabytes. That's a lot for one task.

Sheikh Shuvo: Well, that's a fun engineering problem with latency to solve.

Ahmed Rashad: It's an incredibly fun engineering problem. We're exploring options that involve splitting stuff apart and then stitching them back together.

Sheikh Shuvo: It's going to be fun. That's cool. Shifting gears a bit, just talking about various AI tools in general. There's obviously an explosion in the market right now. I'm wondering, just looking at your individual workflows, are there any ways you're using AI tools on a day-to-day basis?

Ahmed Rashad: There are ways that we are using AI. So, there are three ways we do this. One is for our own productivity internally and so on. Two, we are actually developing AI applications to label parts of the task. Because if you can automate it, you don't need humans to do it, which is easier, simpler, faster, cheaper, all of those things. And the third way is in our quality tracking, where we use machine learning linters to validate or predict the probability of a task being good or not. We actually use quite a bit of AI, but our principle is not to use AI for the sake of checking a box. We use AI if it's going to make the overall process faster, cheaper, and with higher quality. If not, we're going to either stop doing it or iterate on it. I don't want to check a box; I want it to be useful in the process.

Sheikh Shuvo: Yeah, that's a great framework to have. The very last question I have for you, Ahmed, is looking at team and talent as you're building your team at Sapien, outside of just solid experience, what are some character traits you look for in new hires and what are some of your interview strategies to assess those?

Ahmed Rashad: Actually, I've documented this to a large extent because I want to standardize and normalize it for the company now and moving forward. Early stage, you need different types of people than mid-stage and later stage. I'm only focusing on the now, but there are a few things I want people to have. One, I want them to always have a free-for-all best idea wins approach. Two, I want people who are owners, no task is above or beneath them. We're only in the business of delivering results and hiring and developing the best. Nothing else matters. We'll do whatever we need to do. Three is people who have an internal locus of control.

Ahmed Rashad: Basically, they solidly believe that they happen to the world, the world doesn't happen to them. It doesn't matter what's happening. You cannot control the world. The only thing you can control is yourself and your actions and your choices and your thoughts. I want people with that mindset. And then, four, I want people who do not split the difference. If we're splitting the difference, if we're negotiating, it means we don't have the same objective. We have different views. We're trying to solve for different things. I want people who are obsessed with defining the problem properly, aligning on what the right outcome is, and then working together to find the right outcome, wherever it comes from. My opinion, your opinion, her opinion, his opinion, it doesn't matter. Rank, pedigree, experience, all that stuff, it doesn't matter. At the end of the day, it's about the quality of your ideas and your ability to execute. And then lastly, I want people who operate in tight formation and people who are specific. So, one thing borrowed from Amazon actually is no, we don't use reason words. For example, I almost completely abandoned words like 'well,' 'better.' What does 'better' mean? I don't understand what that means. Show me the number. So we are growing. Okay. How much? Why is this good or bad? I don't know. So be specific. Always be very specific. It helps keep everyone focused.

Sheikh Shuvo: Those are great. It seems like very much a haiku of the LPs that you're living right there. It had to converge at some point. Yeah, yeah. Awesome. Well, Ahmed, that's all I have for now. Thank you so much for sharing more about your world and what led you here. It sounds like you're on an exciting mission right now, and it'll be fun to talk to you again in a couple of months to see how the experiments have panned out.

Ahmed Rashad: That sounds wonderful. I'm very excited. It's weird, incredible, scary, and beautiful at the same time. It's amazing.

Sheikh Shuvo: Well, thank you for having me. Yep. Best of luck. Bye.