Building for Long-Term Value - Why Retention Should Be Your Top Priority

Thank you, Summer. I owe you much more, and I've learned way more from you. That's one of the reasons why I'm here. The last couple days have been really eye-opening to me, learning from all of you. I'm standing up here to talk about some lessons learned and things that I hope can spark a discussion. Definitely ask questions throughout. I'll ask you some questions throughout and keep it conversational and fun, and we'll definitely have time to answer some deeper questions, too.

I want to tell a little bit about my story and my career over the last 25 years. I thought about making a slide similar to yesterday's of how many billions of users I've been part of helping onboard to products. I didn't make that slide, but one thing I can tell you that's certain is I've been part of products that have churned or not retained more users than they've retained. I've probably been on products that have grown over 2 billion users. But if you actually counted up how many people did not retain, it's over 2 billion. It's a greater number. And it always has been for the majority of the products I've worked on.

So today I'm going to be talking about building long-term user value and why retention should be the top priority. Whether you're a founder or a builder or a VC, I hope the things that I share with you today are able to help you reflect on the role that you have for your team, for who you're building for, and most importantly for the people and societies that we're building for and trying to bring great things to. I think AI is an evolution of technology, but also the application in ways where a lot of you and a lot of the stories that I've heard of over the last few days are taking huge risks and growth and your ability to succeed is really based on learning. So I hope throughout this story and throughout our discussion we're able to think of long-term value and retention as really a proxy for learning.

A bit about my history because when I get introduced as I would say at Google or Instagram or Facebook, previous companies I worked at, people often think big data. There's a lot of data and why does it matter? I'm a founder, I'm doing a startup, I'm working on my first few customers. It's not a translatable story or the research methods that you employ on teams with large numbers of users are different than the research methods or the customer interactions that you can have in your smaller startup efforts that will grow.

I started my career 25 years ago working in Latin America on a project called Linkos with MIT Media Lab as an ethnographer. I was living and working in Costa Rica, Dominican Republic for weeks or months on end just living with communities and trying to understand how these shipping containers that were converted into internet cafes could help the community. How could they help community health workers? How could they help students connect to the internet and have that initial exposure to what the mobile web at the time could bring? This was a project of deep ethnography, very, very small sample size. It started with one shipping container, grew to five in Costa Rica, in Dominican Republic started with one site, grew to about 20. But it wasn't about numbers, it was about people's lives, it was about people's stories.

From there, went on to do projects with Media Lab Asia, applying a similar approach to corporate investment and trying to bring technology to the next billion we referred to at the time. Again, that was a project based on ethnography and embedding yourself in the communities and really understanding people's lives and how access to new technologies or things that were being built, for example, tablets and hand-helds at the time in the early 2000s, could improve people's lives.

I had a boot doggle down to Antarctica. I talked to someone the other day who was about doing ethnographic research. This was initially a proposal to do ethnographic research on isolated environments, the International Space Station, which turned into an Antarctic project. But I was an embedded ethnographer there, again, just listening, learning, and spending time with the people who were trying to understand and reduce off-the-job injury for.

Fast forward to the mid-2000s when phones became ubiquitous, I landed myself in sub-Saharan Africa. This is where I had a big career change with regard to research methods in my view of DEPA. I went to Penn and studied demography and large-scale survey deployment, trying to understand how social networks improve people's lives. This is where it just opened up my view. I'm sharing this because if you're on the qualitative side, I really encourage you to open up your view to the quantitative side and vice versa. Then worked on mobile banking and ultimately landed Facebook and Instagram. Now I'm at Google leading our Gemini UX research team, formerly known as BARN. If you use that, you have that identity crisis.

This, I hope, tells you a bit about my background and the diverse experiences that I've had. But I can say with certainty that I've made more mistakes, I've had teams that have made more mistakes than successes. And as I said, we've churned and not retained more users than we probably have retained. And in reflecting on what I can share with all of you today, I hope I'm able to share a few tips and ideas that help you reflect on what things to lean into and try. What are some frameworks? What are some things you can do to really focus on user value? And how can you go deep on those?

Yesterday we talked about ideas that are sort of cheap and easy. It's all about the execution. The things I'm going to share today are all about the execution. One thing here. So why retention? Can someone tell me what their definition of retention is? Or how you use it in your team today?

Recurring revenue. The number of users who come back to the platform under X number of days. So number of users who come back to your product. I'll give a squishier one. It's sort of like how, like building a long-term relationship with the product or the brand. Building a long-term relationship. Let's do one more. Measure of value. If you deliver on the promise, then they would like to come back. Okay, measure of value, this return. So there's the seam of returning coming back.

I want everyone to take out their phone and unlock your screen and look at your first page. If you have a bunch of folders and not apps, click on a folder. Look at apps. I'd like someone to raise their hand and share an app that they use every day. In the back here. YouTube. YouTube. And can you tell me in a sentence what value YouTube brings to you? Entertainment and relaxation. Entertainment and relaxation.

Now can someone raise their hand where they're looking on their home screen, hopefully, but if you have to swipe, it's okay, and find an app that you haven't used in the past month? We got one back here. Microsoft Authenticator. And why have you not used it in the past month? Because I use it to catalog my expenses, and I do that once a month, but I'm usually late. So it was going over a month, and I just hate using this thing. All right. That last sentence. Let's do one more. I got Smart Things, which is the Samsung thingy for smart devices. And I've got TV that I can actually connect to it, and Samsung fridge that's useless. How does that make you, okay, useless. I was waiting for your turn to do that.

So I'm glad none of you said Gemini. If you don't have a Gemini installed yet, you should. But I think there are a couple words there. Hate and useless. Those are not quantitative words. Those are emotions. Those are feelings. And what I want you to think about is retention is not a metric. We often think of retention as some metric you have to strive for, some goal or number you have to hit. And yes, it can be a metric, but it doesn't have to be. You don't need to have billions of users to have a retention curve or a retention metric. You can do this with more than one user, hopefully, but just a handful of users, 10 users, and really think about the value of the emotion and what you're building for that individual or for that group.

So retention not being a metric, it's really a mindset. This is something that, you know, starting with the discussions that we had a couple of nights ago on the Thursday night kickoff and yesterday, that really resonated with me was many of you, whether you're a founder, a builder, or a VC, talking about your approach, talking about building empathy, talking about interacting with customers to really understand them, and to going into an VC meeting saying, great business pitch, but they don't have craft. The buttons are there, but it's not filled with craft. There's an emotion behind this.

And whether you're trying to track something over time or make a case, I want you to be thinking of retention as a mindset, not just a mindset that you have to have, but that you need to push and strive for your teams to have. If you don't have this singular mindset, and this is where through the different experiences I've had, I've seen more failures than successes, this is the missing ingredient when there's failures. People are looking to grow. People are saying, how many new users have you gained? What is your daily active usage? What's your monthly active usage? I think one reason why you don't see companies report out retention metrics is because they show weaknesses. They show the gaps in the user value, and they lean into the numbers that show upward to the right curves, and not that good retention, which I'll show you later in this talk, can be 30%, can be 50%, and that's okay. But you probably don't want investors or the public thinking about that when they're looking at your growth opportunity. Here's the lightly copyedited version of the transcript, preserving all original content:

I started by saying I'm going to share a few tips, approaches that in my experience have been the keys to success for different teams. As I share these, I would love to have you share some examples. I'll also be sharing some examples and happy to answer questions, how we've applied these in different teams that I've been part of.

Ultimately, these three approaches to mean are the necessary ingredients or the minimum ingredients to drive, most importantly, long-term value for the people you're building for, which is also a proxy for retention. I'll go through these one at a time, but defining the problem, understanding your customers, and tracking progress. What I'm telling you here is nothing new. All of these topics were discussed in different ways yesterday. Many of you, I'm sure, have experience in a lot of things that I could learn from in each of these areas.

Starting with the first, and I'll tell a short story about each, defining the problem. I think one thing that I've been very grateful to have had the experience is to work on teams that are constantly evolving and constantly changing. With evolution and change and growth comes a lot of change in processes. I'm sure you've been part of teams, small teams or big teams, that have different templates to say, fill this out. This is what our next review cycle is looking like. This is what we would like you to bring to the leadership team or to our team to discuss an instructional format.

This here is a very simple format of how we define problems on teams that I've been part of. Something that started with a very long list and has been pared down over time. Defining what the problem is, and for those of you writing business cases or pitches or seeing a lot of proposals as we see, I'm sure these look like very simple questions, but they're often not answered. They're often not answered even by teams that have very experienced leaders behind them and a ton of data behind them. There are meetings that I've gone into with Mark Zuckerberg or Prepped for Sundar and different execs for. In some of these questions, we do not know the answer to and we're going into these meetings pitching a feature update, pitching a business plan.

Where we fail is when we haven't done some of the simple network companies. Defining what is the problem. That's something where I always try to ground things in people's needs and what success means for those people. Understanding how you know it's a problem. That allows you to do desk research. That allows you to talk to other people and really understand is it a problem that exists? How big is it? What's the opportunity? Has the problem been attempted to be solved before? This allows us to go very deep and look to see who are the other players in the field. What's the opportunity in the market for you? Where have others failed?

I think one blind spot we often have is we have a great idea and we believe in our team's ability to execute. But we don't understand how others have tried to solve the same problem that we're working on ourselves. And then lastly, when will the problem be solved? This is also one of those things that I think people don't want to answer. Even ourselves, we don't want to answer. If you're building a dating app, you want growth, you want people to be using it. But should you be thinking about when should someone stop using that dating app? What does success mean? Maybe when they find that partner that they're committed to and they stop using it. Do you count that as success?

Are you working on an app to help people through anxiety? And have you helped them get through that anxiety and help them get through important stages where they no longer need your app or service for that day-to-day support? And do you count that as success? Or do you look at it and say we need more growth? So where do you get these wins? And how do you really know when the problem is solved? And that problem could be solved in a way that relates to fewer customers, fewer people relying on your services.

Just do it quickly. So understanding your customers. How many of you know the milkshake story? I see a few hands. The reason I'm sharing this, and Aicha was at Instagram at the time with me and probably remembers, is at Facebook and Instagram we had a big push for jobs to be done. And had external consultants come in, run courses, some of the people that helped develop it played Christiansen. And it was really thoughtful but a very simple approach to really just taking a step back and saying okay, what's the job to be done? I assume most people have heard of job to be done. I see the thumbs.

The milkshake story is really the first story around it that helped inform why people hire products and what you can do to make them better so that they rely on them more, get more value out of them. Long story short, and I encourage you to go look at it online. You can just Google milkshake story job to be done. There's a great video of Clay Christiansen telling this story who's a HBS professor. Basically McDonald's wanted to build a better milkshake to sell more milkshakes. They had ethnographers, the team that we were working with, sat at a McDonald's from 6 a.m. to 11 p.m. watching people buy milkshakes.

And the first thing that they learned, this is one store, one small team of two people, more than half the milkshakes are sold in the morning. And almost all of them are sold in drive-thrus. And it's the only thing that people buy, just one milkshake. I was like, that sounds amazing. My parents would never let me. I love McDonald's milkshakes. But they learned through just observation and qualitative work that people were buying milkshakes when they were driving to work. They had a long, boring drive. They needed something fast. And they actually liked the milkshake because it could last their entire drive. It's that thick consistency that goes through a really small straw by design.

So actually how they improved the milkshake and they ended up selling more is they made it thicker and slower to drink. You're stuck in your car. You're not going to do a U-turn and go back and fill up on another milkshake. So it's a great example where people want to make process. They have needs. This is a very simple framework. And I sort of subscribe to jobs to be done as a mindset and approach, not a specific set of research methods. But these are the three key things, functional, social, and emotional jobs.

And when we were at Instagram, we came back from a Thanksgiving break. And Kevin Sistrom, one of the co-founders and CEO, sent a post out. I was leading the research team at the time. Sent a post out to the entire company of why jobs to be done is the most important thing. And he actually outlined Instagram's jobs to be done in terms of what the job was six years ago when it was founded and why after he read Clay Christensen's book over Thanksgiving, he feels that this is the enlightenment moment that allowed him to get clarity on what Instagram was actually solving for.

So Instagram's job to be done was to help you help reduce that social pressure of posting a photo by allowing you to put filters on it. And the inflection point of Instagram that I think was very important here is, as a leader, Kevin, not without a sense, said, I want every project or product review to start with a job to be done. The first line on the document has to be the job to be done. So what we did is we deeply studied the functional, social, emotional components of the job. And we ended up studying the obstacles and struggles that people faced and ultimately what they hire, fire, and switch.

And this was our sentence structure. There's a few different variations of this. The different teams have used Intercom. There's one company that's written a great guide on it, Sean Townsend was on our team at Instagram and a helped author. But you can basically write out this structure and fit a lot of data into it with regard to the job that you either are being hired for or I want to encourage you to think about the job you want to be hired for. If you do not have a clear articulation of this, I guarantee you, you probably will not build stronger attention. You won't deeply understand the emotional context and the situation someone is in.

For McDonald's one, it may seem pretty simple, but when I'm driving to work and bored, I want to get something to eat that's not messy, that's not going to fall all over me and ruin my work clothes so that I can make it to lunch without being hungry. Every single thing that you're working on, you can write these today. I think the beauty of what we're able to do at Instagram is overnight, it just became a requirement. We had to put these into the product documents and we didn't go out and do new research. We just sat down and wrote them and brainstormed them, and I think that's okay. Even as a researcher on our team, I say, no, we have to go collect data and validate this and make sure it's up to date. I think there's a lot of knowledge that you already have about your customers that you can probably sit down and brainstorm these and write them today.

So the third part, and then I'll just, how are we on time? Five? Okay. And we'll open up for questions. Tracking your progress. And this is where things really, the rubber hits the road. And I'd love to answer questions on this, because of course we have a lot of examples from the different projects we've worked on, but these are some of the staples that I'd really encourage you to have. Understand the drivers of retention and sharing. Here's the lightly copyedited version of the transcript, preserving all original content:

Again, it doesn't have to be data, but if you don't know these, you have a ton of blind spots and are going to make a lot of mistakes and waste a lot of money. At Facebook when I was on the growth team, when I started my career there, we had a list of retention and sharing drivers that were based on both qualitative work but also surveys that we would put into the product. We would survey churned users via email. If they churned, we would reach out to them and say, "Hey, we haven't seen you in a month. Why is that?" Or reach out to them through different surveys and try to have a one-on-one conversation and talk to them and understand what happened, what were the things that led up to that churn.

On the retention side, if you don't in your product have something that prompts users or your customers to understand why they're coming back. "Hey, I haven't seen you in a while. What happened?" Add it or find a way to find those users and call them or meet with them and understand what brought them back. What was that thing that pulled them back in? Did they try another product and switch and it didn't work? Or did they see something or experience something that you did? Maybe knowingly, maybe unknowingly to bring them back.

Have a growth equation. One of the simplest things, but most companies do not have it. New users, what we all look for, how do you get growth? Open up the funnel, message people, do advertising, do all these things. Costs a ton of money, usually. You have organic growth, amazing. Minus churned users, those are the sad ones. Those are the ones that say, "I hate it" or "it's not useful." And resurrected users.

This equation, when I was on Facebook, I joined when I was 650 million and within six months we hit a billion. We were growing two million new users a day organically. We were churning one million monthly active users a day and we were resurrecting 500,000 users a day through very, very expensive SMS campaigns and updates, tens of millions of dollars on SMS campaigns and notifications that many of you probably got too when you came back. Resurrecting users is really, really expensive. And what can you do to prevent having to spend that time and money on that and focus more on reducing that churn so that your net growth is something that you feel proud of?

Divining aha moments. This is one where I just want to get a quick, I know we're running short on time probably, but I want to get a quick hand raise on what your aha moment is in your product. How many people have an aha moment? That moment where you have confidence that every customer has to hit to gain value? Can I get one example of an aha moment? Yes.

"So our product, we help businesses. And so our first aha moment is the first impact that they have in monetary value."

Oh, so I have more money now. You did that for me. Okay. So in your example, your aha moment, your customer knows or your user knows when they hit it? Yes. Can I get one more example of an aha moment?

"We have an app that uses AI to give you a personalized therapeutic exercise for the day or that moment. And the aha moment is when you're receiving your affirmation or your grounding technique, and then you're on that screen and you stop and you reflect and then leave."

And do your customers know when they have that aha moment?

"Through qualitative research, yes. They told us that it helps them and they know and they close it."

Okay. Yeah, I see one more hand.

"I think in our demo, we have a very specific feature, and this is before they opt in to pay us, where we show them their breath. It works every time to be able to see their breath."

I got to do that. But that was an attribute that you can't experience it in real life, other than when it's really cold out. Yeah. Cool. It's very relevant to this.

"I have an analytics product that shows drivers behind the metric. For me, when you're looking at the retention of the metric, you can go in and look at the drivers. That's our aha moment."

Facebook's aha moment that they write about externally from years ago, I think it was established in 2008, 2009. The growth marketing team was getting seven friends in the first ten days. Reality is there's many more aha moments under the hood. But oftentimes your users or your customers don't know their aha moment, and that's okay. Know what the what is, what are the gates or the things you want to help people get through knowing that that's going to lead to successes for them and for you.

Last thing and I know where I'm at, what's your retention term? Most people don't know this. This is what a typical retention curve looks like. No one retains 100% of their units. That's okay. 50% or somewhere is pretty good. There's a great... 50% is pretty great. I don't know if you know that book. Check it out online. There's a chapter where he surveyed a bunch of founders and people about what good retention looks like. There's some more categories but I put the simple ones here. Retention is hard. The numbers are not going to look good. They're going to make you look weak. And what time period is this? Is this day one or day 30 or day 7? Monthly. So it's gonna you know some people subscribe to monthly. We currently use weekly. I do 1, 7, 14, and 30. But know this. You know draw it out. You can even draw it by hand.

Awesome. Thanks. So since we did the questions, we can continue. That's why today is great. Because we have dedicated online. So today is the schedule. By the way, I know he couldn't talk about Gemini. We didn't hear it. It's great so we can get something maybe. I don't know. Questions later and nobody recording.

‍