Stop Tracking Everything. Here’s What Actually Matters.

I told a room full of women founders to f*** their feelings. Then I showed them how to read data instead. 42% of startups fail because they built something nobody wanted — and most of them were tracking the wrong numbers the entire time. This is workshop one of a three-part series on metrics, hypotheses, and tracking what actually matters when you’re building from zero.

TL;DR: Most early-stage founders are tracking the wrong numbers. Followers, page views, TikTok likes — that’s noise. Signal is the data that tells you whether your product is working, whether your customers are activating, and whether the thing you’re building actually matters. a16z explicitly calls downloads a vanity metric in their canonical list of 16 metrics investors care about. This is what I taught our Women Build Cool Sh*t cohort about separating signal from noise — with real examples from their own businesses and research to back it up.

What you will learn in this post

• What Signal vs. Noise Actually Means (With Data)
• Why Your Feelings Don’t Matter (And Data Does)
• Real Examples of Noise Metrics From Our Founders
• What Activation Actually Means (And Why 75% of Users Churn in Week 1)
• Why You Should Stop Tracking Everything

What signal vs. noise actually means

Here’s the simplest way I can explain a noise metric: I have 5,000 followers.

Okay — but if those 5,000 followers aren’t paying you for anything, then that’s noise. I once got 300,000 impressions on a video. Sounds incredible, right? It turned into nothing. It had nothing to do with what I was actually talking about. It just went viral. That’s noise.

CB Insights analyzed post-mortem data from over 100 failed startups and found that 42% failed because they built something with no market need. Not because they ran out of money first. Not because the team fell apart. They built the wrong thing — and the data they were tracking never told them. Andreessen Horowitz’s list of 16 startup metrics makes this explicit: downloads are a vanity metric. What investors actually care about is engagement, retention, and unit economics.

The size of something or the appearance of something does not mean it’s positive intent for your business. What happens when you see these noise metrics is that you expect them to do something. You want them to. But they don’t always have the outcome you’d like.

“Remove your emotional ties to your expectations of what something will do and look at these things analytically. What actually happened? What was the information? How can I use that?”

Why your feelings don’t matter

I was telling one of the women in our cohort this week: f*** your feelings. Your feelings do not matter. This is about data and paying attention to numbers.

You’re probably thinking, I didn’t sign up to be an analyst. Well, yes, you actually did. We are all analysts. There’s research to back up why this is so hard: a 2000 study by Iyengar and Lepper at Columbia and Stanford.pdf) found that when consumers were shown 24 options, only 3% bought. When shown 6 options, 40% bought. That’s the choice overload effect — and it applies directly to founders drowning in dashboards. 62% of employees say their work suffers from information overload. Founders are worse. You’re tracking 15 metrics when you need 3.

A lot of founders struggle here because they’ve never been taught how to pick up on patterns. How to see a pattern before the pattern emerges. Data analysis is genuinely challenging. If you haven’t been accustomed to reading data and correlating different signals to outcomes, it will take time. Harvard Business School’s research on lean startup methodology found that a hypothesis-driven approach to entrepreneurship outperforms traditional planning precisely because it forces founders to separate what they know from what they assume. Most founders are running on assumptions. Your job is to test them.

I asked two of our founders who have backgrounds in data analysis how long it took them to really learn to read data that wasn’t directly in front of them. The answer? Years. And your job is to find a story in the numbers. Don’t let the goals of the business change just because a number moved. That’s chasing shiny objects. YC identifies this as a common first-time founder mistake — obsessing over non-essential activities instead of focusing on the things that actually move the needle: team, market, and early traction.

We had Amy Nelson come speak at Startup Weekend Women this past weekend. She founded The Riveter, raised $30 million, her husband was wrongly sued for a federal crime, she won, and she has a million followers. When she was raising capital, she said the sexism in the room was obvious. Her advice? You just have to not give a shit. You can’t change it. You bulldoze past it. Your feelings do not matter in this game.

Real examples of noise metrics

I asked our Women Build Cool Sh*t cohort: what noisy data points have you been paying attention to that you thought were important but aren’t? Here’s what they said:

• TikTok likes and views. The classic. The average organic social media conversion rate is 1.5–1.7% across all industries. Unless those views convert to something you can measure, they’re entertainment metrics, not business metrics.
• Signups without activation. One founder had 9 signups but only 3 were actually her ICP. Going to the registration page takes 12 seconds. That doesn’t mean success. According to Product School, 75% of users churn in the first week. If they’re not activating, signups are vanity.
• People who say they care but don’t act. One founder tracking recycling behavior said her biggest noise metric was survey respondents who claim they care but never follow through. Want-to-be data is dangerous.
• Page views from spam. Imperva’s 2025 Bad Bot Report found that 51% of all web traffic is now bots — automated traffic surpassed human activity for the first time in a decade. Bad bots alone account for 37%. Use Google Search Console instead of raw page views — you can see what humans are actually searching for.
• Logins without engagement. Users who log in but never do the core action. There’s a nuance though — if they’re logging in consistently, figure out what page they’re going to and how long they’re on it. You might find lurkers who absorb knowledge. If there are 10 more like them, that’s a segment worth understanding.
• Views from overseas bots. One founder discovered most of her traffic was bots crawling from China. AI/LLM crawlers alone have quadrupled their traffic share from 2.6% to 10.1% in 8 months. That’s why raw analytics are increasingly unreliable.

The 53,000 impressions I posted about on my blog SEO article? That number sounds insane. It means something is working. But if you read the article, you’d see my click-through rate was terrible. The 53,000 is actually the vanity metric. The story behind that number — why people saw it but didn’t click — is the signal I need to focus on.

Here’s a good rule of thumb: if a number makes you feel good but doesn’t change what you do tomorrow, it’s noise. If a number makes you uncomfortable but tells you exactly what to fix, that’s signal. The Startup Genome Report studied thousands of companies and found that 74% of high-growth startups fail because of premature scaling — acting on vanity metrics as if they were proof of product-market fit. The founders who survive are the ones who stay uncomfortable long enough to find real signal.

What activation actually means

Activation is the time it takes for a user to sign up and have their first time to result. Their first moment of, oh, I get it. And then they become sticky. The data here is brutal: 60% of SaaS trial users never return after their first use. 75% churn in the first week. The average SaaS activation rate is just 37.5%. That means nearly two out of three people who sign up for your product never experience the value you built.

But here’s the upside: improving activation by just 25% can increase revenue by 34% — activation has the biggest impact on MRR in a 12-month period compared to any other funnel lever. And users who complete onboarding are 80% more likely to become long-term customers with 3x higher lifetime value. The problem? The median onboarding checklist completion rate is just 10.1%.

For Theanna, I’m using the Women Build Cool Sh*t cohort as my activation mechanism. Before this cohort, people were signing up — 25 to 30 per month — but they didn’t know how to get to that first result. So I thought, let me just show them. Let me do a cohort and walk them through the process to get to that first aha moment.

Activation for your product is different. It might be the first time a user feels seen. It might be the first moment a mother feels relief. It might be the first time a young professional hits a career milestone. Your job is to define what that moment is. Then track how long it takes to get there, what steps are involved, and how to make it faster. The top free-trial performers get time-to-first-value under 10 minutes. What’s yours?

There’s a reason Superhuman requires every single new user to do a 30-minute onboarding call. It’s not scalable. But it doubled their activation and referral rates compared to self-serve. They hit 65% full email migration in the very first session. That’s what Paul Graham meant when he wrote that the most common unscalable thing founders have to do at the start is recruit and onboard users manually. At this stage, your job is to be the product’s concierge.

“Since I started the cohort, my retention has gone up and my churn has gone down. I had a 12% churn rate. I had a hypothesis that I was losing people because they didn’t know what to do. I dropped to 7%. Track your information based on the hypotheses you’re making.”

Stop tracking everything

Don’t track everything. When you do, it becomes ten different jobs and you get overwhelmed. The choice overload research.pdf) applies to founders too — more options, more metrics, more dashboards leads to worse decisions, not better ones. More is not better. It’s just more. Amplitude analyzed over 11,000 companies and found that the ones who align around a single North Star metric grow significantly faster. That’s the opposite of tracking everything. That’s tracking one thing obsessively and letting it guide the rest.

Social media is particularly noisy because you need to build an audience and it takes a massive number of touch points before a consumer buys. Google’s research on digital consumer behavior found the average purchase journey involves anywhere from 20 to 500+ touchpoints depending on the complexity of the purchase. Their 7-11-4 rule says consumers need 7 hours of engagement, 11 touchpoints, and 4 separate locations before they buy.

I get people who say, I’ve been following you for a year and I’m finally ready. That’s the long game. That’s why you have to be consistent. But I don’t track my short-term social gains because I know the short-term doesn’t convert. Meanwhile, the average organic social conversion rate is 1.5–1.7%. For SaaS specifically, treat 1% as your baseline. That’s the reality of the funnel.

What I am finding is that since I’ve shifted to blog content and gotten precise about all of my keywords in LLMs, it’s working faster. Why? Because I can show up in all these ways that people are searching — and their attention span for video is lowering. Written content compounds. A blog post that ranks today keeps ranking for months. A TikTok video that goes viral today is gone in 48 hours.

Anytime you’re looking at a data point, ask why. Then sit there and think on that data. Try to string the story together. And remind people who you are — they forget. They’re constantly bombarded. If you think of it like they have the memory of a goldfish, it won’t feel cringey anymore.

The Startup Genome Report found that startups who pivot 1–2 times raise 2.5x more money and have 3.6x better user growth than those who pivot more than twice or not at all. The difference? They tracked signal, ignored noise, and made decisions based on what the data actually told them. That’s the skill you’re building right now.

Up next: the three numbers that matter

Now that you know what to ignore, the next question is: what should you actually track? In Part 2, I break down the only three numbers that early-stage founders should be watching — your North Star metric, your activation metric, and your retention signal — plus how to read between the lines when the data doesn’t tell you the answer directly.

This is a three-part series from our Women Build Cool Sh*t workshop on data, metrics, and hypothesis testing. Part 2 drops tomorrow.