Knowledge Base

Validating a startup idea is not about guessing; it is a systematic process. Based on the insights from Y Combinator's "How to Validate Your Startup Idea", here is the definitive 5-step framework.

1. Find the Problem

Great startups are not "ideas" looking for a problem; they are solutions to existing problems. The best problems are ones you experience yourself.

2. Determine if it's a "Hair-on-Fire" Problem

Is this a mild inconvenience or a critical issue? Users should be desperate for a solution. Valideum assists here by analyzing forum sentiment to measure the "desperation level" of your potential niche.

3. Find Customers (Talk to Them)

You cannot validate in a vacuum. You need to verify that people exist who have this problem. This is where Valideum shines. We automate the process of finding these users via targeted ads and bringing them to a landing page to measure their intent.

4. Build an MVP (Wizard of Oz)

Don't build the full product. Build the manual version. Do things that don't scale to prove the value proposition.

5. Iterate

Use the feedback to refine the solution. If Valideum's report shows low conversion, you iterate the offer, not the code.

Many new tools promise to "validate" your idea by asking ChatGPT if it's a good idea. This is dangerous.

AI models are trained on past data. They cannot predict if a specific human being, today, will click "Buy" on your specific value proposition.

True validation requires Skin in the Game. A user must trade their attention, email, or money for your promise. That is why Valideum runs real ads on real networks. We don't guess; we measure.

When testing a business idea, the Landing Page is your laboratory. If the lab is dirty, the results are useless.

• Vague Value Props: Be specific. "We help you work better" vs "We save you 10 hours a week on Excel."
• Too Much Friction: Don't ask for a credit card just to gauge interest. An email is often enough for Step 1 validation.
• Talking About Features, Not Benefits: Users buy the hole, not the drill.

In the early days of startup growth, entrepreneurs struggled with a fundamental question: How do you know when you've actually achieved product-market fit? Marc Andreessen famously described it as something "you can feel," but feelings don't scale and they certainly don't convince investors.

Enter Sean Ellis, the growth hacker who coined the term "growth hacking" and helped scale companies like Dropbox, LogMeIn, and Eventbrite. Ellis developed a quantitative approach that transformed PMF from a subjective feeling into a measurable benchmark.

The Question That Changed Everything

The test is elegantly simple. Ask your users:

The 40% Benchmark

After surveying over 100 startups, Ellis discovered a pattern: companies where at least 40% of users said they'd be "very disappointed" almost always achieved strong, sustainable growth. Those below the threshold typically struggled.

This isn't arbitrary. The 40% threshold represents a critical mass of users who view your product as a "must-have" rather than a "nice-to-have." These are the users who will evangelize, retain, and ultimately drive organic growth.

Case Study: Superhuman's Journey

Perhaps the most famous application of this test came from Rahul Vohra, founder of Superhuman. When they first ran the survey, only 22% of users would be "very disappointed" — well below the benchmark. Instead of panicking, they used the data strategically:

1. Segment the supporters: Identified who loved the product most
2. Analyze the feedback: Understood what those users valued
3. Build a roadmap: Focused on amplifying what lovers loved
4. Repeat: Re-surveyed continuously

Within three quarters, Superhuman's score rose from 22% to 58%. The methodology worked.

How to Run the Test Properly

• Minimum sample size: 30 responses for directional data; 100+ for confidence
• Target active users: Survey people who have used the core feature at least twice
• Time it right: Users need enough exposure to form an opinion (usually 2+ weeks)
• Combine with qualitative: Follow up with "why" questions

What If You're Below 40%?

Don't scale. A PMF score below 40% means you're trying to fill a leaky bucket. Every marketing dollar spent acquiring users will be wasted on churn. Instead:

• Dig into feedback from your "very disappointed" users
• Understand what specific problem you're solving for them
• Double down on that use case
• Iterate until you cross the threshold

Further Reading

• How Superhuman Built an Engine to Find Product-Market Fit — First Round Review
• Measure and Improve Product/Market Fit — Reforge
• In-Depth: How to Measure Product-Market Fit — PostHog

Before Steve Blank, the startup world operated on a dangerous assumption: that if you build a great product, customers will come. This "Field of Dreams" mentality burned through billions in venture capital and left a graveyard of failed startups in its wake.

Blank, a serial entrepreneur with multiple successful exits, noticed a pattern in his failures and successes. He codified his observations into a methodology that would eventually become the foundation of the Lean Startup movement.

The Core Insight

The keyword is search. Startups are not smaller versions of large companies executing known business models. They are search operations, testing hypotheses about customers, problems, and solutions.

The Four Steps

1. Customer Discovery

This is where most founders skip ahead — and where most startups die. Customer Discovery is about testing your hypotheses about the problem, not the solution. You must:

• Articulate your assumptions about who has the problem
• Get out of the building and talk to real potential customers
• Validate that the problem exists and is painful enough to solve
• Iterate until you understand the customer segment deeply

2. Customer Validation

Now you test if your solution matches the validated problem. This step proves you have a repeatable sales process. Key questions:

• Can you sell the product consistently?
• Is the customer acquisition cost sustainable?
• Is there a clear path from prospect to paying customer?

Failure here means returning to Customer Discovery. This is called a pivot — not a failure, but a learning.

3. Customer Creation

Only after validating the business model do you begin scaling customer acquisition. This is the first time you invest heavily in marketing and sales. Doing this before validation is the #1 cause of startup death.

4. Company Building

The transition from a learning organization to an execution machine. You've found product-market fit; now you build the infrastructure to scale.

Get Out of the Building

Blank's most famous directive is deceptively simple: "There are no facts inside the building, so get outside."

This means:

• Stop building features in isolation
• Stop assuming you understand your customer
• Stop running surveys that confirm your biases
• Actually talk to people who might pay for your solution

The Scientific Method, Applied

Customer Development treats your business model as a set of hypotheses to be tested, not truths to be assumed. Each customer conversation is an experiment. The goal isn't to sell — it's to learn.

Further Reading

• Steve Blank's Blog — Original Source Material
• Customer Development — Wikipedia
• How to Road-Test Your Business Model — Inc.

Here's an uncomfortable truth: every customer interview you've ever conducted was probably worthless.

Not because you're bad at interviewing. Because humans are pathologically incapable of giving honest feedback about hypothetical products. We want to be nice. We want to encourage. And so we lie — not maliciously, but reflexively.

Rob Fitzpatrick's book The Mom Test explains why even your mother — the person who loves you most — will give you terrible feedback on your startup idea. And more importantly, how to fix it.

The Three Rules of The Mom Test

Bad Questions vs. Good Questions

Why Hypotheticals Are Poison

When you ask "Would you use this?", you're asking someone to predict their future behavior. Humans are terrible at this. Studies consistently show that:

• People overestimate their likelihood to adopt new behaviors
• Social desirability bias makes them say what sounds good
• They genuinely believe they would act differently than they will

Past behavior is the only reliable predictor of future behavior. If someone hasn't already spent time or money trying to solve this problem, they won't buy your solution either.

The Commitment Test

Compliments are worthless. Commitments are everything. Look for these signals:

• Time commitment: They agree to a follow-up call, trial period, or pilot
• Reputation commitment: They introduce you to colleagues or decision-makers
• Financial commitment: Pre-orders, deposits, letters of intent

If someone says "This is amazing, I'd definitely buy it!" but won't commit to a 15-minute follow-up call, they're lying to you (and possibly to themselves).

For B2B: Follow the Money

In enterprise sales, always ask: "Where does the money come from?" This reveals:

• Who actually has budget authority
• What approval processes exist
• Whether this problem is important enough to fund

The Anti-Pattern: Fishing for Compliments

Watch for these warning signs that you're seeking validation, not information:

• You talk more than you listen
• You pitch before you learn
• You feel good after every conversation
• Everyone seems to love your idea

If everyone loves your idea, you're asking the wrong questions.

Further Reading

• The Mom Test — Official Book Site
• The Mom Test for Better Customer Interviews — Looppanel
• SaaS Idea Validation with Rob Fitzpatrick — SaaS Club Podcast

The most expensive way to test a business idea is to build the product. The cheapest? Pretend it already exists.

Fake Door Testing — also known as smoke testing, painted door testing, or landing page validation — is a technique where you gauge customer demand by presenting a product that doesn't exist yet and measuring how many people try to access it.

The Logic Is Simple

People lie in surveys. They exaggerate in interviews. But clicks don't lie. When someone clicks "Buy Now" or "Get Early Access," they're demonstrating real intent with their most precious resource: attention.

How It Works

1. Create the "door" — A landing page describing your product, a button for a feature that doesn't exist, or an ad for an unavailable service
2. Drive traffic — Use paid ads, social media, or existing product traffic
3. Measure engagement — Track clicks, email signups, or simulated purchases
4. Reveal the truth — Show a "coming soon" message and optionally collect emails
5. Analyze the data — Calculate conversion rates and compare to benchmarks

Types of Fake Door Tests

External Fake Doors (New Products)

For validating entirely new business ideas:

• Create a standalone landing page
• Describe the product as if it exists
• Include a CTA: "Join Waitlist" or "Pre-Order Now"
• Run targeted ads to your ideal customer profile

Internal Fake Doors (New Features)

For testing features within an existing product:

• Add a button or menu item for the hypothetical feature
• When clicked, show: "This feature is coming soon! Want it sooner?"
• Measure click-through rate vs. existing features

Real-World Example: Buffer

Joel Gascoigne validated Buffer with a two-page website:

1. Page 1: Described what Buffer would do, with a "Plans & Pricing" button
2. Page 2: Showed a "coming soon" message with email signup

The click-through rate from Page 1 to Page 2 proved demand existed. Only then did he build the product.

What Metrics Matter

The Ethics Question

Is it deceptive? Done right, no. Best practices:

• Immediately reveal that the feature is "in development"
• Offer something of value (early access, discount)
• Don't collect payment for something that doesn't exist
• Use it sparingly — don't abuse user trust

When NOT to Use Fake Door Testing

• For features users already expect (erodes trust)
• For regulatory or safety-critical products
• If you've already validated demand through other means

Further Reading

• What Is Fake Door Testing — Amplitude
• Fake Door Smoke Test — Real Startup Book
• Fake Door Testing Play — Learning Loop

Surveys are useful. NPS scores have their place. But when it comes to truly measuring product-market fit, there's only one metric that matters: do users keep coming back?

Retention cohort analysis doesn't ask users what they think — it measures what they actually do over time. And unlike vanity metrics like "total users" or "daily signups," retention curves can't be gamed or misinterpreted.

What Is a Retention Cohort?

A cohort is a group of users who share a common characteristic — usually their signup date. Retention analysis tracks what percentage of each cohort continues using your product over time.

Anatomy of a Retention Curve

A typical retention curve has three phases:

1. Initial Drop (Day 0-7): Casual users and mismatched signups leave. This is normal and expected — even great products lose 60-80% of users here.
2. Stabilization (Week 2-8): If you have PMF, the curve flattens. You've found users for whom your product delivers recurring value.
3. Long-term Retention (Month 2+): The curve should remain flat or slightly increase (through habit formation and network effects).

Reading the Curves: Three Scenarios

✓ Healthy Curve (PMF Achieved)

Sharp initial drop, then flattens at 20-40%+ retention. Users who stay become habitual. This is what you want.

⚠️ Warning Curve (PMF Uncertain)

Gradual decline that never fully stabilizes. You might have PMF for a narrow segment, or your product delivers one-time value only.

❌ Flatline Curve (No PMF)

Continuous decline toward zero. No user segment finds lasting value. Back to the drawing board.

Key Retention Benchmarks

Note: Benchmarks vary by product type. Social apps need higher retention than tax software.

Defining Your "Retention Event"

The most common mistake is measuring retention by any login. Instead, define a core action that represents real value delivery:

• For a messaging app: Sent a message
• For an analytics tool: Viewed a report
• For a marketplace: Completed a transaction

Users who login but never take the core action aren't truly retained — they're just window shopping.

Improving Retention

If your curve declines toward zero, focus on:

1. Onboarding: Get users to the "aha moment" faster
2. Activation: Ensure first-time users experience core value
3. Habit loops: Build triggers, actions, variable rewards, and investments
4. Segment analysis: Find which user type retains best and double down

Advanced: Cohort Comparison

Compare cohorts over time to measure product improvement:

• Are newer cohorts retaining better than older ones?
• Did a specific product change improve Day 7 retention?
• Which acquisition channel produces the most retained users?

Tools for Cohort Analysis

• Amplitude — Industry standard for retention analysis
• Mixpanel — Strong cohort visualization
• PostHog — Open-source alternative
• Heap — Automatic event capture

Further Reading

• Measuring Product-Market Fit with Retention — Segment
• How to Evaluate PMF with Retention Cohorts — Lean B2B
• What a Healthy Retention Curve Looks Like — Do I Have PMF