Two hundred signups is enough to run a launch and not enough to run a regression. The OperatorStack AI Advisor will still write you five bullet points about your referral program at that size, and some of those bullets are solid while others are one signup away from saying the opposite. The difference is not in how confident the writing sounds. It is in which number the bullet is built on.
At a few hundred signups, trust the AI Advisor on counts, queues, and presence or absence: how many signups, which messages are unanswered, whether errors are firing, whether the script is still reporting. Treat every percentage and every week-over-week comparison as provisional, because at that denominator a single new signup moves the number by a visible amount.
The One-Signup Test
Before acting on any advisor bullet that contains a percentage, do this: add one to the numerator and recompute. If the number moves enough to change your decision, the bullet is describing your sample, not your product.
Referral share is the clearest case. The advisor computes it as:
referral_pct = round(referral_signups / total_signups * 100, 1)
With 200 total contacts, every referral signup is worth 0.5 points of referral share. A card reading "referrals are 4.5% of signups, down from 6%" is describing a swing of three people. At 2,000 contacts the same three people move the number by 0.15 points and the trend means something. The formula did not change. The denominator did.
The comparison the advisor draws against its own previous response makes this worse, not better. It is given your last analysis for the same section and asked to note whether things are improving or worsening. Two noisy readings produce a confident-sounding trend between them. "Referral share is declining" can be three people who happened to sign up through a direct link this week.
Counts Survive Small Samples, Rates Do Not
The advisor pulls nine sections, and they do not degrade equally as your data shrinks. Sort them by what they are actually measuring.
Reliable at any size (counts and presence):
- Total signups, 30-day signups, 7-day signups. These are facts.
- Unanswered messages. A queue of 3 unanswered pricing questions is 3 real people waiting, whatever your scale.
- Error signatures. One JavaScript error that breaks the signup form breaks it for everyone who hits that path. You do not need a second occurrence to justify fixing it.
- Installation status. The script is either reporting or it is not.
- Which pages exist and which ones got traffic at all.
Provisional at small samples (rates and comparisons):
- Conversion rate. At 200 visitors, one signup is half a percentage point.
- Referral share, as above.
- Week-over-week change on any metric.
- Message response rate, computed as
responded / len(messages). With 6 messages, each one is 16.7 points. - Best and worst day in the daily signup series.
That last one deserves its own section, because it is the bullet founders act on most and the one small samples distort hardest.
Why "Your Best Day Was Tuesday" Is Usually Noise
The signups section hands the model a 30-day daily series, with missing days filled in as zeros:
def _fill_daily(counts: dict[str, int], days: int = 30) -> dict[str, int]:
"""Ensure every day in the range has an entry (fill gaps with 0)."""
if not counts:
return counts
today = date.today()
filled: dict[str, int] = {}
for i in range(days - 1, -1, -1):
d = (today - timedelta(days=i)).isoformat()
filled[d] = counts.get(d, 0)
return filled
Then it asks the model to identify the best and worst days and judge whether momentum is building or fading.
Two hundred signups spread across 30 days averages 6.7 a day. Real daily counts will land somewhere between 2 and 12 by ordinary variation alone, with no change in your marketing, your page, or anything else. A day with 11 and a day with 3 sit in the same series, and the model will faithfully report that your best day was nearly four times your worst.
The zero-fill is the right engineering choice and a trap for interpretation. It means a day you got no signups appears as a real 0 in the series rather than a gap, which keeps the trend math honest. It also means a quiet weekend produces two zeros that look dramatic next to a weekday 8, when the whole range is a handful of people either way.
What to take from the daily series at this size: the shape over weeks, not days. If weeks one through three run 60, 58, 62 and week four runs 12, that is worth a look. If Tuesday beat Thursday, that is a coin flip you are narrating.
Why did the AI Advisor show fewer cards than last week?
Each of the nine sections has its own data gate and returns nothing when its underlying count is zero. No form submissions in the last 90 days means no forms card. No chat conversations means no chat card. A missing card is the advisor declining to comment, not an error, and it is the correct behavior at small scale.
Can I trust the performance card's LCP p75 with low traffic?
Not hourly. The p75 is computed as the value at index int(samples * 0.75), so an hour with 2, 3, or 4 pageviews reports your single slowest pageview as the p75, and 5 through 8 samples report the second slowest. Read the daily or weekly performance score instead, and ignore hour-to-hour swings until each hour carries double-digit samples.
Does the advisor know my sample is small?
Partly. The system prompt tells it to judge metrics relative to the project's own scale, so a drop from 5 visitors a day to 3 is not flagged red the way 500 to 50 would be. It is also told not to manufacture insights when nothing stands out. What it does not do is attach a confidence interval to a percentage, so you still have to apply the one-signup test yourself.
Should I wait to launch until I have enough data for the advisor?
No. The counts, queues, and error signatures that matter most before launch are reliable at any size. Waiting for statistical power on a conversion rate means waiting for traffic you only get by launching. Use the advisor for what is broken or unanswered now, and defer the rate-based decisions until the denominator is large enough to survive one more signup.
::