Stop Touching the Product

By Georg Malahov

At the start of July I pointed a review agent at all the changes I'd piled into Call Copilot. It found an enormous number of bugs — genuinely staggering. In its view, almost nothing in there works at all. It was hard to read, but I had nothing to say against it: for months I'd been tweaking the extension in bits and pieces, between other projects, and it had been a long time since I last walked the whole path by hand.

On the upside, the same agent produced a detailed fix plan, and from there I just worked down the list. Then came a night of live testing, and I shipped two new versions to the store — published, found what broke, fixed it, checked again right on production. Not the prettiest release process, but at least it ended with something in the store instead of yet another local branch.

The main thing that made it into these versions: abuse protection. Before, anonymous users were generously handed tokens without any registration — the backend tables show how many of those requests there were. Now no extra tokens are given out, registration is mandatory, and I added feedback collection on top. The last unsolved piece is email communication with the people who register. That one I never finished: this week I only spent a little time on the mail setup and dropped it. That's my open debt from these weeks.

The numbers as of today, July 17. Organically the store gets roughly one to one and a half thousand impressions a day, of which forty to sixty page views and on average about ten new installs a day. And that number is drifting down over time — partly because I hadn't released anything at all since late spring, and the store doesn't like that.

After the release, user behavior changed noticeably. Registrations went up a little: on one of the days about three people registered, and one of them made it through the full flow. But at the same time people seem to be either using it less or uninstalling. My take is that I'm now fairly effectively cutting off people who were never going to pay, while mandatory registration won't stop those who are actually ready to pay. But for now that's exactly "I think," not "I know."

And right there I caught myself with a perfectly familiar urge to improve everything immediately: rewrite the store listing, new headlines, new copy. I stopped. If I change the listing and start showing the paywall more aggressively at the same time, I'll never be able to tell what caused what — did the new headlines lift conversion, or did the gate? So I removed exactly one line from the listing — the one saying no registration is required, because now it is. Everything else stays untouched. The old promo videos are now slightly out of step with the text, but that doesn't seem critical.

Whether I tightened the screws the right amount, a month will show: change nothing and watch. If, with the listing unchanged, registrations hold or grow and installs don't crater — the gate cut off only those who were never going to pay, and I can calmly move on to emails and credit top-ups. But if installs start sliding down along with activity — I tightened too early, and I'll have to roll back rather than hunt for excuses in the stats.

So the plan for the near future is a strange one for me: do nothing on Call Copilot. Just observe and let the statistics accumulate. We'll see how long I last.