Things I learned from my SaaS project’s private beta period

My new SaaS startup https://checklyhq.com was in private beta for about two months. I just moved to “regular” beta! Yay for more betas! The private beta had two goals:

1. Technical: sanity check on the code and infrastructure.
2. Business: Get initial user feedback on features and usability.

Here’s how it went and what I learned.

Getting signups

I opened up a Mailchimp mail list and added a subscription box to Checkly’s homepage. My goal was to get 💯signups and then start sending invites. 100 does not sound like a lot, but given my budget and timelines I thought it would be enough to give the infrastructure a workout and get enough initial feedback to move to the next stage.

Homepage with beta signup

Inviting someone to sign up for a free private beta is vastly different than asking money for it. That’s why I wanted the private beta to be short and sweet, because it does not really tell me anything about the validity of the business.

To get actual signups, I used the following methods. Disclaimer: I had 0.0 social media following or any audience to speak of. One of my Github projects has 69 stars ✨…

1. Social Media & Blogging

My first strategy was to start blogging about building Checkly. Exactly as I am doing right now. Two blog posts did pretty well: one syndicated through Hackernoon and one via Free Code Camp. This generated by far the best stream of signups. The one about Vue.js did best.

Five things I learned building a Saas App with Vue.js

!! extra free 1337 growth h4x !! I spend quite some time on the blog posts. Most of ‘m are about a 9 minute read. They come with a lot of custom images and diagrams. They come with a nice colourful hero image, courtesy of my Art History degree. No cheap click bait. Apparently people like that. Shocker. 😱

2. Meetups

I arranged to be a speaker at the Vue.js and Node.js meetups here in Berlin. Specifically the Node.js meetup was a big success. Major turnout, lot’s of interest from developers from the Berlin developer community. Attributing signups is a bit hard, but my gut says the effort was worth it. It was also just fun to do.

!! extra free 1337 growth h4x !! I was very sneaky by either turning a blog post into a talk or vice versa. Two birds with one stone & rather lazy than tired!

3. Free tools

I built a spin-off tool from Checkly at https://puppeteersandbox.com. Checkly uses the Puppeteer Node.js framework under the hood to automate browser interactions so most of the infrastructure was already in place. Puppeteer sandbox is like JsFiddle or CodeSandbox, but for Puppeteer Node.js scripts.

body[data-twttr-rendered="true"] {background-color: transparent;}.twitter-tweet {margin: auto !important;}

Did a small (big) update to the puppeteer sandbox runner I had. It's now on the https://t.co/x9m8vKuZqo domain and you can save your scripts and embed them in Markdown and HTML, kinda exactly like JsFiddle 🎉🦋🍍 @ChromiumDev

 — @tim_nolet

function notifyResize(height) {height = height ? height : document.documentElement.offsetHeight; var resized = false; if (window.donkey && donkey.resize) {donkey.resize(height); resized = true;}if (parent && parent._resizeIframe) {var obj = {iframe: window.frameElement, height: height}; parent._resizeIframe(obj); resized = true;}if (window.location && window.location.hash === "#amp=1" && window.parent && window.parent.postMessage) {window.parent.postMessage({sentinel: "amp", type: "embed-size", height: height}, "*");}if (window.webkit && window.webkit.messageHandlers && window.webkit.messageHandlers.resize) {window.webkit.messageHandlers.resize.postMessage(height); resized = true;}return resized;}twttr.events.bind('rendered', function (event) {notifyResize();}); twttr.events.bind('resize', function (event) {notifyResize();});if (parent && parent._resizeIframe) {var maxWidth = parseInt(window.frameElement.getAttribute("width")); if ( 500 < maxWidth) {window.frameElement.setAttribute("width", "500");}}

I tweeted it around and mentioned it in Slack groups dedicated to this subject. There was even a small pop-up in the corner that would prompt people to sign up for Checkly. It has/had some usage, but I really need to put more effort into it as I’m planning to keep this free tool around for some time.

4. Beta list

I’m on the fence about Beta list. I signed up, paid for the most basic plan. I got some traffic and some signups. I’m however a bit skeptical about the value of their traffic.

body[data-twttr-rendered="true"] {background-color: transparent;}.twitter-tweet {margin: auto !important;}

Checkly: Active monitoring for developers and startups https://t.co/2Y3ofzkD2m

 — @BetaList

function notifyResize(height) {height = height ? height : document.documentElement.offsetHeight; var resized = false; if (window.donkey && donkey.resize) {donkey.resize(height); resized = true;}if (parent && parent._resizeIframe) {var obj = {iframe: window.frameElement, height: height}; parent._resizeIframe(obj); resized = true;}if (window.location && window.location.hash === "#amp=1" && window.parent && window.parent.postMessage) {window.parent.postMessage({sentinel: "amp", type: "embed-size", height: height}, "*");}if (window.webkit && window.webkit.messageHandlers && window.webkit.messageHandlers.resize) {window.webkit.messageHandlers.resize.postMessage(height); resized = true;}return resized;}twttr.events.bind('rendered', function (event) {notifyResize();}); twttr.events.bind('resize', function (event) {notifyResize();});if (parent && parent._resizeIframe) {var maxWidth = parseInt(window.frameElement.getAttribute("width")); if ( 500 < maxWidth) {window.frameElement.setAttribute("width", "500");}}

It seems like their tweets to 50k+ followers fall on dead ears and are just retweeted by bots about 160 times. Exactly for every product with a variance of about 5/10 tweets. I assume the best intentions from Betalist’s side. I did however opt for the refund which they gave without any fuss.

4. Twitter Ads

I spent €100 on Twitter ads. I got a ton of impressions and quite some clicks: that’s the easy part. I got about 3 signups.

Twitter’s ad dashboard

This is probably due to me being a complete noob with regard to targeting, segmenting, timing, ad copy etc. etc. Props to the Twitter team for making the tools themselves easy to use and quite powerful. This is something I need to revisit at a later stage.

In the end a bit over 200 people signed up. This was more than the 100 I aimed for! Turns out, 200 people signing up for your beta does not mean all 200 actually log in and start using your app. Shocker ! About 110 people actually logged in and did stuff in the app, so about a 50% drop off rate

Google Analytics showing signup attributionSignups via Mailchimp

Getting Advice

As I’m “flying solo” I really had the urge to level with someone in the field that has more experience on running a developer-oriented SaaS company. I reached out to TravisCI founder and ex-CEO Mathias Meyer and he gracefully agreed to meet over coffee so I could pick his brain. Mathias was incredibly friendly and supporting and dispensed some great tips about running betas and all kinds of other stuff. Since our chat, Matthias has actively reached out to founders to contact him for advice or just a chat

body[data-twttr-rendered="true"] {background-color: transparent;}.twitter-tweet {margin: auto !important;}

Meanwhile, if your company or team is looking for help to get over its next (or the first!) growth hurdle, or you're an individual founder or entrepreneur looking for someone for coaching or mentoring, my DMs and emails (mathias@paperplanes.de) are open.

 — @roidrage

function notifyResize(height) {height = height ? height : document.documentElement.offsetHeight; var resized = false; if (window.donkey && donkey.resize) {donkey.resize(height); resized = true;}if (parent && parent._resizeIframe) {var obj = {iframe: window.frameElement, height: height}; parent._resizeIframe(obj); resized = true;}if (window.location && window.location.hash === "#amp=1" && window.parent && window.parent.postMessage) {window.parent.postMessage({sentinel: "amp", type: "embed-size", height: height}, "*");}if (window.webkit && window.webkit.messageHandlers && window.webkit.messageHandlers.resize) {window.webkit.messageHandlers.resize.postMessage(height); resized = true;}return resized;}twttr.events.bind('rendered', function (event) {notifyResize();}); twttr.events.bind('resize', function (event) {notifyResize();});if (parent && parent._resizeIframe) {var maxWidth = parseInt(window.frameElement.getAttribute("width")); if ( 500 < maxWidth) {window.frameElement.setAttribute("width", "500");}}

!! extra free 1337 growth h4x !!: Ask advice from someone in the field. Asking for advice is a very different than trying to sell something. This is why they will probably agree to meet you if you’re not a total psycho.

Getting User Feedback

Prior to launching, I installed the Incoming Feedback widget from Hotjar. The signup email everyone got made explicit mention of this and encouraged people to use that widget for any bugs, wishes, thoughts or whatever.

Welcome email explicitly asking for feedback

This worked fantastic. I got 50+ pretty detailed responses. Many users responding included their email address (otherwise it’s anonymous) and I made sure to follow quickly with, whether the response was bug report or a feature suggestion.

Hotjar Incoming Feedback responses

What’s nuts is that at least five times users suggested a feature, addition or tweak that I literally had on my to-do list (a simple Trello board) but wasn’t sure about. This helped a lot with reprioritising my roadmap and I even pumped out some tiny additions within days of receiving the feedback.

Things I didn’t do

I’m an engineer so I agonised over so many things before letting people into Checkly. Some things were actually important, but it turns out most things are not. I hate to regurgitate startup lore, but “doing things that don’t scale” and “you’re probably not gonna need that” are very, very true. So, here’s a short list of things I didn’t do:

• A real “invite” system. I just mass emailed all signups the exact same login link.
• Plan restrictions. everyone can use everything forever. I’ll deal with time- or feature limits later.
• Incorporation. I didn’t ask for money, there is strictly speaking no business deal. Not yet of course. I’m working on that part as we speak.
• CI/CD, automation etc: code was pushed using an NPM script or Heroku. I logged into servers via SSH to check when something went wrong.

If you liked this article, please show your appreciation by clapping 👏 below or follow me on Twitter! But wait, there’s more!

I’m building an active monitoring solution for dev & ops team at https://checklyhq.com. Sign up for a free 15-day trial

Things I learned from my SaaS project’s private beta period was originally published in Hacker Noon on Medium, where people are continuing the conversation by highlighting and responding to this story.