Setting up distribution and automatic updates

Two weeks ago I started inviting people to a first beta of Lotus, so I needed to figure out how to package the app so that others can install it. I also wanted to have automatic updates from day one, because I don't want to spam everyone and ask them to re-download the app.

Both distribution and update system took more time than I would have hoped. I think there's a lot of room for improvement here for Electron apps and this week I want to share what obstacles I have found and some ideas for fixing them.

Distribution

There are several tools available that can package Electron apps to make them installable on other machines. I went with Electron Builder, because its configuration seemed easier to understand than the alternatives.

My distribution workflow is simple:

1. Generate a production build of Lotus.
2. Package it into .app and .dmg files using Electron Builder.
3. Copy & paste .dmg file into getlotus.app website hosted on Vercel.
4. Deploy website with a new app version in a public folder.

It's not automated, but it works and I get great download speeds, because Vercel's CDN makes Lotus available from multiple geographical locations. That way your request is routed to the nearest server and it takes just a few seconds to download it.

I chose .dmg format, because it ensures Lotus is installed into "Applications" folder by showing a minimal UI for dragging Lotus icon into that folder.

It's possible to customize the look of that UI, so I'm planning to polish it up soon, because the default design doesn't look pretty at all.

While building an Electron app for production sounds simple, it's actually quite time consuming in practice.

First, Mac app must be assigned a certain category, like "Productivity" or "Developer Tools". Category is identified by a string like public.app-category.developer-tools in package.json:

{
	"build": {
		"mac": {
			"category": "public.app-category.developer-tools"
		}
	}
}

This part of config was odd to me, as someone who hasn't built Mac apps before.

• I need to look up available category IDs in Apple's docs, but first I need to find a link to that in Electron Builder's docs.
• Format is not human friendly, it could just be developer-tools instead of public.app-category.developer-tools.
• Electron Builder could offer a wizard CLI like create-react-app, which would list available categories and just let me pick one.

Then, turns out I also need to add darkModeSupport option to let macOS know that Lotus supports dark mode. It's weird, because it works without it in development, but it's needed when distributing the app.

{
	"build": {
		"mac": {
			"category": "public.app-category.developer-tools",
			"darkModeSupport": true
		}
	}
}

Another painful part of the build config is entitlements. This is where you describe which permissions your app needs. Turns out Electron apps require a few custom entitlements to start up and they're not specified by Electron Builder by default.

Entitlements are set in an XML file and they also look like category IDs - com.apple.security.cs.allow-jit. After Googling what do these mean and which ones does Electron need, I just stuffed these in without knowing too much about what they do:

• com.apple.security.cs.allow-jit
• com.apple.security.cs.allow-unsigned-executable-memory
• com.apple.security.cs.disable-library-validation
• com.apple.security.cs.debugger
• com.apple.security.network.client
• com.apple.security.network.server

I wish good defaults for entitlements were set by Electron Builder so that developers don't have to figure it out on their own.

Last but not least - notarization. Apple requires apps that are distributed outside of Mac App Store to be signed using a special "signature", so that macOS knows it's safe to run them. The problem is you have to do it for every app build and it takes about 3 minutes for Apple to sign it.

Notarization is also not built into Electron Builder, so there's a separate script that handles it via electron-notarize package. Fortunately, there's a great tutorial about implementing the whole thing.

Automatic updates

Electron has a nice built-in API for updating your app - autoUpdater. It repeatedly checks my server to see if there's a new version available, downloads it and replaces the binary. That server is powered by Hazel. It finds the latest version by looking at releases inside a GitHub repository, which is quite convenient.

When Lotus detects there's a new version available, it shows a dialog asking user to restart the app to update.

This feature was mostly effortless to implement, except a few issues.

First, Electron Builder corrupts application's signature when it creates a .zip file with an .app file in it. Then, macOS can't verify that it's notarized and won't let the app start. To work around it, I need to manually wrap an .app file into .zip and upload it to GitHub as a release artifact. This is a known issue, but it's not fixed yet.

Second, Electron doesn't provide a default UI for asking user to update and it doesn't add "Check for Updates" item to the menu.

Would be nice if it handled a state when update is being downloaded in the background too.

Third, both Electron Builder and Hazel don't support builds for Macs with M1 chip.

Fourth, there's no way to roll out an update incrementally in Hazel. For example, deliver an update to 5% of users, then 50%, and then to all 100% of them. It's useful to make sure critical bugs are caught before it spreads to all Lotus users.

Fixing the pain

There's too much friction in distributing and updating Electron apps. That creates an opportunity to try and fix it for myself and others.

There's already a great example project that solved the pain of publishing npm packages. It's a small utility called np by Sindre Sorhus. It runs a variety of checks to ensure you're publishing code that works, bumps the version, publishes the package on npm and it even creates a GitHub release.

It would be lovely to have something like np, but with checks specific to Electron apps. For example:

1. Bump the version.
2. Check that native dependencies are correctly set up and rebuilt for Electron's Node.js version.
3. Build a production-ready app with minified code.
4. Sign & notarize the .app file.
5. Verify that app can start.
6. Create a new GitHub release.
7. Generate .zip and .dmg "wrappers" and add them to that release as artifacts.
8. Optionally set a rollout percentage.

As for building the app for distribution, it would be great if it was just one command. When executed for the first time, it would ask a series of questions and generate a configuration for Electron Builder based on my answers.

It would also take care of generating builds for Macs with Intel and M1 chips without me setting up anything. Then update server would determine which build should you download depending on which chip you've got.

I think having such tools would increase confidence in releases and remove the need to set up the same boilerplate for every Electron project. I'm going to take a Basecamp approach with this and build this tooling for myself first, tune it as I go, then release it on GitHub for everyone to use when it's solid.

Hope you enjoyed this deep dive into Lotus internals this week! See you next weekend.

– Vadim.