Hello! This week's email about building Lotus is really exciting for me. First, I renamed Weekend Mode feature to Do Not Disturb and built it. But more importantly, I migrated from Redux-like state management to storing all the data in a SQLite database. There's lots to talk about, so let's dive right in!
You may remember that I've already talked about Weekend Mode feature in one of the earlier newsletters. Now it's called Do Not Disturb to make it sound more generic, because I decided that it would be useful to ignore work-related notifications after 5pm as well and not just on weekends.
To set it up all you have to do is to add an organization or a repository to Do Not Disturb list and notifications from these will be silenced.
GitHub's API isn't as flexible as I hoped it would be, so this functionality is a combination of three separate searches:
All of them are cached as well to avoid over-fetching and potentially hitting rate limits on the GitHub's side. After you add a repository to the list, this is how the settings look:
Icon on the left indicates whether this is an organization or a repository. Note that
pastel are my repositories and Lotus is kind enough to remove my username from the label.
When Do Not Disturb is activated, you'll also see a neat moon icon on the Inbox page with a tooltip that explains which notifications are hidden.
I'm quite happy with how this feature turned out, but there are still to-dos to make it complete:
Until this week, Lotus had a primitive solution to data storage - a big JSON blob saved in the file and synced back and forth between main and renderer (UI) processes. In the UI, state was managed in one huge
useReducer React hook, which is basically Redux.
It let me prototype Lotus quickly and change how data is stored immediately. However, I realized that it won't serve me well long term and that JSON blob will only get bigger over time. That's why I decided to migrate to a proper database - SQLite.
SQLite is a variant of a SQL database where all of its data is stored in a single file, making it perfect for macOS apps. I use better-sqlite3 library to interact with SQLite database and I'm impressed with out-of-the-box performance and how easy it is to use.
I'm glad to say that after this week's work all Lotus data is stored in a local SQLite database! Data is no longer kept in two places and synced all the time. Now it's stored in a database and UI only pulls what it needs and when it needs to, making the app more efficient as well.
Because SQLite is still a SQL database, it means there has to be a schema for all the data I want to store. This presents an interesting challenge of database migrations. For example, Lotus v1.0 has tables
users, but Lotus v1.1 requires a
do_not_disturb table to exist. Now Lotus needs to detect this case and "upgrade" your database to a new schema.
Thankfully, I've already built a custom library for database migrations in my other project Linkjar, so I just extracted it from there and added into Lotus. Now all data migrations are automatic between Lotus versions, which is quite cool.
Now that the biggest technical roadblock is out of the way, I can focus on building the actual user-facing functionality. Next week I want to finish Do Not Disturb feature and start implementing the Screener. So excited for this one!
Granted, these libraries made my life easier, but I still think it took way too much time than it should've. Same thing applies to quite a few of other areas of Lotus codebase, so I keep thinking about extracting stuff and releasing it as open source. Other potential candidates for extraction are:
I'm sure there will be more as I move forward. While I'm not going to extract and release these now to ensure I'm not distracted from Lotus, I think it fits my goal of sharing knowledge as I build this whole thing.
I remember thinking "what can I write about" or "what is something I know that I could share" and having a hard time answering these questions. I even have the same thoughts when I write this newsletter every week! Well, it's so much easier to answer these as you're doing something. Turns out the recipe is simple - build, learn, share, repeat. Whether you actually stick to that routine is another topic, but at least the steps are there for you.
Thank you once again for sticking with me here! I hope you enjoy seeing small incremental steps in building Lotus and watching it become a "grown-up" app.
Until next week!
P.S. You can now read past newsletters on the getlotus.app website.
I'm building Lotus in the open and I'm sending out progress updates just like this one every Sunday.
I won't send spam and you can unsubscribe anytime.