It is 2022-07-07 today. Two days ago I’ve stupidly and accidentally lost literally all data from this server including this blog and some pet projects.
There was no regular backuping. All backups I had at the moment was dated may which is better than nothing at all. Well, either I became very tidy in my tech skills or life still teaches me nothing ¯\_(ツ)_/¯
I set up server from scratch and deployed my projects again from those backups. It took me about several hours during two days. Unfortunately, I absolutely don’t remember exactly what has been permanently lost. Shit happens.
Before that accident I didn’t bother with backups: I already have some ones, server is (almost) always stable and fine, bills are always paid in time, what may happen? This was wrong laziness.
This time I was lazy properly, as any tech guy should: if you don’t want to spend time on boring routine — automate it and don’t spend.
Since my VPS deployed without any containers-n-shit, my solution will be simple too: bash + mysqldump + gzip + rsync + crontab + second VPS as reserve remote storage.
This second you maybe already understand what I will do and talk about because our grandfathers already
At this second, you maybe already understood what I’ll do and tell and decided to close the tab, because even your grandfather didn’t do backups like that. Well, I don’t give a shit and anyway will tell my solution and attach gist which is ready to use.
Notion is very cool application to organize any info, team work or plan anything. It has simple but powerful frontend to solve your needs and supports markdown format.
Obsidian is mainly knowledge base builder based on filesystem, excelent themes and plugin support and fast crossplatform apps.
Both are proprietary, free for personal use and have own pros and cons. But also they have key differences which doesn’t allow them to replace each other painlessly.
I personally use Notion as PKM (personal knowledge management) system. But I’m curious how can I try to take 100% control of my data without losing most of UX I’m used to. So I found Obsidian as possible alternative.
First difference is how notes are stored and the concept of their hierarchy. Notion’s ones are pretty abstract, every page is kinda “folder” itself since it can have subpages and so on like it’s implemented in Confluence. In Obsidian every page is md file which can be deeply inside valet subdirectories, so directory itself is not a page and hasn’t any ‘main’ root page.
When you export your workspace from Notion you get your data in zip-archive containing in fact the same file structure with directories and files that would be suitable for Obsidian but with some important nuances.
Every file has its own UID — 32-length string of digits and letters. It’s specified in every filename and page link inside page. You can see those UIDs while navigating between pages in your Notion.
Tables (databases) are just CSV-files.
Every file attached to page stored in the same dir where page itself is stored.
Of course, you can try to use this data as vault in Obsidian. But these three points are totally inacceptable if you want to work effectively.
1) UIDs are bullshit. Obviously, they are only used in Notion to support multiple pages with the same name. Since Obsidian is based on a file system, the fs itself controls names. They only lengthen page names, hinder their reading and just looks ugly. Main problem here is that these UIDs are litteraly everywhere and it’s pretty non-trivial to get rid of them.
2) What about CSVs, you gotta
convert them to md tables and inject them right into pages
or convert into separate md file and edit the original link to it in the page.
I bet it’s absolutely impossible to implement correctly on both ways because in most cases they will be parsed wrong: nothing limits you or your teammate to write commas anywhere inside. That’s sucks, table definitely shouldn’t be presented as CSV-files.
3) When you try to move assets or pages into another folder you also need to update all the links to them in page. Every page.
4) And on top of that, not all pages saved with the Notion Webclipper will exist at all, unlike links to them.
Well, seems like we have to try to fix this shit in order to use it somehow.
2) Then the simplest way is to use one of community converters because Obsidian can convert data only from Zettelkasten and Roam.
First time I was encouraged to develop my own converter in golang but then I decided to use N2O which requires node.js (damn) but pretty good anyway. Or (if you wanna some hard sex) you may use one of community recipes to convert your workspace in manual or semi-automatic way (e.g. this one). Keep in mind that none of converters is silver bullet and you can meet problem with your vault any moment.
3) Part of Notion functionality available in Obsidian only with plugins. Good point there is that plugins are crossplatform and are always stored inside vaults. I listed below some 3rd-party plugins I used to at least partially cover missed functionality.
Obsidian Auto Link Title (github, obsidian) — synchronizes page name with first h1 and vice versa.
Banners (github, obsidian) — provides emojis as page icon and pictures in header by local file or http(s)-link. Basically it looks good but emoji icon is visible only on a page and banner positioning conflicts with code blocks plugins.
CSV Editor (github, obsidian) — makes possible to view and edit CSV files as tables (not md) with sorting. Remember that they can be parsed wrong.
Homepage (github, obsidian) — adds new button in sidebar to navigate to “home” page of vault quickly which is specified in plugin’s settings. Home page is usual md file you edit manually.
Table of contents (github, obsidian) — adds new command to generate ToC inside page by its headers.
Icon Folder (github, obsidian) — adds new command to change folder or page icon visible in main sidebar (vault tree). Remix and Font Awesome packs are fully available.
Also you can turn on at least the ‘Starred’, ‘Slash commands’ and ‘Command Palette’ core plugins.
What about attached files — well, while I experimented on all this plugins, converters, manuals and shit I just gave up and leave them as fucking is.
The second big part is to make your valet synchronizing between devices. Here comes the Syncthing. Basic short algorithm is:
Go to 127.0.0.1:8384, add your devices, add new folder and share it for them.
You can find detailed step-by-step manual here, it’s good enough to start.
So, now we have almost ready to use valet synchronizing between our devices in real time. Why ‘almost’? The main disadvantage of this hell is that sometimes you will meet unexpected behaviour such as creating new note instead of navigating to existed one or Or missing assets just because of broken links in page.
But what goals have we achieved and why you can use this bundle?
Full data control. Of cource, Obsidian is not classic FOSS but it’s built with electron. So you always can press Ctrl+Shift+I (desktop) and inspect what’s going on there.
Totally free. Obsidian provides own sync mechanism which will cost you $8/mo, so I was very surprised when I found out that Obsidian team silently allow community to make 3rd-party sync plugins or discuss this subject in forum.
No registration required. You always can register Obsidian account. But it’s not nesessary at all. Obviously, their applications are fully-functional for purposes described here and no registration or paid features needed.
Syncthing itself. This is FOSS with powerful community. This is standalone software: if something go wrong at least you’ll have a copy of your data on every device you sync. And this is a universal solution for synchronizing any data: ST will easily replace any cloud storage, as it uses secure connections through relay servers and, of course, you can set up your own one and send traffic through it.
Vendor-unlocked. You are still able use any other software you like such as git, vscode or pure cli editors, etc. to control, edit, view, share and sync your knowledge base.
In conclusion, I’d say that Obsidian is good for starting your knowledge base from scratch but not for migrating your current one from Notion. It’s not worth the time.
If you ask me if I will use this bundle I will answer: “Definitely not, and probably never will.” So why the fuck I wrote this article?
It was very curious to touch Obsidian itself.
I already have some data to experiment with in Obsidian, so generally I get good expirience if we don’t count several nuances comparing with Notion.
Just to show you one of possible ways to keep you ‘second brain’ more ‘yours’ and problems about it.
As for me, main motivators are simplicity and good-looking UI with wide and easy-to-access functionality. Notion provides all of this for me and I’m almost happy with it. Earlier I tried to find (and still looking for) some alternative but at this moment I still didn’t found anything to migrate to. This is not a goal but curiosity.
I like simple, intuitive and fancy UI and cool UX things and hate to perform dozen of geeky actions with git and different software to make basic things in non-work purposes cuz just tired of that shit. Such stuff comes in handy in my work, but in life I want simplicity.
My only serious complaint about Notion is speed and heaviness. The web version does not cache anything, the entire hierarchy and pages are sometimes loaded in tens of seconds. The mobile version (thier mob app is just the wrapper of it, btw) takes even longer to load, while the interface freezes. For me it’s just small cost I pay to get huge possibilities.
So, yes, currently I’m vendor-locked but if I were not — Obsidian with Syncthing could became my choice. Now I just don’t want to change my concept about how to maintain my “second brain” because I don’t need to bother about it.
Update 17.02.2022
I wrote a tiny tool to convert csv data to markdown table. Take a look, maybe it will be useful for you.
