Updates on nest.pijul.com

Sunday, July 2, 2017
By Pierre-Étienne Meunier

As nest.pijul.com is slowly stabilising, I’d like to blog about a few lessons learnt during its development so far.

Problem statement

Nest.pijul.com contains servers for three different protocols: SSH, HTTP and HTTPS. I’m currently the only author, and I don’t have too much time to dedicate to this. Also, I’m pretty bad at system administration/devops, and I don’t particularly enjoy it.

I was also interested in trying out Tokio when I started, because it is fun, fast and reliable. It also solves a number of security issues easily: for instance, timeouts don’t require their own thread, which makes it harder to attack the server.

My current stack

There weren’t many things ready in Rust when I started. This is my current stack today, including things I had to write:

#[sql("SELECT id, is_active FROM users WHERE login = $user")]
pub struct UserReq {
    pub user: String,
    pub is_active: bool,
    pub id: Option<Uuid>

Note that this philosophy is the opposite of ORMs like Diesel, which are great, but seem a little scary when it comes to scalability: are all my queries writable in Diesel? Will I need to rewrite all my code if not?

One of the reasons of my skepticism is that some of my more complex queries for nest.pijul.com use on the order of five JOINs, and some need three intermediate tables built using WITH. Also, I usually test my queries in the postgres client to decide what I want.

And for system administration?

Securing a system is hard. There are many components, sometimes moving fast, sometimes with breaking changes. There are several configuration files for each of them, with different syntaxes. Except when this is your main job, these files tend to break at the moment you’ve completely forgotten their syntax. Configuring and installing a new machine is not atomic, and not reversible.

Fortunately, I use a tool called NixOps to solve all these problems. Nix is a small programming language to describe machine configurations. Every deployment uses a form of copy-on-write, which means that deployments (new packages/configuration changes) are atomic.

There is no container, and it is super easy to test a future deployment locally.

The only problem was, in order to guarantee reproducibility, the interface with Rust had to recompile everything from scratch with every deployment. Even a small mistake (for instance a new file I would forget to add in a commit) would cause a full rebuild, in release mode, of all dependencies (which would take about half an hour).

This is why I wrote a tool called Nix-Rust to reuse already built crates, and even share them between projects. Also, I use Nicolas Pierron’s overlay to get different version of Rust. And I use nightly, initially to get impl trait, but now also for my SQL syntax extension.