This is the blog post form of a presentation given at Rust London - 27 April 2021.

A video of the talk is available on youtube, and slides are available in the project’s repo.

The slides are written in markdown using remark, and this blog is also markdown. Let’s see how well this translation job goes.

Outline

Backstory

We started with a few ESP32 dev-boards like this:

These cost around US$16 each, and don’t last more than about a day on battery power.

ESP32 is a super-cheap system on chip with bluetooth and wifi, but dev-boards will always be more expensive than commercial off-the-shelf hardware.

During lockdown, we were setting around the dinner table, and I asked my housemate “Wouldn’t it be nice to have a hundred temperature sensors? What could we do with that many sensors?”

So we bought 20 of these, at $3 each, and hooked them up to the internet.

System Overview

This is what we built:

Let’s take a look at the decisions we made, how they turned out.

Rust

We picked Rust because we were both starting to use Rust for work, so using Rust for a personal project was a good opportunity for learning, for both of us.

Andrew is working on crosvm and Virt Manager for Android.

I was using Rust for the backend of the FutureNHS project (I have since worked on other Rust projects at Red Badger, and moved on to work at Tably.com, with backend and frontend written in Rust).

It was also a good chance to work on something together during lockdown.

I also found a blog post describing how to connect to these sensors with Rust. This gave us the burst of momentum that we needed to start the project, but in the end, we outgrew its initial structure, and made our own project.

MQTT

MQTT is the pubsub of choice for low-powered gadgets.

Homie is an auto-discovery convention built on MQTT.

In Rust, the rumqttc library is pretty good:

  • It works using channels, which is a nice interface.
  • Andrew has submitted patches, and they were well received.

Rust Bluetooth in 2020

The state of Rust Bluetooth in 2020 was a little underwhelming. The options were:

  • blurz - “Bluetooth from before there was Tokio”
    • We started with this.
    • Talks to BlueZ over D-Bus, but single-threaded and synchronous.
    • Blocking device.connect() calls. 😧
    • Unmaintained (for 2 years).
  • btleplug - “cross-platform jumble”
    • Theoretically cross platform, but many features not implemented.
    • Linux implementation needed root access.
    • Too many panics for us to use.

Aside: Concurrency

  • The main problem with blurz was that it exposed a single-threaded blocking library interface:

We realised that there was a third approach:

  • dbus-rs - aka “roll your own BlueZ wrapper”
    • We could generate a “-sys” crate from D-Bus introspection, using the tools provided by the dbus-rs project.
    • The dbus-rs codegen produces syncronous or async interfaces, so you can pick whichever approach you want.

After switching to an async library, we got:

This almost solves the problem, but not quite. In our case, everything lives in a big Arc<Mutex<GlobalState>>.

The solution is to hold the Mutex for as little time as possible.

This is much better.

These are the concurrency tools that we use:

  • Arc<Mutex<GlobalState>

    • Used for all of our state.
    • Easy refactor from &mut GlobalState.
    • Fine as long as you know where the lock contention is.
    • Only hold the mutex when you need it, be careful of await points.

  • Unbounded Channels

    • Used for all bluetooth events, and all MQTT traffic.
    • Fine if you know they’re not going to back up.

  • Stream<Item = Event>

    • Used as the consumption API of the Channels.
    • Just the async version of Iterator.
    • map(), filter() and select_all() are easy to use.

Bluetooth Developments

We ended up building on top of our “-sys” Bluetooth library, and created: bluez-async

  • Linux only
  • Typesafe async wrapper around BlueZ D-Bus interface.
  • Sent patches upstream to dbus-rs to improve code generation and support for complex types.
  • Didn’t announce it anywhere, but issues filed (and a PR) by two other users so far.

Andrew has also been contributing to btleplug

  • Ported btleplug to use bluez-async on Linux.
  • Exposes an async interface everywhere.
  • There are a few bugs that need fixing before they make a release though.

Results

We now have graphs like this, with inside and outside readings:

and readings from our fridge:

and we can plot trends using Pandas and Plotly:

Will’s setup, with MiFlora sensors

I gave some to my workmate:

so you can tell when Will waters his plants:

and when the dehumidifier kicks in in the cellar:

CloudBBQ

We also got it working with a meat thermometer (backstory: one of the people who sent us patches was using it with a bbq meat thermometer, so I bought one for Andrew as a joke present):

so now we have a graph of our roast:

Closing Remarks

Separating things into layers (and crates) worked well:

  • App (mijia-homie) -> Sensor (mijia) -> Bluetooth (bluez-async) -> D-Bus.
  • App (mijia-homie) -> Homie (homie-device) -> MQTT.
  • MQTT -> Homie (homie-controller) -> homie-influx -> InfluxDB

Deployment

  • Everything is supervised by systemd.
  • Built with Github Actions and cross, packaged with cargo-deb.
  • Test coverage is a bit thin (blame me for this).

One major limitation is that the Raspberry Pi only supports 10 connected BLE devices (10 « 100). The way to get around this problem would be to make the mijia sensors include the temperature and humidity data in their advertising broadcast packets, and then passively listen to them on the raspberry pi. There are a handful of projects that provide flash custom firmware for the mijia sensors, and many of them let you do exactly this. If anyone has done this to their sensors, we would be really interested to hear from you. Adding support for reading sensors in this way would allow us to deploy many more sensors, and would also drastically reduce how many cell batteries we go through in a year.