Detecting Northern Lights using Raspberry Pi and Convolutional Neural Network - Part 1

Pi Camera and a Northern Lights image with prediction.

At the moment, I'm fortunate to live in a part of the world where visible Northern Lights are fairly common. The problem is though, they may only show up for a few minutes during the night so you have to either be a real night-owl or get lucky. I wanted to build something to help me catch the Northern Lights more often.

The idea mostly came after seeing the recent release of the Raspberry Pi camera module 3 and wanting an excuse to do something with it.

In this blog post I'll go over the proof of concept of the idea and how I hope to develop the project into something a little nicer.

I already had a bit of a head-start on the some of the infrastructure from a previous project of mine, a Raspberry Pi baby monitor. It's a two-part setup a camera-equipped Pi Zero W and a separate server component running on my home server. The Pi Zero is a 'dumb' piece that only takes pictures every few seconds and sends it via http, while all the business logic of collecting, storing, and displaying pictures sits on my server. This makes it easy to unplug and move the camera around without causing too much trouble to the whole system's operation.

For this particular project, my Pi Zero got a camera 3 upgrade and was pointed straight at the sky out my north-facing, upstairs window. It captures a picture every 5 seconds and sends it to my home server where it's stored for later; most nights capturing about 7000 images. Each image is stored in a folder and metadata about the image is written to a small sqlite3 database. To label the images, I was able to use my computer's image previewer to quickly scrub through a nights worth of images and mark them in the database as either containing Northern Lights or not. It only took about 30 minutes per night but it was pretty boring work none-the-less.

With labelled images, I could sort them into a proper training dataset of 'aurora' and 'sky' pictures. I then trained a large convolutional neural network built using the Keras Python package to classify the pictures. The Keras documentation has a nice article called 'Image Classification from Scratch' which was a good starting point for this task. It shows how to build a model to classify pictures as either dogs or cats, so it was fairly straightforward to adapt it to the task of classifying 'Aurora' or 'No Aurora'.

Below is a YouTube video showing a full night of particularly active Northern Lights. The 3rd number in the top-left of each image is the prediction from the trained neural network, the closer to 1 the number is the more confident the model is that the picture contains Northern Lights. None of the pictures in this timelapse were used in training the model. Skip ahead to 0:50 to see the Northern Lights in action!

With the concept proven, I'd like to build the whole project out a little bit more. The step 2 is to build a proper webapp that can receive pictures from the Pi, classify them, and display them. The best pictures (according the model) will get saved, and on really good nights it should send me a text message or some other notification. It will probably be a bit tricky to come up with a good heuristic for this, as sometimes Northern Lights will be gone within a few minutes.

Step 3 is to build a better enclosure for the Raspberry Pi so that I can move it outside to a more permanent location. I'm more of a software person so I haven't thought too much about this. If anybody has any god suggestions please let me know.

That's it for now. Thanks for reading. I'll try to put the code and training pictures on GitHub soon.


I've been playing a bit with Google Colab and thought it would be a good way to share the training code and data for this project.

Python Notebook:

Training Images (1.4GB):


Interactive Digit Classification Using Neural Network Trained on MNIST Data

Several years ago, I created a fully connected neural network from scratch in C as a learning exercise. I followed the first few chapters of Michael Nielson's book 'Neural Networks and Deep Learning' that I highly recommend.

The network was designed to train on the MNIST number dataset, which is a well-known dataset used in many machine learning examples. The goal is to identify hand-written digits as any number between 0 and 9. The final network performed quite well and achieved 97.14% accuracy on the test dataset. Not bad for a bit of matrix algebra wrapped up in some C code.

Anyways, ever since then I've had the idea to create a little browser widget to let people use the model I trained in an interactive way. Of course, I was beaten to the punch once, twice, and many more times I'm sure. But even still I wanted to see how well my model would perform at this task.

Homemode Fully Connected Neural Network

Before starting to work on the widget, I beefed up my neural network a little bit and was able to train one with a 98.2% accuracy on the test MNIST data. I then used a web framework called Svelte to create a drawing and predicting widget. Since my model is all simple linear algebra, exporting the weights from C and hard-coding them into Javascript was not too much work. Libraries like Math.js made it pretty easy to recreate everything. The final product is the widget you see below. It runs entirely client side in the browser using my trained neural network.

* does not run on iOS Safari, possible macOS Safari as well *

If you tried a few numbers, you probably noticed that the predictions can often be rather poor. I found that it has a really difficult time with '1's , '0's, and '9's. It was a bit disappointing, even with a 98.2% accuracy on test data, it still has a lot of trouble with new numbers. My guess is that, due to the fully-connectedness of the network, it has a difficult time generalizing new data. Like, if a '1' is off to the side or at a wrong angle that isn't present in the training data then it will predict incorrectly.

Keras/Tensorflow Convolutional Neural Network

Another type of network often used on the MNIST data is a convolutional network. I won't go into the topic here but this explanation was pretty helpful for my understanding. Convolutional networks work so well on MNIST that it's actually one of the 'getting started' examples for Keras.

I wanted to see how much the widget would improve with a convolutional network instead of my fully-connected version. So, I followed the Keras example and trained one in Python that reached an accuracy of 99.3% on the test data. Crucially though, I believe that it generalizes much better and is therefore more tolerant to digits that may not be presented in exactly the same way as in the training data. And the results definitely show, in my testing it seems to predict the correct digit much more often then my homemade model.

Again, the widget below is running entirely in the browser using the the Tensorflow.js library. Tenforflow.js allowed me to export the model from Python and import it directly into the Svelte widget.

* does not run on iOS Safari, possible macOS Safari as well *

Embedding Widgets

Because the widgets run entirely client-side, feel free to embed them anywhere on your own site using the code snippets below. They are web components that use a shadow DOM so should always look the same no matter where they are embedded. Kind of like frame, but for the modern age.

<script src=""></script>
<div><mnist-checker-widget /></div>
<script src=""></script>
<div><mnist-convolution-checker-widget /></div>

Full source code for this project is posted on my GitHub.


More Threading Possibilities in bittyhttp

bittyhttp is a C library that I've been working on that aims to make building web services in C as easy as possible. Check out my last post for a better description and to see some examples.

One use-case that I see for bittyhttp is adding HTTP functionality to existing C applications. For example, if there is a long-running process somewhere on a server, it would be easy to expose some of its configuration over a web interface using bittyhttp. So to target this use-case there are a few changes to bittyhttp to make this particular situation easier.

Users can now start the server in its own thread. For example:

main(int argc, char **argv)
    bhttp_server *server = bhttp_server_new();
    /* setup server, add handlers, etc. */
    bhttp_server_start(server, 1);
    /* continue with normal application execution */

    return 0;

This starts bittyhttp in a separate thread and returns immediately. So one could insert this at the appropriate place in an application to easily spin off an HTTP server. Then, when needed, a simple call to bhttp_server_stop and bhttp_server_free will shut down the server and free resources.

In this mode it is also possible to register new callback handlers while the server is running, adding even more flexibility. Overall, this is a small update but I think it makes it much easier to use and incorporate into other applications.

In other developments, I've also been experimenting with some sort of Lua interface to allow writing callbacks in Lua. I've actually used this in another project of mine but I'm not particularly happy with the implementation. So it's still squarely in the experimental category.

bittyhttp is distributed under GPLv3. However, if you're interested in incorporating it into your project, and need another license, feel free to reach out.

Thanks for reading!


bittyhttp - A Threaded Library for Building REST Services in C

bittyhttp is a new library that I've been working on that aims to make building web services in C as easy as possible. Microservices and HTTP APIs are very common these days and bittyhttp offers the ability to implement these in C without much hassle. It takes care of running the server so all the user needs to do is implement their callbacks.

When using bittyhttp, the user registers handlers to URLs with a callback function pointer. If an HTTP request is received that matches the handler URL, the callback is executed. Inside the callback, information about the HTTP request is exposed and allows the user to decide how they would like to handle the request. If no handler is found, bittyhttp defaults to acting like a standard webserver.

Check out the full project on GitHub.

A Quick Example

For instance, we can register a simple handler like this:

                            BHTTP_GET | BHTTP_POST,  // http methods
                            "/helloworld",           // pattern to match
                            helloworld_handler);     // callback function

And then implement whatever logic we need in the callback like this:

int helloworld_handler(bhttp_request *req, bhttp_response *res)
    /* business logic */
    bstr bs;
    bstr_append_printf(&bs, "<html><p>Hello, world! from URL: %s</p><p>%s</p><p>%s</p></html>",
    bhttp_res_set_body_text(res, bstr_cstring(&bs));
    /* add custom headers and response code */
    bhttp_res_add_header(res, "content-type", "text/html");
    res->response_code = BHTTP_200_OK;
    return 0;

Because bittyhttp uses a separate thread to handle each request, some care needs to be taken in the callbacks to prevent race conditions. Changing data that is not allocated inside of a callback will often require the use of mutexes or similar data structures. A database connection pool, for example, would need to be properly managed.

Use Cases

I see bittyhttp having 2 primary use cases. The first being to implement often-used API endpoints in C, for performance reasons. Perhaps authentication endpoints get hit a lot and you want to speed these up.

The second is adding HTTP support to an existing C application. Maybe you have an existing long-running application on a server somewhere and you want to expose some of its configurations via HTTP. In that case, bittyhttp would be an excellent option.

bittyhttp is GPLv3 licensed, and available on GitHub, but I would be open to relicensing it for specific cases. If you are interested in using it in your application, feel free to get in contact with me at [email protected].

squid poll

As a fun, little proof-of-concept, I created a site called squid poll. It's basically a Straw Poll clone that uses bittyhttp as its API backend. Feel free to visit the site to create your own poll, or fill out the one below:


Building a Video Baby Monitor with a Raspberry Pi and Wt

How it looks on the Raspberry Pi LCD touchscreen.

There are a lot of baby monitors out there on the market. But none of the ones with video really seemed like a good fit for us. Most either work over wifi, in which case who knows where else that video is going, or talk directly to a monitoring device, which then means keeping track of and charging another thing. I thought I could do a little better.

The task seemed straightforward enough: somehow get pictures from a camera pointed at the baby, onto my local Raspberry Pi server, where they could then be viewed by anybody with a web browser on the LAN. This way, everything stays local to our home network and we could use our smartphones to check in on the little sleeper.

I tackled the receive image and show it on a webpage side of the project first. For this I used the Wt C++ toolkit. Wt is kind of like Qt but for the web. It makes designing layouts and interfaces like building any other GUI app. It's also possible to create REST endpoints using Wt. This meant I was able have the ability to receive an image and have the viewing interface in the same application. This simplified things a lot since all the business logic could be placed in a single executable.

One cool side-effect of working with pictures is that it makes creating timelapses, like the one below, extremely easy. Watching a full night timelapse can then be pretty entertaining to see exactly how the little one ends up in such weird positions. Sometimes, the baby is most definitely not how you left it.

Timelapse shot using a few frames.

The second piece, the camera, is built from a Raspberry Pi Zero W with an infrared camera and a couple of IR LEDs, all mounted to a mini-tripod. All the camera does is take pictures every few seconds and send them to the local server, running the Wt application, via an HTTP POST request. Because all the important logic is in the Wt application, the camera unit itself remains relatively dumb. This is important so that the camera can be unplugged or moved without affecting the viewer application. No need to worry about pulling the power cord whenever.

Any web browser on the local network can access the viewing page. In practice this usually means a smartphone, but also sometimes another browser window set to 'always on top' when on a PC. The viewer shows the most recent picture it received and the timestamp that it was taken. The timestamp is important so it is easy to tell if something has stopped working. The viewer will also push the newest image over websockets to the browser and update itself without having to do any page reloads.

Overall, this turned out to be one of my more useful household projects. Both my wife and I have been using it to keep an eye on the little one while napping or in the evenings during bedtime.

Check out the full code on GitHub if you're interesting in the specifics.

I should also note, that this isn't - and shouldn't be - used as the main source of monitoring a child or baby. Our apartment is rather small so we are always in earshot. This is not a safety device! It's simply to check in on how the little one is sleeping.