Technology

The Cacophony Project is built with a wide variety of technologies including embedded systems, web applications, backend services and machine learning pipelines. Let's take a look at the various components of the project in a little more detail.

Remember, the project is completely open source. This means that anyone can contribute.

Thermal video image with classifier identifying a hedgehog

The Cacophonator

The Cacophonator name refers to our rapidly evolving embedded platform. Equipped with a thermal camera, speakers and sensors it is deployed into New Zealand's native bush to automatically lure, identify and eliminate invasive predators. The Cacophonator software is a combination of Go and Python code.

Base hardwareRaspberry Pi 3
Thermal cameraFLIR Lepton 3
Additional hardwareCustom interface hat
ATtiny microcontroller
3G modem
USB audio adapter
Real time clock
Operating SystemRaspbian
Key Softwarelepton3
thermal-recorder
thermal-uploader

If you're interested in embedded systems, whether it's getting computationally heavy tasks to run efficiently on small computers, or making embedded systems weatherproof and robust, helping to improve the Cacophonator might be for you.

The internals of a Cacophonator
A Cacophonometer powered by a solar panel

The Cacophonometer

This is where the Cacophony Project started. The Cacophonometer is an Android application which wakes up at regular intervals, records audio and then uploads it to the Cacophony Project API server for analysis. Among other things, the recordings can be used to make estimates of bird population health.

If you're an Android or Java developer you may wish to contribute to the Cacophonometer app.

API Server

In many ways, the API server is the heart of the project. It's where all video and audio recordings for the project are stored, processed and then made available for later querying and download. All the components of the project interact with the API server in one way or another.

PlatformNode.js
DatabasePostgreSQL
Object StoreMinio (S3 compatible)
Test infrastructureImplemented in Python
Source codecacophony-api

Web Application

The Cacophony Project has a full-featured JavaScript web application for querying, tagging and managing uploaded recordings. It interacts exclusively with the Cacophony API server.

If you're into web development or design, we'd love your help to extend and improve our web application.

Machine Learning

Automatic classification of predators in a thermal video feed is a crucial part of the project. This magic is implemented using a TensorFlow based machine learning model which is trained by our classifier pipeline. We make heavy use of a number of data science tools including NumPy, SciPy, OpenCV and HDF5.

If you're into data science or machine learning, maybe you could help us improve our classifier's accuracy and efficiency.

Thermal video with the classifier identifying a hedgehog

Getting Involved

Every software and hardware component of the Cacophony Project is open source and free for anyone to look at, improve or use elsewhere. If you're experienced with any of the technologies we use, or are willing to learn, we'd love to hear from you.

Here's a few ways to get started...

Taiga

The core team breaks up development work into two week iterations. For a view of what we're currently working on check out our kanban board on Taiga.

For a view of some of the big ideas for the project and what we're working towards check out the list of "epics" on Taiga.

If you spot something on the board that interests you, it's a good idea to talk to us via one of the other communication methods listed here before making a start. Some projects will require specific hardware and we may be able to help with that. Again, talk to us and we'll sort something out.

Github

All of the project's source code is hosted on Github under the "TheCacophonyProject" organisation. You'll find that each repository has an up-to-date list of issues for things that need to improved or fixed. Some of these are small tasks and some are major undertakings.

A great way to get your feet with the project is to look for issues labelled " good first issue".

Mailing list

The project has a mailing list hosted by the New Zealand Open Source Society (NZOSS). You can use the mailing list to ask questions and keep up with project news.

Rocket.Chat

The project's developers coordinate their day-to-day work using a Rocket.Chat instance (again, kindly hosted by the NZOSS). This is a great way to directly contact the project's developers. You'll need to register a (free) account for yourself.

Twitter

The development team regularly use Twitter to announce details of their work using the #cacophonydev hashtag.

Email

If you'd just prefer to send an email straight to the development team, just use dev@cacophony.org.nz