Training our Machine Vision for Wallabies

Earlier in the year we were asked if we could expand the machine vision used with our thermal camera to automatically detect wallabies with the goal of monitoring and controlling the wallaby population. 

Data Collection

The first step was to collect some tagged videos of wallabies to train our model with. A camera was set up in the Willowbank wallaby reserve. Over the following months we collected over 2000 tagged wallaby videos.

Model Training

As an initial attempt to automatically detect wallabies we used our original model design with a Resnet architecture. This is a frame by frame approach with long short term memory (LSTM), where each frame is classified and the model maintains a history (LSTM) of previous frames to better understand the current frame.  This model was trained on all labels e.g. “bird”,”possum”,”wallaby” etc.

An example frame input:

This performed rather poorly and appeared to be overfitting in training. So we simplified the problem by reducing the classifications to "Wallaby or Not" and removed the LSTM from the AI model architecture. Also we changed our training process to transfer learn from a pre-trained Tensorflow Keras ResnetV2.  This gave us significant improvements but still performed badly with just 41% accuracy on our Wallaby Test set.

Movement

The poor results caused us to rethink our model.  By watching the wallaby clips and noting how a human can detect the wallaby in a video accurately, it was obvious that the wallabies have a unique hopping movement. In order to capture this in an AI model we adapted our approach to classify based on a sequence of frames. The idea is to capture the movement of the animal across the clip.

The examples above show 25 frames from a clip, the path the animal took over the whole clip, and an overlay of the entire clip.

By using this approach and changing our model architecture to ‘Inception’ we were able to achieve a test set accuracy of 94% on Wallaby clips and 90% on Not.  As our dataset is still fairly small this is likely to improve as we capture more and more wallaby data. We will continuously retrain this model to achieve a better and more robust AI classification.  We will also look into adapting this model to work for all animal classifications.

This technology is now available to anyone who is interested in wallaby monitoring. The capability of the technology will improve as we get more people using it and more videos captured to learn from. 

If you’re interested in purchasing a camera, check out the thermal camera product page.