You might be rolling your eyes as you watch the drone take to the skies and hover over the Australian coast with the camera pointing straight down into the glistening turquoise water. “Another TikTok influencer trying to get the perfect shot,” you mutter to yourself. But if you look closely at the remote, you’ll notice that it has a sign with bright yellow and red letters “Keep Distance” next to it. This is not a TikTok influencer.
He’s an Australian surf lifeguard who uses the above drone to spot sharks on the beach before they get too close to swimmers like you. The New South Wales government has decided to invest millions of dollars (more than A$85 million to be exact) in shark mitigation measures over the next few years in an effort to better coexist with these predators. Sure, there are helicopter patrols and perpetually splitting nets into sharks and drums, but a 2020 study found that drone-based shark research was what the public preferred when it came to shark safety.
This is not new as the state government has been using drones to detect sharks since 2016 and has teamed up with Surf Life Saving NSW to continue the work from 2018. Drones ascend to over 196 feet (60 meters) piloted by life-saving pilots in the rain to maneuver the technology over the blue ocean while watching a live video feed of what the drone sees. They don’t just look at the great waves – they hunt the shapes of sharks swimming below the surface. While the pilots are trained to recognize shark shapes between themselves (for example, the great white shark looks different than the tiger shark) and among other animals (the shark looks different than a seal or a big fish), it can be a bit difficult when the weather conditions are not the best. The wind can make it difficult to get a clear picture, the sunlight hits the wrong spot, or the water is just too cloudy and dark and there are seaweed everywhere…
Drone pilots generally make the right calls 60% of the time – which is both reassuring and a bit irritating for some. That’s why a team of scientists set out to see if artificial intelligence (AI) is the answer. Dr Cormac Purcell has received funding from the NSW Department of Commodity Industries to conduct research at Macquarie University with Dr Paul Butcher, Assistant Professor at Southern Cross University and Deakin University. Together, the team set out to build the “most robust” shark AI detector and test it directly in Australian waters. You see, while most AI does quite well in the lab, it has many challenges to overcome in the real world, which is why Purcell and Butcher wanted to test their detector in the wild. “Early results from previous AI-assisted shark detection systems suggested that the issue was resolved as these systems report detection accuracy of over 90%,” the authors wrote in The Conversation. “However, scaling these systems to make a real difference on NSW beaches has been a challenge. […] Basically, machine learning operations clearly recognize that AI-powered software needs regular updates to stay effective.”
Researchers were able to create a mobile app for surf lifeguards by meticulously tracking and identifying sharks to feed the information into AI software so it can “learn,” as the authors explained: “Using this new dataset, we trained a machine learning model to recognize ten species marine life, including different species of dangerous sharks such as the great white shark and the whale shark. We then put this model into a new mobile app that can highlight sharks in live drone footage and predict the species. We have worked closely with the NSW Government and Surf Lifesaving NSW to try out this app on five beaches in summer 2020.”
How did it go? “Our AI shark detector performed quite well. In 80% of cases, he identified dangerous sharks frame by frame in realistic conditions. We deliberately did everything we could to make our tests more difficult by forcing the AI to operate on unseen data collected at different times of the year or from beaches with different appearances.” Now there were some limitations in the app – for example, sharks with similar outlines were difficult to identify, and smaller animals were difficult to detect. But the team believes the AI is “now mature enough to be used in drone shark detection operations on Australian beaches. But unlike regular software, it will require frequent monitoring and updates to maintain high reliability in detecting dangerous sharks.”
Since it’s summer down below, drones are scanning the waters again and hopefully protecting beachgoers. “Artificial intelligence can play a key role in making these flights more efficient, providing more reliable drone surveillance, and ultimately could lead to fully automated shark detection operations and trusted automatic alerts,” the authors conclude.