This is one of the most open and honest messages you will see from a thermal camera manufacturer. FLIR says, “Finding a hiker lost in the woods is a challenging mission—if the person is lost in densely forested terrain, then sunlight is largely blocked by trees and other vegetation, and the ground reflects very little light. Because of this, thermal imaging is often used to help spot warm human bodies in a forested environment. But even thermal imaging can’t see through trees, making rescue missions a difficult task even when first responders are aided by a thermal-equipped drone or helicopter.” – Source
Thank you, FLIR!
The inability of thermal cameras to find people if vegetation is around is a major frustration. My readers are reporting a 96%+ failure rate to locate subjects in areas with tree canopies and vegetation coverage. I wish people would get lost in open fields. They don’t in my neck of the woods.
For years, all thermal camera manufacturers’ marketing has bugged me because it is made to appear that a thermal camera on a drone is going to be some magic wand and find people.
The FLIR article describes new Artificial Intelligence (AI) capabilities under development that hope to reach the market in five years.
The Use of AI
As I said in my 2021 drone prediction piece, AI advancements are going to help move the search and rescue field forward.
The FLIR announcement of technology under developed references David Schedl, Indrajit Kurmi, and Oliver Bimber at the Johannes Kepler University, Faculty of Engineering and Natural Sciences department.
Their new paper out on Search and Rescue with Airborne Optical Sectioning (AOS) contains the type of fascinating research we eagerly await.
The FLIR announcement of how this works nails the issue squarely when it says, “For future testing, the team is looking to upgrade to a drone that can fly for a longer period of time (up to several hours), and a thermal camera like the FLIR Boson that includes an SDK to process the images in real-time. Their goal is to have an autonomous drone that downloads thermal images in real-time, processes the images, and then computes where to fly next. If it sees something that might be a person, it will investigate further, finding missing people as quickly and as reliably as possible. It will also be capable of communicating with the rescue team and sending its results to their mobile devices.”
Yes, please!
This AI technology works because while it does not allow the thermal camera to have magical capabilities to see through vegetation, it does allow the AI to piece together glimpses of thermal information to assemble images that create a clearer picture of what is happening below the tree canopy.
For example, here are two images of a mixed forest environment (d) and a conifer forest (e). The larger image shows the type of image we see with current cameras and the smaller inset is utilizing the AOS technology.
Using AOS, onboard AI, and the ability for the UAS to interpret the images and output, this technology will significantly reduce the time to find missing targets.
Hopefully, it will tell the difference between a cow, horse, deer, and human at some point. I’m tired of finding wildlife on SAR missions.
I don’t need any more radio calls that say, “Steve, we found the target. It’s a horse, a goat, and a turkey.”
The AOS research paper gets into the technical specifics of how the AOS system works. For those that are nerds like me, it is fascinating.
Here are some highlights from the paper.
- “Color and thermal images that are recorded within the shape of a wide synthetic aperture area above a forest are combined (registered and integrated) to computationally remove occluders, such as trees and other vegetation. Applied to thermal imaging, our technique makes the radiated heat signal of largely occluded targets (e.g., a human body hidden in dense undergrowth) visible by integrating multiple thermal recordings from slightly different perspectives. The outcome is a mostly occlusion-free view of the forest ground.”
- “While the heat footprint in single thermal recordings shows mostly a random pattern due to varying partial occlusion from different views, AOS results often reveal the recognizable shape of a human.”
- “Our findings pave the way for future autonomous SAR technologies that focus on finding lost and injured people in dense forests. Since fast operation is critical to such missions, computer-supported analysis of the enormous amount of image data is essential. However, human detection by means of AOS requires a specialized training dataset.”
The proper identification of targets will come from a training dataset, much like the type of training currently being used to help with the development of autonomous driving.
The trained datasets could then be used to identify specific wildlife or humans. The AI dataset can be used for any intended target classification once the AI is taught what to look for.
The future of thermal in SAR looks amazing and I can’t wait for it to arrive.