8084 / Log Detection For Autonomous Forwarding Using Auto-Annotated Data From A Real-Time Virtual Environment

Authors

Mattias Lehto, Håkan Lideskog, and Magnus Karlberg

Paper presented at ISTVS 2024 | 21st International and 12th Asia-Pacific Regional Conference of the ISTVS Keywords: Transfer learning; Domain generalization; Virtual training; Auto-annotation; Real-time; Co-simulation; Logging; Tree harvesting; Forwarder; Cut-to-length; CTL https://doi.org/10.56884/XD21D6FR

Abstract

An integral part of autonomous forestry is the ability of the vehicles, e.g., forwarders and harvesters, to perceive their environment. At Luleå University of Technology, object detectors have previously been developed, allowing forestry vehicles to detect and position important objects in forestry, such as tree stumps, stones, and logs. These detectors have been developed by training on physical manually annotated data, which is both time-consuming and costly. Training on virtual data allows for significant time- and cost reductions. Since the ground truth in the virtual model is known, the training data can be auto-annotated, allowing for the creation of larger training datasets, at a lower cost. In this work, a virtual environment in Unity is used in co-simulation with a real-time digital twin of a physical forestry vehicle, to generate auto-annotated training data, as captured by an onboard stereo camera. A detailed emulation of the stereo camera is used to achieve realistic results. First, a log detector trained on physical manually annotated data, is evaluated on virtually created data. It is shown that the log detector trained on physical data can detect logs in the virtual environment. Second, new detectors are trained, using different shares of physical and virtual data. It is shown that a detector trained using only virtual data, can learn to detect logs in the physical world. Moreover, virtual pre-training is shown to improve the performance of physically trained and tested detectors, both at low availability of physical training data, and in terms of domain generalization. Furthermore, the real-time capable virtual models also enable future machine learning tasks utilizing different levels of Hardware-in-the-Loop.

---

Full paper purchase: https://www.istvs.org/proceedings-orders/paper ISTVS members receive three complimentary papers per year: https://www.istvs.org/members