Last updated
Mattias Lehto, Håkan Lideskog, and Magnus Karlberg
An integral part of autonomous forestry is the ability of the vehicles, e.g., forwarders and harvesters, to perceive their environment. At Luleå University of Technology, object detectors have previously been developed, allowing forestry vehicles to detect and position important objects in forestry, such as tree stumps, stones, and logs. These detectors have been developed by training on physical manually annotated data, which is both time-consuming and costly. Training on virtual data allows for significant time- and cost reductions. Since the ground truth in the virtual model is known, the training data can be auto-annotated, allowing for the creation of larger training datasets, at a lower cost. In this work, a virtual environment in Unity is used in co-simulation with a real-time digital twin of a physical forestry vehicle, to generate auto-annotated training data, as captured by an onboard stereo camera. A detailed emulation of the stereo camera is used to achieve realistic results. First, a log detector trained on physical manually annotated data, is evaluated on virtually created data. It is shown that the log detector trained on physical data can detect logs in the virtual environment. Second, new detectors are trained, using different shares of physical and virtual data. It is shown that a detector trained using only virtual data, can learn to detect logs in the physical world. Moreover, virtual pre-training is shown to improve the performance of physically trained and tested detectors, both at low availability of physical training data, and in terms of domain generalization. Furthermore, the real-time capable virtual models also enable future machine learning tasks utilizing different levels of Hardware-in-the-Loop.
Full paper purchase: ISTVS members receive three complimentary papers per year: