Enhancing tree species identification using multi-view Convolutional Neural Networks

Background:

Tree species identification is essential for biodiversity monitoring, forest management, and understanding ecological processes. Advances in computer vision and deep learning have enabled the use of multi-view convolutional neural networks (CNNs) to classify species by integrating complementary information from different views. While such techniques hold promise, the variability in resolution, size, and perspective of the data presents challenges that must be addressed for robust identification. Citizen science data provide a wealth of georeferenced plant images captured by volunteers. These datasets, which include diverse environments, seasonal conditions, and perspectives, can enhance the training and generalization of multi-view CNN models. This thesis explores the integration of multi-view data and citizen science images to develop a scalable, high-accuracy tree species identification framework.

Objectives:

• Develop a multi-view CNN-based framework for tree species identification, integrating aerial and street-level images with citizen science photographs.
• Evaluate the impact of fusion strategies (early, late, score, and progressive fusion) on the performance of the CNN.
• Investigate how incorporating citizen science data can enhance model robustness and accuracy.

Methods:

A number of >333’414 RGB images and the corresponding tree annotations are available for two views and further citizen data will be integrated. Convolutional neural networks or Transformers will be implemented. Analysis and model training will be conducted using Python. The methodology involves implementing four fusion strategies: early fusion, late fusion, score fusion, and progressive fusion to integrate multi-view data for tree species identification.

Wanted:

A highly motivated student interested in remote sensing and deep learning and willing to learn new practical techniques and ecological concepts.

You will get to:

• Learn about multi-view CNNs and, more importantly, how to apply these technical skills to a relevant application for the forestry community.
• Get familiar with the latest developments in computer vision and remote sensing.
• Gain important computer vision knowledge for future work.
• Expand your network by discussing your work with experts from the intersecting fields of forest sciences, and computer sciences.
• Be a co-author on a publication resulting from this work.
• Be part of a motivated, fun, and energetic team of scientists.

Supervisor:

Dr. Lukas Drees from the Department of Mathematical Modeling and Machine Learning at the University of Zurich and Dr. M. Beloiu Schwenke from the Department of Environmental Systems Science at ETH Zürich.

The project has a flexible starting date.

If the idea of participating in cutting-edge interdisciplinary research excites you, please contact mirela.beloiu(at)usys.ethz.ch or lukas.drees(at)uzh.ch. The FORM team is looking forward to hearing from you!