Tree Species Segmentation Practice based on UAV Imagery

doi:10.13466/j.cnki.lyzygl.2023.01.014

Abstract

Abstract:

The characteristics of the forest land with a canopy density of more than 60% include complex tree species,a narrow interspecific character gap,and modest interspecific distances.Traditional convolutional neural networks (CNN) are superior to other classification approaches in terms of accuracy and automation,but they also have a low learning efficiency,a tough time enhancing recognition accuracy,and poor interpretability.Additionally,the typical approach of creating maps of tree species through regular segmentation overlooks changes in tree species and boundary characteristics of remote sensing objects,making it simple to create the salt and pepper phenomenon in forest areas with dense canopies.Therefore,a Grad-Swin transformer (G-ST) based on class activation mapping and Swin transformer (ST) classification model was suggested as the solution to the aforementioned issues.It incorporates transfer learning,the ST classification model,and gradient descent class activation mapping,and by integratinglong-distance features,data enhancement,feature knowledge from other fields,and prediction training attention,it increases the G-ST classification accuracy as well as the model's generalization and interpretability abilities.Thematic maps of tree species are created by using a straightforward linear iterative clustering algorithm.The results demonstrated that the accuracy of the tree species map produced by this method was greater than that of the conventional CNN combined with regular segmentation,and the boundary between trees and remote sensing objects was closer to the vectorization result.This method can therefore serve as a useful reference for tree species image segmentation,mapping,and distribution statistics.

Key words: G-ST, ST, data augmentation, CNN, transfer learning, tree species imagery segmentation and mapping

CLC Number:

TP753

PU Tao, WANG Ni, GONG Yuhong, WANG An. Tree Species Segmentation Practice based on UAV Imagery[J]. FOREST RESOURCES WANAGEMENT, 2023, 0(1): 115-126.

Figures/Tables 11

Fig.1

Fig.2

Tab.1

Fig.3

Fig.4

Fig.5

Fig.6

Tab.2

Tab.3

Fig.7

Fig.8

References 20

[1]	戴鹏钦, 丁丽霞, 刘丽娟, 等. 基于FCN的无人机可见光影像树种分类[J]. 激光与光电子学进展, 2020, 57(10):36-45.
[2]	欧阳光, 荆林海, 阎世杰, 等. 基于卷积神经网络的高分遥感影像单木树种分类[J]. 激光与光电子学进展, 2021, 58(2):349-362.
[3]	Xi Yanbiao, Tian Jia, Jiang Hailing, et al. Mapping tree species in natural and planted forests using Sentinel-2 images[J]. Remote Sensing Letters, 2022, 13(6):544-555. doi: 10.1080/2150704X.2022.2051636
[4]	He Zhi, He Dan. Bilinear squeeze-and-excitation network for fine-grained classification of tree species[J]. IEEE Geoscience and Remote Sensing Letters, 2021, 18(7):1139-1143. doi: 10.1109/LGRS.2020.2994952
[5]	龚健雅, 宦麟茜, 郑先伟. 影像解译中的深度学习可解释性分析方法[J]. 测绘学报, 2022, 51(6):873-884.
[6]	滕文秀, 王妮, 施慧慧, 等. 结合面向对象和深度特征的高分影像树种分类[J]. 测绘通报, 2019(4):38-42.
[7]	Deur M, Gasparovic M, Balenovici. An evaluation of Pixel-and object-based tree species classification in mixed deciduous forests using pansharpened very high spatial resolution satellite imagery[J]. Remote Sensing, 2021, 13(10):1868. doi: 10.3390/rs13101868
[8]	Schieffr F, Kattenborn T, Frick A, et al. Mapping forest tree species in high resolution UAV-based RGB-imagery by means of convolutional neural networks[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 170:205-215. doi: 10.1016/j.isprsjprs.2020.10.015
[9]	He Xin, Zhou Yong, Zhao Jiaqi, et al. Swin transformer embedding UNet for remote sensing image semantic segmentation[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:1-15.
[10]	滕文秀, 温小荣, 王妮, 等. 基于深度迁移学习的无人机高分影像树种分类与制图[J]. 激光与光电子学进展, 2019, 56(7):277-286.
[11]	Qin Haiming, Zhou Weiqi, Yao Yang, et al. Individual tree segmentation and tree species classification in subtropical broadleaf forests using UAV-based LiDAR,hyperspectral,and ultrahigh-resolution RGB data[J]. Remote Sensing of Environment, 2022, 280:113143. doi: 10.1016/j.rse.2022.113143
[12]	夏爱梅, 聂乐群. 安徽植被带的划分[J]. 武汉植物学研究, 2004(6):523-528.
[13]	Kawamura K, Asai H, Yasuda T, et al. Discriminating crops/weeds in an upland rice field from UAV images with the SLIC-RF algorithm[J]. Plant Production Science, 2021, 24(2):198-215. doi: 10.1080/1343943X.2020.1829490
[14]	Homan D, Du Preez J A. Automated feature-specific tree species identification from natural images using deep semi-supervised learning[J]. Ecological Informatics, 2021, 66:101475. doi: 10.1016/j.ecoinf.2021.101475
[15]	赵霖, 张晓丽, 吴艳双, 等. 面向机载高光谱数据的3D-CNN亚热带森林树种分类[J]. 林业科学, 2020, 56(11):97-107.
[16]	Zhang Cheng, Jiang Wanshou, Zhang Yuan, et al. Transformer and CNN hybrid deep neural network for semantic segmentation of Very-High-Resolution remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:1-20.
[17]	Huang Xin, Dong Mengjie, Li Jiayi, et al. A 3-D-Swin Transformer-Based hierarchical contrastive learning method for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:1-15.
[18]	Tu Jingzhi, Mei Gang, Ma Zhengjing, et al. SWCGAN:Generative adversarial network combining swin transformer and CNN for remote sensing image Super-Resolution[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022, 15:5662-5673. doi: 10.1109/JSTARS.2022.3190322
[19]	Selvaraju R R, Cogswell M, Das A, et al. Grad-CAM:Visual explanations from deep networks via Gradient-based localization[J]. International Journal of Computer Vision, 2020, 128(2):336-359. doi: 10.1007/s11263-019-01228-7
[20]	Su Hongjun, Yao Wenjing, Wu Zhaoyu, et al. Kernel low-rank representation with elastic net for China coastal wetland land cover classification using GF-5 hyperspectral imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 171:238-252. doi: 10.1016/j.isprsjprs.2020.11.018

类别	训练集	验证集	合计
香樟叶	84	21	105
香樟(Cinnamomum camphora)	112	28	140
女贞(Ligustrum lucidum)	205	51	256
龙柏(Sabina chinensis)	140	35	175
栀子花(Gardenia jasminoides)	82	20	102
石楠叶	128	31	159
石楠(Photinia serratifolia)	109	27	136
鹤望兰(Strelitzia reginae)	80	20	100
建筑	84	20	104
月桂(Laurus nobilis)	81	20	101
月桂叶	32	7	39
马路	111	27	138
裸地	84	20	104
湖	87	21	108
合计	1419	348	1767

类别	VT模型				G-ST模型								RegNet模型
类别	Vitb/ 16	Vitb/ 32	Vitl/ 16	Vitl/ 32	Swin_ small		Swin_ base		Swin_ tiny		Swin_ large		RegNetY_ 400MF		RegNetY_ 200MF		RegNetX_ 600MF
香樟叶	0.24	0.00	0.48	0.48		0.29		0.14		0.76		0.00		0.52		0.38		0.52
香樟(Cinnamomum camphora)	0.18	0.00	0.11	0.36		0.32		032		0.25		0.75		0.79		0.71		0.68
女贞(Ligustrum lucidum)	1.00	0.22	0.94	0.98		0.78		0.98		1.00		0.96		0.90		0.92		0.86
龙柏(Sabina chinensis)	0.91	0.00	0.97	0.94		0.89		0.69		0.49		0.43		0.34		0.37		0.69
栀子花(Gardenia jasminoides)	0.05	0.15	0.05	0.00		0.00		0.00		0.00		0.00		0.15		0.25		0.20
石楠叶	0.65	0.48	0.26	0.23		0.55		0.26		0.58		0.61		0.61		0.74		0.65
石楠(Photinia serratifolia)	0.19	0.00	0.37	0.37		0.70		0.85		0.85		0.19		0.04		0.04		0.04
鹤望兰(Strelitzia reginae)	0.40	0.00	0.45	0.35		0.45		0.30		0.05		0.30		0.20		0.25		0.15
建筑	0.40	0.00	0.35	0.25		0.20		0.15		0.20		0.35		0.35		0.30		0.30
月桂(Laurus nobilis)	0.00	0.00	0.00	0.00		0.26		0.38		0.28		0.22		0.00		0.00		0.00
月桂叶	0.74	0.15	0.78	0.78		0.67		0.78		0.70		0.59		0.89		0.89		0.93
马路	0.20	0.00	0.50	0.70		0.50		0.65		0.40		0.25		0.50		0.60		0.35
裸地	0.29	0.00	0.05	0.33		0.48		0.24		0.57		0.52		0.33		0.52		0.43
湖	0.80	0.40	0.80	0.95		0.95		0.85		0.80		0.75		1.00		1.00		1.00
总分类精度	0.52	0.12	0.51	0.56		0.55		0.52		0.55		0.49		0.53		0.56		0.56
平均分类精度	0.43	0.10	0.44	0.48		0.48		0.44		0.51		0.41		0.47		0.50		0.49
Kappa系数	0.47	0.05	0.46	0.51		0.51		0.47		0.48		0.43		0.49		0.51		0.51
运行时间/s	30	25	50	28		25		30		24		40		26		23		22

类别	VT模型				G-ST模型								RegNet模型
类别	Vitb/ 16	Vitb/ 32	Vitl/ 16	Vitl/ 32	Swin_ small		Swin_ base		Swin_ tiny		Swin_ large		RegNetY_ 400MF		RegNetY_ 200MF		RegNetX_ 600MF
香樟叶	0.95	0.86	1.00	0.86		1.00		0.95		1.00		1.00		0.95		1.00		0.95
香樟(Cinnamomum camphora)	0.82	0.71	0.93	0.93		0.93		0.93		0.96		1.00		0.96		0.96		0.96
女贞(Ligustrum lucidum)	0.94	0.98	0.98	1.00		0.96		0.98		1.00		0.98		0.92		0.94		1.00
龙柏(Sabina chinensis)	0.86	0.83	0.97	0.89		0.97		1.00		1.00		1.00		1.00		1.00		1.00
栀子花(Gardenia jasminoides)	1.00	0.75	0.95	0.90		1.00		1.00		1.00		1.00		1.00		1.00		1.00
石楠叶	0.94	0.87	0.97	0.90		1.00		1.00		1.00		1.00		1.00		0.94		0.90
石楠(Photinia serratifolia)	0.74	0.85	0.89	0.78		0.85		0.96		0.85		0.93		0.96		0.93		0.93
鹤望兰(Strelitzia reginae)	0.75	0.85	1.00	0.75		1.00		1.00		1.00		1.00		1.00		1.00		0.95
建筑	0.90	0.96	0.95	0.95		0.95		1.00		1.00		1.00		1.00		1.00		1.00
月桂(Laurus nobilis)	0.00	0.00	0.00	0.00		0.86		0.43		0.71		0.714		0.13		0.00		0.14
月桂叶	0.70	0.85	1.00	0.78		1.00		1.00		0.96		1.00		1.00		1.00		1.00
马路	0.70	0.80	0.80	0.70		1.00		1.00		1.00		1.00		1.00		0.90		1.00
裸地	0.81	0.81	0.91	0.86		1.00		1.00		1.00		1.00		0.95		1.00		0.91
湖	0.95	0.90	1.00	1.00		1.00		1.00		1.00		1.00		0.95		1.00		0.85
总分类精度	0.84	0.84	0.93	0.86		0.97		0.97		0.98		0.99		0.97		0.95		0.95
平均分类精度	0.79	0.79	0.88	0.81		0.97		0.95		0.96		0.97		0.96		0.91		0.90
Kappa系数	0.82	0.82	0.93	0.85		0.97		0.97		0.98		0.98		0.97		0.95		0.94
运行时间/s	23	20	46	28		20		23		19		34		21		17		16