GONG Yuanjin, YANG Shujie. Dynamic water surface environment perception SLAM algorithm based on visual semantics and point-line fusion for unmanned surface vessels[J]. South China Fisheries Science, 2024, 20(3): 120-132. DOI: 10.12131/20240019
Citation: GONG Yuanjin, YANG Shujie. Dynamic water surface environment perception SLAM algorithm based on visual semantics and point-line fusion for unmanned surface vessels[J]. South China Fisheries Science, 2024, 20(3): 120-132. DOI: 10.12131/20240019

Dynamic water surface environment perception SLAM algorithm based on visual semantics and point-line fusion for unmanned surface vessels

More Information
  • Received Date: January 21, 2024
  • Revised Date: March 23, 2024
  • Accepted Date: March 31, 2024
  • Available Online: April 21, 2024
  • Unmanned surface vessels (USVs) have autonomous navigation capacities under complex water surface environment, which is the basis for aquaculture operation, with great development prospects in intelligent aquaculture. Simultaneous localization and mapping (SLAM) technology provides real-time environmental information, playing a pivotal role in enabling autonomous navigation for USVs. However, water surfaces present low-texture scenes with insufficient distinct features, compounded by dynamic invalid features caused by surface ripples and reflections, which result in poor calculation accuracy and serious performance degradation of visual SLAM pose. To address this problem, the paper proposes a SLAM system oriented towards dynamic water surface environments based on visual semantics and point-line fusion. Firstly, the ORB-SLAM3 framework is enhanced by introducing a semantic segmentation thread to eliminate invalid water features using semantic masks, thereby mitigating dynamic water interference. Secondly, the system stability is enhanced by incorporating line features to propose a geometric-constraint-based line-matching method to improve the accuracy of waterline feature extraction and tracking. Moreover, the point-line feature fusion is utilized to enhance data association accuracy, resolving the deficiency of traditional SLAM algorithms in extracting features from low-texture water surfaces. The results of the USVInland dataset demonstrate that compared with ORB-SLAM3 and PL-SLAM algorithms, the positioning accuracy of the improved algorithm has averagely increased by average of 44.74% and 55.48% in straight-line navigation, and up to 76.60% and 70.15% in maneuvering navigation, which mitigates the impact of water surface disturbances on pose estimation effectively, and enhances the accuracy and robustness of visual SLAM systems under low-texture water environment.

  • [1]
    刘世晶, 李国栋, 刘晃, 等. 中国水产养殖装备发展现状[J]. 水产学报, 2023, 47(11): 190-203.
    [2]
    沈启扬, 朱虹, 于庆旭, 等. 河蟹养殖智能化投饵装备适用性能试验研究[J]. 中国农机化学报, 2022, 43(12): 51-59.
    [3]
    李昕聪, 余紫扬, 刘璞, 等. 基于NB-IoT和无人船巡检的水产养殖场物联网系统研究[J]. 渔业现代化, 2022, 49(1): 72-81.
    [4]
    赵同强, 韩超, 徐玉良, 等. 无人船技术应用于城市内河水质监测[J]. 中国给水排水, 2021, 37(7): 71-77.
    [5]
    ROSEN D M, DOHERTY K J, TERÁN ESPINOZA A, et al. Advances in inference and representation for simultaneous localization and mapping[J]. Annu Rev Control Robot Auton Syst, 2021, 4: 215-242. doi: 10.1146/annurev-control-072720-082553
    [6]
    田野, 陈宏巍, 王法胜, 等. 室内移动机器人的SLAM算法综述[J]. 计算机科学, 2021, 48(9): 223-234.
    [7]
    MUR-ARTAL R, MONTIEL J M M, TARDOS J D. ORB-SLAM: a versatile and accurate monocular SLAM System[J]. IEEE Trans Rob, 2015, 31(5): 1147-1163. doi: 10.1109/TRO.2015.2463671
    [8]
    CAMPOS C, ELVIRA R, RODRÍGUEZ J J G, et al. ORB-SLAM3: an accurate open-source library for visual, visual-inertial, and multimap SLAM[J]. IEEE Trans Rob, 2021, 37(6): 1874-1890. doi: 10.1109/TRO.2021.3075644
    [9]
    付洪宇, 史国友, 冉洋, 等. 基于单目相机与K均值聚类分割的船舶航行环境地图深度构建[J]. 上海海事大学学报, 2022, 43(4): 1-8.
    [10]
    郑又能. 面向无人船的计算机视觉应用研究[D]. 绵阳: 西南科技大学, 2020: 10.
    [11]
    高于科, 章伟, 胡陟, 等. 一种水平面估计的水域环境视觉SLAM算法[J]. 测绘科学, 2023, 48(5): 104-112, 128.
    [12]
    ZHAN W Q, XIAO C S, WEN Y Q, et al. Adaptive semantic segmentation for unmanned surface vehicle navigation[J]. Electronics-SWITZ, 2020, 9(2): 213.
    [13]
    JAMES T, SCHILLACI C, LIPANI A. Convolutional neural networks for water segmentation using sentinel-2 red, green, blue (RGB) composites and derived spectral indices[J]. Int J Remote Sens, 2021, 42(14): 5338-5365. doi: 10.1080/01431161.2021.1913298
    [14]
    沈建军, 陶青川, 肖卓. 结合改进Deeplab v3+网络的水岸线检测算法[J]. 中国图象图形学报, 2019, 24(12): 2174-2182. doi: 10.11834/jig.190051
    [15]
    BOVCON B, KRISTAN M. WaSR: a water segmentation and refinement maritime obstacle detection network[J]. IEEE Trans Cybern, 2021, 52(12): 12661-12674.
    [16]
    TAIPALMAA J, PASSALIS N, ZHANG H L, et al. High-resolution water segmentation for autonomous unmanned surface vehicles: a novel dataset and evaluation[C]//2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP). Pittsburgh, PA, USA: IEEE, 2019: 1-6.
    [17]
    TEICHMANN M, WEBER M, ZOELLNER M, et al. MultiNet: real-time joint semantic reasoning for autonomous driving[C]//2018 IEEE intelligent vehicles symposium (IV). Changshu, China: IEEE, 2018: 1013-1020.
    [18]
    SIT M, DEMIRAY B Z, XIANG Z R, et al. A comprehensive review of deep learning applications in hydrology and water resources[J]. Water Sci Technol, 2020, 82(12): 2635-2670. doi: 10.2166/wst.2020.369
    [19]
    ZHAN W Q, XIAO C S, WEN Y Q, et al. Autonomous visual perception for unmanned surface vehicle navigation in an unknown environment[J]. Sensors-BASEL, 2019, 19(10): 2216. doi: 10.3390/s19102216
    [20]
    CHENG Y W, JIANG M X, ZHU J N, et al. Are we ready for unmanned surface vehicles in inland waterways? The USVInland multisensor dataset and benchmark[J]. IEEE Rob Autom Lett, 2021, 6(2): 3964-3970. doi: 10.1109/LRA.2021.3067271
    [21]
    余思雨, 齐林, 李逸文, 等. 语义驱动下水面场景中AR效果提升方法[J]. 计算机应用与软件, 2021, 38(4): 245-249.
    [22]
    GAN Y, ZHANG J H, CHEN K Q, et al. A dynamic detection method to improve SLAM performance[J]. Optoelectron Lett, 2021, 17(11): 693-698. doi: 10.1007/s11801-021-1022-5
    [23]
    CHEN S B, ZHOU B D, JIANG C H, et al. A LiDAR/Visual SLAM backend with loop closure detection and graph optimization[J]. Remote Sens, 2021, 13(14): 2720. doi: 10.3390/rs13142720
    [24]
    TANG J X, ERICSON L, FOLKESSON J, et al. GCNv2: efficient correspondence prediction for real-time SLAM[J]. IEEE Rob Autom Lett, 2019, 4(4): 3505-3512.
    [25]
    ZHANG Z Q, DONG P, WANG J R, et al. Improving S-MSCKF with variational bayesian adaptive nonlinear filter[J]. IEEE Sens J, 2020, 20(16): 9437-9448. doi: 10.1109/JSEN.2020.2989206
    [26]
    CASTELLANOS J A, NEIRA J, TARDÓS J D. Limits to the consistency of EKF-based SLAM[J]. IFAC Proceedings Volumes, 2004, 37(8): 716-721.
    [27]
    GOMEZ-OJEDA R, BRIALES J, GONZALEZ-JIMENEZ J. PL-SVO: semi-direct monocular visual odometry by combining points and line segments[C]//2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Daejeon, Korea (South): IEEE, 2016: 4211-4216.
    [28]
    FORSTER C, PIZZOLI M, SCARAMUZZA D. SVO: fast semi-direct monocular visual odometry[C]//2014 IEEE International Conference on Robotics and Automation (ICRA). Hong Kong, China: IEEE, 2014: 15-22.
    [29]
    COMPANY-CORCOLES J P, GARCIA-FIDALGO E, ORTIZ A. MSC-VO: exploiting manhattan and structural constraints for visual odometry[J]. IEEE Rob Autom Lett, 2022, 7(2): 2803-2810. doi: 10.1109/LRA.2022.3142900
    [30]
    GOMEZ-OJEDA R, MORENO F A, ZUNIGA-NOËL D, et al. PL-SLAM: a stereo SLAM system through the combination of points and line segments[J]. IEEE Trans Rob, 2019, 35(3): 734-746. doi: 10.1109/TRO.2019.2899783
    [31]
    ZUO X X, XIE X J, LIU Y, et al. Robust visual SLAM with point and line features[C]//2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Vancouver, BC, Canada: IEEE, 2017: 1775-1782.
    [32]
    GÁLVEZ-LÓPEZ D, TARDOS J D. Bags of binary words for fast place recognition in image sequences[J]. IEEE Trans Rob, 2012, 28(5): 1188-1197. doi: 10.1109/TRO.2012.2197158
    [33]
    YU C Q, GAO C X, WANG J B, et al. BiSeNet V2: bilateral network with guided aggregation for real-time semantic segmentation[J]. Int J Comput Vision, 2021, 129: 3051-3068. doi: 10.1007/s11263-021-01515-2
    [34]
    余昌黔. 面向场景分割的判别特征感知方法研究[D]. 武汉: 华中科技大学, 2023: 50-59.
    [35]
    高翔, 张涛, 刘毅, 等. 视觉SLAM十四讲[M]. 2版. 北京: 电子工业出版社, 2019: 166
    [36]
    郑行家, 钟宝江. 图像直线段检测算法综述与测评[J]. 计算机工程与应用, 2019, 55(17): 9-19.
    [37]
    张震, 王永雄. 基于Edline线特征的图像快速匹配算法[J]. 计算机与数字工程, 2019, 47(3): 652-656.
    [38]
    王金戈, 邹旭东, 仇晓松, 等. 动态环境下结合语义的鲁棒视觉SLAM[J]. 传感器与微系统, 2019, 38(5): 125-128, 132.
  • Related Articles

    [1]WEI Jie, CAO Xiquan, REN Yongli, AIKEBAIER Rehemana, NIE Zhulan. Anatomy and histological observation of digestive system in Schizothorax eurystomus[J]. South China Fisheries Science, 2020, 16(1): 120-126. DOI: 10.12131/20180262
    [2]LIU Yaqiu, LI Xinhui, LI Yuefei, YANG Jiping, XU Tianzhen. A histological study on gonadal development of black amur bream (Megalobrama terminalis)[J]. South China Fisheries Science, 2019, 15(1): 113-118. DOI: 10.12131/20180139
    [3]ZHANG Li, YANG Xueming, JIANG Linyuan, YANG Qiong, HUANG Guanghua, LI Jianbin. Histological characteristics of gonadal development of Holothuria scabra from Hainan Island[J]. South China Fisheries Science, 2016, 12(3): 51-57. DOI: 10.3969/j.issn.2095-0780.2016.03.007
    [4]XIE Mujiao, OU Youjun, LI Jia′er, WEN Jiufu, WANG Pengfei, WANG Wen, CHEN Shixi. Histology observation in digestive tract of Eleutherollema tetradactylum at different developmental stages[J]. South China Fisheries Science, 2016, 12(2): 51-58. DOI: 10.3969/j.issn.2095-0780.2016.02.008
    [5]OU Youjun, LI Jia′er, GOU Xiaowei, MA Zhiming, CHEN Yibiao. Histological and histochemical study of digestive tract of Siganus oramin[J]. South China Fisheries Science, 2013, 9(5): 51-57. DOI: 10.3969/j.issn.2095-0780.2013.05.009
    [6]HU Lingling, LI Jiaer, QU Youjun, YU Na, WANG Gang. Study on morphology and histology of digestive tract in cultured Oplegnathus fasciatus[J]. South China Fisheries Science, 2010, 6(6): 65-69. DOI: 10.3969/j.issn.1673-2227.2010.06.011
    [7]SU Youlu, SUN Xiuxiu, FENG Juan, GUO Zhixun, XU Liwen, WANG Jiangyong. Morphological and histological observations of digestive system of cobia Rachycentron canadum[J]. South China Fisheries Science, 2008, 4(6): 88-94.
    [8]GOU Xiaowei, QU Youjun, LIAO Rui. Histological and histochemical study of digestive system of Rhabdosargus sarba[J]. South China Fisheries Science, 2008, 4(5): 28-36.
    [9]LV Linlan, DONG Xuexing. Morphological and histological observation on the digestive tract of Carcinoscorpius rotundicauda[J]. South China Fisheries Science, 2006, 2(5): 12-18.
    [10]HUANG Jian-hua, MA Zhi-ming, ZHOU Fa-lin, YE Le, JIANG Shi-gui. Study on anatomy structure and histology of the ovary of wild Penaeus monodon(Fabricius) from the north of South China Sea[J]. South China Fisheries Science, 2005, 1(3): 49-53.
  • Cited by

    Periodical cited type(2)

    1. 徐思琪,张世勇,张文平,刘洪岩,王明华,钟立强,边文冀,陈校辉. 温度诱导斑点叉尾鮰雌性化研究. 福建农业学报. 2022(06): 741-747 .
    2. 周胜杰,胡静,杨蕊,杨其彬,马振华. 三斑海马早期发育观察. 南方农业学报. 2021(08): 2302-2310 .

    Other cited types(1)

Catalog

    Article views (211) PDF downloads (51) Cited by(3)
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return