Insert title here Insert title here Insert title here

Semi-autonomous Exploration Robot in Disaster Area [ October 2018 ~ Present ]

Research Background
Semantic Survey Map Building Framework
In this research, we propose a semi-autonomous mobile robot system that builds a wide-area survey map including semantic information to carry out damage monitoring in disaster areas such as Fukushima Daiichi nuclear power plant station. To this end, the following technologies are developed and SMLO loop-based seamless integration is realized as shown in Fig. 1.

• A sensor system that measures heat source, radiation source, water source, other substance information as well as color and shape information in the environment.
• SLAM (simultaneous localization and mapping) scheme that generates precise a wide area semantic survey map for learning-based motion generation of the mobile robot.
• A route generation system that performs reinforcement learning based on the built map from the SLAM scheme. An operator is able to control the robot semi-automatically based on the generated route.

The generated semantic survey map can be used for the prevention of secondary disasters and recovery plans, given that it contains useful information for the disaster environment.


Fig. 1 Conceptual image of semantic survey map building process based on SMLO loop.


Mov. 1 Environmental sensing and reinforcement learning by SMLO system
(quote from Kono Lab. at Tokyo Polytechnic University).


Mov. 2 Semantic survey map building framework using semi-autonomous mobile robot in disaster area
[ https://clads.jaea.go.jp/video/].
3D Temperature Mapping by Mobile Robot
In the decommissioning of nuclear facilities, it is important to investigate the temperature distribution inside. In this study, we propose a method to generate 3D map information including temperature distribution using a mobile robot equipped with a depth camera based on NIR (near-infrared) and a thermography based on FIR (far-infrared). To this end, a novel calibration scheme for the depth camera and thermography line information is proposed given that the NIR image and the FIR image have the similar mechanism of projecting infrared rays into the image, the line distributions in both images are considered to be similar. The experimental results demonstrate that our mapping framework can generate reliable 3D temperature.


Mov. 3 3D Temperature mapping by fusion of depth camera and thermography mounted on mobile robot.



Mov. 4 3D Temperature mapping by fusion of LiDAR and thermography mounted on exploration robot.


Related Paper
• 畠山 佑太, 藤井 浩光, 堂前 雅仁, 河野 仁, 池 勇勳, "暗所探査における視野明瞭化のための温度情報と偏光情報を統合した3次元計測システム," 第22回システムインテグレーション部門講演会（SI2021）, 鹿児島, pp. 33-36, December 2021. (SI2021優秀講演賞受賞)
• 佐藤 弘和, 池 勇勳, 藤井 浩光, 河野 仁, "転移強化学習における環境適応性能向上を目的とした転移率自動調整法," 2021年電気学会電子・情報・システム部門大会, 富山, pp.706-710, September 2021.
• 河野 仁, 池 勇勳, 藤井 浩光, "被災地情報収集のための半自律移動ロボットを用いたセマンティックサーベイマップ生成システムの開発," 2021年電気学会電子・情報・システム部門大会, 富山, pp.702-705, September 2021.
• Ryosuke Kataoka, Isao Tadokoro, Yonghoon Ji, Hiromitsu Fujii, Hitoshi Kono, and Kazunori Umeda, "Performance Improvement of SLAM Based on Global Registration Using LiDAR Intensity and Measurement Data of Puddle," Proceedings of the 18th International Conference on Ubiquitous Robots (UR2021), pp. 553-556, Gangneung , Korea, July 2021. [doi:10.1109/UR52253.2021.9494671]
• 岡 翔平, 池 勇勳, 藤井 浩光, 河野 仁, "被災地環境における直線情報に基づく深度カメラとサーモグラフィの融合による移動ロボットを用いた3次元温度情報マッピング," 日本機械学会ロボティクス・メカトロニクス講演会21講演論文集 (ROBOMECH2021), 大阪, June 2021.
• 片岡 良介, 田所 功, 池 勇勳, 藤井 浩光, 河野 仁, 梅田 和昇, "LiDARの反射強度及び溜水の計測情報を利用した大域的点群位置合わせによるSLAMの性能向上," 日本機械学会ロボティクス・メカトロニクス講演会21講演論文集 (ROBOMECH2021), 大阪, June 2021.
• 長坂 拓海, 藤井 浩光, 河野 仁, 池 勇勳, "環境情報の3次元提示のための距離センサを用いた異種複数センサ統合," 日本機械学会ロボティクス・メカトロニクス講演会21講演論文集 (ROBOMECH2021), 大阪, June 2021.
• Ryosuke Kataoka, Ryuki Suzuki, Yonghoon Ji, Hiromitsu Fujii, Hitoshi Kono, and Kazunori Umeda, "ICP-based SLAM Using LiDAR Intensity and Near-infrared Data," Proceedings of the 2021 IEEE/SICE International Symposium on System Integration (SII2021), pp. 199-104, Iwaki, Japan, January 2021. [doi:10.1109/IEEECONF49454.2021.9382647]
• Ryuki Suzuki, Ryosuke Kataoka, Yonghoon Ji, Hiromitsu Fujii, Hitoshi Kono, and Kazunori Umeda, "SLAM Using ICP and Graph Optimization Considering Physical Properties of Environment," Proceedings of the 2020 21st International Conference on Research and Education in Mechatronics (REM2020), Cracow, Poland, December 2020. [doi:10.1109/REM49740.2020.9313074]
• 菅原 岬, 藤井 浩光, 河野 仁, 池 勇勳, "水の近赤外線の吸光特性を用いた水系領域の3次元提示," 第21回計測自動制御学会システムインテグレーション部門講演会 (SI2020), 1A3-10, pp. 118-112, オンライン, December 2020. (SI2020優秀講演賞受賞)
• 片岡 良介, 鈴木 龍紀, 池 勇勳, 藤井 浩光, 河野 仁, 梅田 和昇, "LiDARの反射強度及び溜水の計測情報を利用したICPによるSLAM," 第38回日本ロボット学会学術講演会予稿集 (RSJ2020), オンライン, October 2020.
• 長坂 拓海, 藤井 浩光, 河野 仁, 池 勇勳, "水面の揺動による偏光情報の変化を用いた溜水位置のステレオ計測," 日本機械学会ロボティクス・メカトロニクス講演会20講演論文集 (ROBOMECH2020), 金沢, May 2020.
• 佐藤 弘和, 大津 亮二, 池 勇勳, 藤井 浩光, 河野 仁, "強化学習における転移学習のための転移率自動推定法," 日本機械学会ロボティクス・メカトロニクス講演会20講演論文集 (ROBOMECH2020), 金沢, May 2020.
• 鈴木 龍紀, 片岡 良介, 池 勇勳, 藤井 浩光, 河野 仁, 梅田 和昇, "環境の持つ物理的属性を考慮した 3 次元地図生成," 日本機械学会ロボティクス・メカトロニクス講演会20講演論文集 (ROBOMECH2020), 金沢, May 2020.
• 片岡 良介, 鈴木 龍紀, 池 勇勳, 藤井 浩光, 河野 仁, 梅田 和昇, "環境の持つ物理的属性を考慮したICPとループ閉じ込みにおけるSLAM," 第25回ロボティクスシンポジア, 函館, March 2020. (Reviewed)
• 菅原 岬, 藤井 浩光, 河野 仁, 池 勇勳, "遠隔操作ロボットによる水源サーベイマップ構築のための近赤外線情報の3次元可視化," 第20回計測自動制御学会システムインテグレーション部門講演会講演論文集 (SI2019), 高松, December 2019.
• Hiromitsu Fujii, Misaki Sugawara, Hitoshi Kono, and Yonghoon Ji, "3D Visualization of Near-Infrared Information for Detecting Water Source," Proceedings of the Fukushima Research Conference 2019 on Remote Technologies for Nuclear Facilities (FRC2019), Fukushima, Japan, p. 5, October 2019.
• Hitoshi Kono, Tomohisa Mori, Yonghoon Ji, Hiromitsu Fujii, Tsuyoshi Suzuki, "Development of Perilous Environment Estimation System by Rescue Robot Using On-board LiDAR for Teleoperator," Proceedings of the 2019 IEEE/SICE International Symposium on System Integrations (SII2019), pp.7-10, Paris, France, January 2019. [Link]
• Yonghoon Ji, Hiromitsu Fujii, and Hitoshi Kono, "Semantic Survey Map Building Framework Using Semi-autonomous Mobile Robot in Disaster Area," Proceedings of the Fukushima Research Conference 2018 on Remote Technologies for Nuclear Facilities (FRC2018), Fukushima, Japan, p. 20, October 2018.