Husky LLM Navigation

ROS Packages for Husky Robot

Dependencies

• fath_pivot_mount_description: $ sudo apt-get install ros-noetic-fath-pivot-mount-description
• flir_camera_description: $ sudo apt-get install ros-noetic-flir-camera-description
• velodyne_description: $ sudo apt-get install ros-noetic-velodyne-description
• realsense2_description: $ sudo apt install ros-noetic-realsense2-description
• LMS1xx: $ sudo apt-get install ros-noetic-lms1xx
• robot_localization: $ sudo apt-get install ros-noetic-robot-localization
• interactive_marker_twist_server: $ sudo apt-get install ros-noetic-interactive-marker-twist-server
• twist_mux: $ sudo apt-get install ros-noetic-twist-mux
• teleop_twist_keyboard: $ sudo apt-get install ros-noetic-teleop-twist-keyboard
• teleop_twist_joy: $ sudo apt-get install ros-noetic-teleop-twist-joy
• rviz_imu_plugin: $ sudo apt-get install ros-noetic-rviz-imu-plugin
• gmapping: $ sudo apt-get install ros-noetic-gmapping
• rtabmap_ros: $ sudo apt-get install ros-noetic-rtabmap-ros

Installation

1. Create a Catkin workspace (if not already present)

$ mkdir -p catkin_ws/src

2. Change directory to the source space (src) of your Catkin workspace

$ cd catkin_ws/src

3. Clone this repository:

$ git clone https://github.com/Tinker-Twins/Husky.git

4. Change directory back to the Catkin workspace:

$ cd ..

5. Build the packages:

$ catkin_make

Usage:

1. Keyboard Teleoperation:

   $ roslaunch husky_gazebo husky_playpen.launch
   $ roslaunch husky_control teleop_keyboard.launch

2. Map-Less Navigation:

   $ roslaunch husky_gazebo husky_playpen.launch
   $ roslaunch husky_viz view_robot.launch
   $ roslaunch husky_navigation map_less_navigation.launch

3. Simultaneous Localization And Mapping (SLAM):

   $ roslaunch husky_gazebo husky_playpen.launch
   $ roslaunch husky_viz view_robot.launch
   $ roslaunch husky_navigation gmapping.launch
   $ roslaunch husky_control teleop_keyboard.launch

   To save generated map to current working directory, run:

   $ rosrun map_server map_saver -f <filename>

4. Adaptive Monte Carlo Localization (AMCL):

   $ roslaunch husky_gazebo husky_playpen.launch
   $ roslaunch husky_viz view_robot.launch
   $ roslaunch husky_navigation amcl.launch
   $ roslaunch husky_control teleop_keyboard.launch

5. Map-Based Navigation:

   $ roslaunch husky_gazebo husky_playpen.launch
   $ roslaunch husky_viz view_robot.launch
   $ roslaunch husky_navigation map_based_navigation.launch

6. RTAB-Map SLAM (3D Point Cloud Mapping):

   Note: Requires Intel RealSense camera enabled. Set the environment variable before launching:

   $ export HUSKY_REALSENSE_ENABLED=1

   SLAM only (build 3D point cloud map with teleoperation):

   $ roslaunch husky_gazebo husky_playpen.launch
   $ roslaunch husky_viz view_robot.launch
   $ roslaunch husky_navigation rtabmap_slam.launch
   $ roslaunch husky_control teleop_keyboard.launch

   SLAM + Autonomous Navigation:

   $ roslaunch husky_gazebo husky_playpen.launch
   $ roslaunch husky_viz view_robot.launch
   $ roslaunch husky_navigation rtabmap_slam_navigation.launch

   RTAB-Map saves its database to ~/.ros/rtabmap.db. To reuse a previously built map for localization only:

   $ roslaunch husky_navigation rtabmap_slam.launch localization:=true

7. LLM Agent Navigation (Scene Graph + GPT-4o):

   Prerequisites:

   $ pip install langchain langchain-openai openai opencv-python
   $ export OPENAI_API_KEY="your_openai_api_key_here"
   $ export HUSKY_REALSENSE_ENABLED=1

   Launch RTAB-Map SLAM navigation first, then start the LLM agent:

   # Terminal 1: Gazebo world
   $ roslaunch husky_gazebo husky_door_world.launch
   
   # Terminal 2: RTAB-Map SLAM + Navigation
   $ roslaunch husky_navigation rtabmap_slam_navigation.launch
   
   # Terminal 3: LLM Agent (interactive text input)
   $ roslaunch husky_llm_navigation llm_navigation.launch

   Example commands in the LLM agent terminal:

   Enter command> Navigate to element 387032
   Enter command> Inspect element 388611, check if the door is open
   Enter command> Go to the nearest door
   

Video Tutorials:

1. Installation
2. Teleoperation
3. Map-Less Navigation
4. SLAM
5. Map-Based Localization
6. Map-Based Navigation