[Unitree Go1][ZED2] Setting up ZED2 for Unitree Go1

Vishnu · April 16, 2024, 9:45am

Hello Team,
Do you have any guidelines on how to set up ZED2 for Go1.

I am facing error “CAMERA FAILED TO SETUP” when I run the zed_wrapper or even using python library. The ZED2 was initially set up by my colleague and he had used through python. But now when i try, i am getting this error.

The ZED SDK is installed in NVIDIA 15 board. And camera is connected to the correct usb (middle one).

Also i am unable to run ‘ZED_Depth_Viewer’ also.

unitree@nx:/usr/local/zed/tools$ ./ZED_Depth_Viewer 
nvbuf_utils: Could not get EGL display connection
QXcbIntegration: Cannot create platform OpenGL context, neither GLX nor EGL are enabled
QXcbIntegration: Cannot create platform OpenGL context, neither GLX nor EGL are enabled
QOpenGLWidget: Failed to create context
QXcbIntegration: Cannot create platform OpenGL context, neither GLX nor EGL are enabled
QOpenGLWidget: Failed to create context
QXcbIntegration: Cannot create platform OpenGL context, neither GLX nor EGL are enabled
QOpenGLWidget: Failed to create context
composeAndFlush: makeCurrent() failed
composeAndFlush: makeCurrent() failed
composeAndFlush: makeCurrent() failed
composeAndFlush: makeCurrent() failed
composeAndFlush: makeCurrent() failed
composeAndFlush: makeCurrent() failed
composeAndFlush: makeCurrent() failed
composeAndFlush: makeCurrent() failed
composeAndFlush: makeCurrent() failed
composeAndFlush: makeCurrent() failed
composeAndFlush: makeCurrent() failed
composeAndFlush: makeCurrent() failed
composeAndFlush: makeCurrent() failed
composeAndFlush: makeCurrent() failed
in bool ImageHandler::initialize(sl::Mat&) : Err [219]: invalid OpenGL or DirectX context.
Stack trace (most recent call last):
#2    Object "./ZED_Depth_Viewer", at 0x459d87, in 
#1    Object "/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1", at 0x7fb17ddc4f, in cuGraphicsMapResources
#0    Object "/usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1", at 0x7fb16c8804, in 
Segmentation fault (Address not mapped to object [0x76bd4b922ac90810])
Segmentation fault (core dumped)

Sohail · April 18, 2024, 8:48am

Dear @Vishnu,

The ZED2 has to be installed according to the compatible board via the downloads. Also, as mentioned in their documentation, it should be attached to a 3.0 USB port. For your usecase, if the ZED is installed try and run the ZED2 ros driver and check if you are getting the required data from there.

Vishnu · April 18, 2024, 9:19am

Hello @Sohail ,

I ran ‘roslaunch zed_wrapper zed2.launch’ and getting CAMERA FAILED TO SETUP
Is this the driver file?

With what i found in references, go1 doesnot have usb 3.0. Is it correct?
For now, i have connected to the usb linked to board 15.

Output of zed2.launch

unitree@nx:~$ roslaunch zed_wrapper zed2.launch 
... logging to /home/unitree/.ros/log/785719f8-fd5d-11ee-bd55-48b02da81fb9/roslaunch-nx-12919.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://192.168.123.15:42383/

SUMMARY
========

PARAMETERS
 * /rosdistro: melodic
 * /rosversion: 1.14.13
 * /zed2/zed2_description: <?xml version="1....
 * /zed2/zed_node/auto_exposure_gain: True
 * /zed2/zed_node/auto_whitebalance: True
 * /zed2/zed_node/brightness: 4
 * /zed2/zed_node/contrast: 4
 * /zed2/zed_node/depth/depth_downsample_factor: 0.5
 * /zed2/zed_node/depth/depth_stabilization: 1
 * /zed2/zed_node/depth/max_depth: 20.0
 * /zed2/zed_node/depth/min_depth: 0.3
 * /zed2/zed_node/depth/openni_depth_mode: False
 * /zed2/zed_node/depth/quality: 1
 * /zed2/zed_node/depth/sensing_mode: 0
 * /zed2/zed_node/depth_confidence: 30
 * /zed2/zed_node/depth_texture_conf: 100
 * /zed2/zed_node/exposure: 100
 * /zed2/zed_node/gain: 100
 * /zed2/zed_node/gamma: 8
 * /zed2/zed_node/general/base_frame: base
 * /zed2/zed_node/general/camera_flip: False
 * /zed2/zed_node/general/camera_model: zed2
 * /zed2/zed_node/general/camera_name: zed2
 * /zed2/zed_node/general/gpu_id: -1
 * /zed2/zed_node/general/grab_frame_rate: 15
 * /zed2/zed_node/general/resolution: 2
 * /zed2/zed_node/general/self_calib: True
 * /zed2/zed_node/general/serial_number: 0
 * /zed2/zed_node/general/svo_compression: 2
 * /zed2/zed_node/general/verbose: False
 * /zed2/zed_node/general/zed_id: 0
 * /zed2/zed_node/hue: 0
 * /zed2/zed_node/mapping/clicked_point_topic: /clicked_point
 * /zed2/zed_node/mapping/fused_pointcloud_freq: 1.0
 * /zed2/zed_node/mapping/mapping_enabled: False
 * /zed2/zed_node/mapping/max_mapping_range: -1
 * /zed2/zed_node/mapping/resolution: 0.05
 * /zed2/zed_node/object_detection/body_fitting: False
 * /zed2/zed_node/object_detection/confidence_threshold: 50
 * /zed2/zed_node/object_detection/max_range: 15.0
 * /zed2/zed_node/object_detection/mc_animal: True
 * /zed2/zed_node/object_detection/mc_bag: True
 * /zed2/zed_node/object_detection/mc_electronics: True
 * /zed2/zed_node/object_detection/mc_fruit_vegetable: True
 * /zed2/zed_node/object_detection/mc_people: True
 * /zed2/zed_node/object_detection/mc_sport: True
 * /zed2/zed_node/object_detection/mc_vehicle: True
 * /zed2/zed_node/object_detection/model: 0
 * /zed2/zed_node/object_detection/object_tracking_enabled: True
 * /zed2/zed_node/object_detection/od_enabled: False
 * /zed2/zed_node/point_cloud_freq: 10.0
 * /zed2/zed_node/pos_tracking/area_memory: True
 * /zed2/zed_node/pos_tracking/area_memory_db_path: zed_area_memory.area
 * /zed2/zed_node/pos_tracking/fixed_z_value: 0.0
 * /zed2/zed_node/pos_tracking/floor_alignment: False
 * /zed2/zed_node/pos_tracking/imu_fusion: True
 * /zed2/zed_node/pos_tracking/init_odom_with_first_valid_pose: True
 * /zed2/zed_node/pos_tracking/initial_base_pose: [0.0, 0.0, 0.0, 0...
 * /zed2/zed_node/pos_tracking/map_frame: map
 * /zed2/zed_node/pos_tracking/odometry_frame: odom
 * /zed2/zed_node/pos_tracking/path_max_count: -1
 * /zed2/zed_node/pos_tracking/path_pub_rate: 2.0
 * /zed2/zed_node/pos_tracking/pos_tracking_enabled: True
 * /zed2/zed_node/pos_tracking/publish_map_tf: True
 * /zed2/zed_node/pos_tracking/publish_tf: True
 * /zed2/zed_node/pos_tracking/save_area_memory_db_on_exit: False
 * /zed2/zed_node/pos_tracking/two_d_mode: False
 * /zed2/zed_node/pub_frame_rate: 15.0
 * /zed2/zed_node/saturation: 4
 * /zed2/zed_node/sensors/max_pub_rate: 200.0
 * /zed2/zed_node/sensors/publish_imu_tf: True
 * /zed2/zed_node/sensors/sensors_timestamp_sync: False
 * /zed2/zed_node/sharpness: 4
 * /zed2/zed_node/stream: 
 * /zed2/zed_node/svo_file: 
 * /zed2/zed_node/video/extrinsic_in_camera_frame: True
 * /zed2/zed_node/video/img_downsample_factor: 0.5
 * /zed2/zed_node/whitebalance_temperature: 42

NODES
  /zed2/
    zed2_state_publisher (robot_state_publisher/robot_state_publisher)
    zed_node (zed_wrapper/zed_wrapper_node)

ROS_MASTER_URI=http://192.168.123.14:11311

process[zed2/zed2_state_publisher-1]: started with pid [12945]
process[zed2/zed_node-2]: started with pid [12946]
[ INFO] [1713431836.021705102]: Initializing nodelet with 6 worker threads.
nvbuf_utils: Could not get EGL display connection
[ INFO] [1713431836.219566520]: ********** Starting nodelet '/zed2/zed_node' **********
[ INFO] [1713431836.219805403]: SDK version : 3.8.2
[ INFO] [1713431836.220004573]: *** GENERAL PARAMETERS ***
[ INFO] [1713431836.222631735]:  * Camera Name			-> zed2
[ INFO] [1713431836.232263190]:  * Camera Resolution		-> HD720
[ INFO] [1713431836.247098250]:  * Camera Grab Framerate	-> 15
[ INFO] [1713431836.251359828]:  * Gpu ID			-> -1
[ INFO] [1713431836.256913195]:  * Camera ID			-> -1
[ INFO] [1713431836.264091986]:  * Verbose			-> DISABLED
[ INFO] [1713431836.268836353]:  * Camera Flip			-> DISABLED
[ INFO] [1713431836.273015563]:  * Self calibration		-> ENABLED
[ INFO] [1713431836.278185694]:  * Camera Model by param	-> zed2
[ INFO] [1713431836.278287167]: *** VIDEO PARAMETERS ***
[ INFO] [1713431836.280924057]:  * Image resample factor	-> 0.5
[ INFO] [1713431836.283214960]:  * Extrinsic param. frame	-> X RIGHT - Y DOWN - Z FWD
[ INFO] [1713431836.283315889]: *** DEPTH PARAMETERS ***
[ INFO] [1713431836.285578503]:  * Depth quality		-> PERFORMANCE
[ INFO] [1713431836.290806715]:  * Depth Sensing mode		-> STANDARD
[ INFO] [1713431836.293219539]:  * OpenNI mode			-> DISABLED
[ INFO] [1713431836.298550152]:  * Depth Stabilization		-> ENABLED
[ INFO] [1713431836.301029633]:  * Minimum depth		-> 0.3 m
[ INFO] [1713431836.303663867]:  * Maximum depth		-> 20 m
[ INFO] [1713431836.315824691]:  * Depth resample factor	-> 0.5
[ INFO] [1713431836.315980085]: *** POSITIONAL TRACKING PARAMETERS ***
[ INFO] [1713431836.321216041]:  * Positional tracking		-> ENABLED
[ INFO] [1713431836.324180966]:  * Path rate			-> 2 Hz
[ INFO] [1713431836.327442151]:  * Path history size		-> 1
[ INFO] [1713431836.335369333]:  * Odometry DB path		-> /home/unitree/.ros/zed_area_memory.area
[ INFO] [1713431836.340286694]:  * Save Area Memory on closing	-> DISABLED
[ INFO] [1713431836.345305528]:  * Area Memory			-> ENABLED
[ INFO] [1713431836.352174364]:  * IMU Fusion			-> ENABLED
[ INFO] [1713431836.357637970]:  * Floor alignment		-> DISABLED
[ INFO] [1713431836.367090416]:  * Init Odometry with first valid pose data -> ENABLED
[ INFO] [1713431836.379986992]:  * Two D mode			-> DISABLED
[ INFO] [1713431836.385990795]: *** MAPPING PARAMETERS ***
[ INFO] [1713431836.390720378]:  * Mapping			-> DISABLED
[ INFO] [1713431836.396058319]:  * Clicked point topic		-> /clicked_point
[ INFO] [1713431836.396223857]: *** OBJECT DETECTION PARAMETERS ***
[ INFO] [1713431836.401420260]:  * Object Detection		-> DISABLED
[ INFO] [1713431836.401553414]: *** SENSORS PARAMETERS ***
[ INFO] [1713431836.403985694]:  * Sensors timestamp sync	-> DISABLED
[ INFO] [1713431836.406390038]:  * Max sensors rate		-> 200
[ INFO] [1713431836.406523063]: *** SVO PARAMETERS ***
[ INFO] [1713431836.411239462]:  * SVO input file: 		-> 
[ INFO] [1713431836.413619997]:  * SVO REC compression		-> H265 (HEVC)
[ INFO] [1713431836.418297996]: *** COORDINATE FRAMES ***
[ INFO] [1713431836.431934899]:  * map_frame			-> map
[ INFO] [1713431836.432072885]:  * odometry_frame		-> odom
[ INFO] [1713431836.432392696]:  * base_frame			-> base
[ INFO] [1713431836.432719963]:  * camera_frame			-> zed2_camera_center
[ INFO] [1713431836.433160095]:  * imu_link			-> zed2_imu_link
[ INFO] [1713431836.433235008]:  * left_camera_frame		-> zed2_left_camera_frame
[ INFO] [1713431836.433558947]:  * left_camera_optical_frame	-> zed2_left_camera_optical_frame
[ INFO] [1713431836.433728901]:  * right_camera_frame		-> zed2_right_camera_frame
[ INFO] [1713431836.434004232]:  * right_camera_optical_frame	-> zed2_right_camera_optical_frame
[ INFO] [1713431836.434253962]:  * depth_frame			-> zed2_left_camera_frame
[ INFO] [1713431836.434508941]:  * depth_optical_frame		-> zed2_left_camera_optical_frame
[ INFO] [1713431836.434697327]:  * disparity_frame		-> zed2_left_camera_frame
[ INFO] [1713431836.434951569]:  * disparity_optical_frame	-> zed2_left_camera_optical_frame
[ INFO] [1713431836.435248180]:  * confidence_frame		-> zed2_left_camera_frame
[ INFO] [1713431836.435501463]:  * confidence_optical_frame	-> zed2_left_camera_optical_frame
[ INFO] [1713431836.440513800]:  * Broadcast odometry TF	-> ENABLED
[ INFO] [1713431836.446047807]:  * Broadcast map pose TF	-> ENABLED
[ INFO] [1713431836.453954318]:  * Broadcast IMU pose TF	-> ENABLED
[ INFO] [1713431836.454204496]: *** DYNAMIC PARAMETERS (Init. values) ***
[ INFO] [1713431836.457092845]:  * [DYN] Depth confidence	-> 30
[ INFO] [1713431836.460777393]:  * [DYN] Depth texture conf.	-> 100
[ INFO] [1713431836.463712782]:  * [DYN] pub_frame_rate		-> 15 Hz
[ INFO] [1713431836.466505258]:  * [DYN] point_cloud_freq	-> 10 Hz
[ INFO] [1713431836.468950658]:  * [DYN] brightness		-> 4
[ INFO] [1713431836.471595037]:  * [DYN] contrast		-> 4
[ INFO] [1713431836.473922836]:  * [DYN] hue			-> 0
[ INFO] [1713431836.476569358]:  * [DYN] saturation		-> 4
[ INFO] [1713431836.479820910]:  * [DYN] sharpness		-> 4
[ INFO] [1713431836.482287303]:  * [DYN] gamma			-> 8
[ INFO] [1713431836.485021826]:  * [DYN] auto_exposure_gain	-> ENABLED
[ INFO] [1713431836.492454732]:  * [DYN] auto_whitebalance	-> ENABLED
[ INFO] [1713431836.564751193]:  * Camera coordinate system	-> Right HANDED Z UP and X FORWARD
[ INFO] [1713431836.565195165]:  *** Opening ZED 2...
[ INFO] [1713431843.131710449]: ZED connection -> CAMERA FAILED TO SETUP
[ INFO] [1713431853.182131892]: ZED connection -> CAMERA FAILED TO SETUP
[ INFO] [1713431862.771295316]: ZED connection -> CAMERA FAILED TO SETUP
[ INFO] [1713431869.885206405]: ZED connection -> CAMERA FAILED TO SETUP
^C[zed2/zed_node-2] killing on exit
[zed2/zed2_state_publisher-1] killing on exit
ZED Nodelet destroyed
shutting down processing monitor...
... shutting down processing monitor complete
done

Sohail · April 18, 2024, 9:41am

The camera is not connected properly or connected to the wrong board, try connecting the camera to your laptop and check to ensure its not the cable or the camera.

I do not remember if the GO1 has USB 3.0 ports you can verify from unitree docs. I think it does not have.

Vishnu · May 17, 2024, 10:36am

Update :
I was able to program ZED with python.

The issue was I had to use reduced resolution (VGA) in the init parameters.
With that change i was able to access ZED (running ssh).

The default ZED apps was still not running in my PC via SSH. Seems to be some issues related to OpenGL libraries. The same worked when ssh from a Nvidia board.