Magni human/robot web interface:

General Questions
1

Has anyone developed a web based html document to display Magni status
and control motion? I am working on an implementation to show enable/disable status, battery status and left, front, right distance and provide both keyboard and joystick motion control. I am having difficulty displaying the video image from Magni’s camera. Links to successful efforts would be appreciated.

1 Like
2

Yes, we have an application like that in development. It has video streaming, robot control, telemetry, rosbag recording, rqt reconfigure and full lidar slam display and route navigation. And supports all browsers mobile/desktop. I have a somewhat dated gazebo demo video on hand. Currently closed source for the time being though, for obvious reasons.

I can give you some tips regarding video streaming though, the easiest way to do it is to simply subscribe to the camera topic using rosbridge, but that’s pretty slow (image sent as hex string).

So the next option is using the web_video_server which can do http streaming which at least no longer clogs up rosbridge, but is still pretty slow with larger resolutions (image sent as jpeg).

The best option is however webrtc_ros, which does full webrtc streaming (image sent as h264) that is fast enough to allow teleop with basically no latency over decent wifi (but takes up a fair bit of CPU on the Pi). Browser support for webrtc is a bit iffy though, so we still use the web video server as a fallback.

1 Like
3

Thanks for the excellent reply. Hoping to get rosbridge working as a first step.

If you have the rosbridge code working perhaps you can spot my error(s).
My setup is as follows:
Using two systems with multiple terminal sessions:

m - Magni using Pi 4B
u - ubuntu 18.04 on x86 chip set

  • u1: roslaunch rosbridge_server rosbridge_websocket.launch
  • m1: rosrun sonar_detector sonar_detector >/dev/null 2>&1 &
  • m2: rostopic echo /system_control
  • m3: roslaunch raspicam_node camerav2_1280x960.launch

use the file manager to access gui.html file and then
open to chrome on u1804
use chrome extensions to observe results [ctrl+ shift + I]

in gui.html:

div class=“row-4”
div class=“col-41”
video src=“” id=“video” width=“1280” height=“960” alt=“Robot view of world”
/div
/div

in webui.js:

// Subscribe to video topic
var video = new ROSLIB.Topic({
ros : ros,
name : ‘/raspicam_node/image/compressed’,
messageType : ‘sensor_msgs/CompressedImage’
});

video.subscribe(function(m) {
document.getElementById(“video”).value = m.data;
// console.log(m.format);
});

in robot.css:

.row-4
{
/*border: 2px solid black; */
position: relative;
margin-top: 20px;
width: 12808px;
height: 968px;
}
.col-41
{
border: 2px solid black;
position: relative;
margin-top: 2px;
left: 2px;
width: 1280px;
height: 960px;
}

Results:

  1. all fields show correctly
  2. chrome console shows no errors
  3. video frame is empty
  4. message to console for each video input does display [getting video traffic]
  5. message to console for msgs format shows jpeg

Some characters <> removed to allow stmts to display in post.

Thoughts?

4

video src="" id=“video” width=“1280” height=“960” alt=“Robot view of world”

At first glance that might be the problem, the data you get isn’t video, it’s an image and should be fed into the img element. At least I think that’s what I remember working at the time.

Btw if you want code to display in a normal way you should put it in backticks or mark it using the preformatted text thing.

1 Like
5

MoffKalast, Thanks for sharing your most valuable possessions: your time and your knowledge.

1 Like
6

Made the following changes:
in gui.html:
<img src="" id=“video” width=“1280” height=“960” alt=“Robot view of world”

in webui.js, replaced all with:
robot_IP = “magni”

// get handle for video placeholder
video = document.getElementById(‘video’);
// Populate video source
video.src = “http://” + robot_IP + “:8080/stream?topic=/raspicam_node/image&image_transport=compressed&type=ros_compressed”;
video.onload = function (m) {
console.log("video loaded ");
};

Added the web_video_server:

  • m4: rosrun web_video_server web_video_server

Can now display Magni’s view of thr world.

Then modified to reduce the image size and performance is very good.

1 Like