The Manus Glove Is Brilliant. The Data Problem Is Not.

So one of the biggest challenges in humanoid robotics today is the problem of training data. There's never enough and never quite granular enough. ManusMeta have

data. There's never enough and never quite granular enough. ManusMeta have

unveiled a cutting edge haptic glove that offers precision finger tracking and haptic feedback for humanoid teleoperation. So could this be a game changer for robotic control and embodied AI training? We've got just the right mix of experts to figure this out. Scott Walter, Gustav Anderson, and Mehrdad Charamani are in the house. OK,

this out. Scott Walter, Gustav Anderson, and Mehrdad Charamani are in the house. OK,

so where should we begin? So we can see this. This looks very, very good.

And you can just start counting, again, all your joints that you have. And you

have to assume, is stuff going on in the metacarpal and are they including the wrist in there? Now, I don't think they're including the wrist in there. What I

think they're doing moving very much at the MP joints, CMC joint aside from the thumb. I tried to see, do they have the fifth, but it just looks like

thumb. I tried to see, do they have the fifth, but it just looks like it's the MP level, not at the same level. At one point in there, they kind of show what the tracking is. I think it's either in that or maybe it's their website. I have to remember where I looked at it. And they were showing basically all the points that are up there. And so when you begin to

look at the different, yeah. It had a role in the MP joint or rotation. So when it kind of pushes at the side, you saw that the bone

rotation. So when it kind of pushes at the side, you saw that the bone actually rotates at the MP joint of the index finger, which it does in us.

It doesn't do that in the hinge-like robotics, because then it's just flex, extend, abduct, abduct. But there is rotation in all of our joints, basically. I think that's the

abduct. But there is rotation in all of our joints, basically. I think that's the one you're talking about, right? Yeah, exactly. When it pushes there, you can kind of see that the index finger kind of twists in towards the palm. And I mean, unless they constrain it in the blender or whatever software they're running it in, you would get just adaptation of the. This, by the way, is the Manus

haptic glove. Yes. OK. It's not a hand. It's a haptic glove. But

haptic glove. Yes. OK. It's not a hand. It's a haptic glove. But

monis means hands. I know the problem is like on their website, they're running a GIF of some sort that I have a hard time getting a screen grab of because it keeps on moving. And it looks like they're tracking different positions in space. So I'm seeing on the fingers, they basically have four points that they're following.

space. So I'm seeing on the fingers, they basically have four points that they're following.

The tip up here, here, here, and here. They seem to be following those, tracking them. And the thumb, they're down there. It looks like they have like a couple

them. And the thumb, they're down there. It looks like they have like a couple more down in there. And I believe that's where they're, yes, exactly. That's what we're looking for. So I think if you count those cubes, there's like 25 of them

looking for. So I think if you count those cubes, there's like 25 of them and that might be what they're calling their 25 degrees of freedom. 25 degrees of freedom. Yeah. Yeah. Something like that. Um, and so you, you can see, and one

freedom. Yeah. Yeah. Something like that. Um, and so you, you can see, and one of them might be the rest. I'm not a hundred percent sure. Cause you see that one that's floating down the middle there on the very bottom. Can I

add something? Maybe it's an IMU. IMU gives you three degrees of volume, right?

So if you have IMU on this glove, it can show all the movements in this space and also rotation and flexion. Yeah, yeah, yeah. Could be. Could be. It's

the IMU down there. And then you're seeing where the IMU points are down the bottom of where the metacarpals are connecting. So basically, it gives you all the dots that you need to reconstruct that skeleton. Exactly. Yeah, so it says 25 there and somewhere else we talked about 27 Dauphin. So what is the benefit? So now they will feel it in the glove as well when the robot is doing something? So

there's haptic feedback that the glove gives you. It's not just tracking and teleoperations, but also haptic feedback for, I think, this would be in simulation, Scott? Yes, so you could use it in reality and simulation. So if you're teleoperating

Scott? Yes, so you could use it in reality and simulation. So if you're teleoperating a robot, you would feel the force feedback coming from the bot itself when it's grabbing something. And it could also be used in simulation. So if you're in a

grabbing something. And it could also be used in simulation. So if you're in a virtual world and you go to grab a virtual bottle, you would get some sort of haptic feedback to say, oh, guess what? That's what it feels like. And of

course, a lot of this technology has developed from VR gaming, virtual reality gaming. Yes,

yes. Now, I'm not sure how haptic it is in the sense that is it just kind of vibrations or other sensations that's going on there where you can still close in on it versus something like Ready Player One, where the haptics were actually real. And that if your finger hit something hard, you kind of felt it. And

real. And that if your finger hit something hard, you kind of felt it. And

your hand was restricted from movement. But that means you need to have some sort of exoskeleton on there that's tugging. Yeah. And there are those kind of gloves that we use in patients to strengthen them. So where you kind of activate a muscle, then there are tendons as an exoskeleton that helps you grip like a power glove.

Yeah. I couldn't see any cables in this design. Yeah, and actually, you can see the cables. They have some videos, and there's like one service video that shows you

the cables. They have some videos, and there's like one service video that shows you how to wash the glove. And you can remove everything out. So they show you how to take the sensors out, the wires that are going in there. Because at

the tips, you can see there was something in the tips. Yeah, something. Yeah, up

there. And basically, you pop the whole thing out. You pull the electronics that are on the backside out completely. And now you just have a glove, and you can wash the glove. They show you. You know, what temperature settings, you can throw it in the dryer and everything. And then when it's fully dry, you can go ahead and put everything back in. So that means the glove does not have any electronics

in there. The electronics is out at the tip. In terms of granularity of data

in there. The electronics is out at the tip. In terms of granularity of data captured through this glove, how is it different from, let's say, what Sunday Robotics has or what other competitors have? Or in Kinect, the old. Yeah.

Yeah. It's hard to say. I'm not sure. So they're saying, you know, millimeter level precision, which is about the best that I think any of the gloves are out there. Some of them are quite a lot worse than that, being able to capture

there. Some of them are quite a lot worse than that, being able to capture it. The difference with the Yumi is that it is kind of an exoskeleton that

it. The difference with the Yumi is that it is kind of an exoskeleton that you've got on there. So here you've got something that's very soft and you've got IMUs and you're trying to reconstruct the position, the gesture that's going on. And they

seem to be doing a pretty good job of that. As we can see there with those pinches, it's working very well. The idea of the Yumi glove is that you're actually fitting your hand on the glove, which means the actual robotic fingers are moving, and the encoders in there are actually measuring everything that's going on. So you

get a very faithful reproduction as a result of that. I wish they showed that in this clip, like with all the robotic hands, and we might talk about one more later. But they never show the full fist. It's like, I want to see if it does represent the full fist on skin. like halfway Kapani and

maybe a slight fist, but they don't do the full fist. And I mean, I would like to probably can do that. It doesn't really do it. So I wish they should be like, we have to make a guideline, like do these tests to show that you can actually do these simple steps. Then Gustav is happy. Yeah, you

should rush through that project of yours. Gustav score, yeah. Gustav score, yeah. It would

be very popular if you do it. I know some companies, they need such a thing. They make their own thing. For example, this motion is awesome. You mean more

thing. They make their own thing. For example, this motion is awesome. You mean more popular than he already is? Yeah. Don't let it go to his head.

Yeah. What do you mean this motion? Yeah. This motion is important. Most of the robotics hand cannot do this because it requires adoption, adduction, right? And also,

it also requires all of them coming together. Scott has got some thoughts running through it. I didn't hear that. Maybe I shouldn't have. Me neither. No, it's like, what

it. I didn't hear that. Maybe I shouldn't have. Me neither. No, it's like, what are you talking about? Good save.

Yeah, cool. So what potential do you see for companies that choose to go down this path, is the hardware of the robotic hands good enough to deploy the sort of granularity of data that you can capture with your human hand and a haptic glove of this nature?

I mean, will you have enough of them? I mean, if you take a comparison with FSD, you have this billions of miles of data and you want to build you want everyone in the world won't have a haptic glove on them i'm thinking you just need video in the end maybe in the beginning it's good to have like the basic setup but then to train like the all the different types of

motions i can't see that them getting around doing it video based like looking at someone doing it in a video and the AI can infer how that man's glove would be positioned to do that task. But maybe it's maybe I'm not in the field, so maybe I'm thinking wrong. What about hardware limitations? I mean, you look

at the best robotic hands out there right now. Are they in terms of hardware, actuators, flanges, tendons, everything, whatever you can have the best combination of? Do they compare effectively enough to a human hand?

I still haven't seen one that matches one to one. I mean, maybe they don't have to. Yeah. Yeah. They don't have to, because

have to. Yeah. Yeah. They don't have to, because every human hand is different. We still seem to all be able to do the same tasks. You know, some, obviously some people a little bit better, um, the, the

same tasks. You know, some, obviously some people a little bit better, um, the, the hands that are used for like a pipe fitter probably aren't the best for playing a piano and vice versa. Um, but. But still, there's a wide range of things that we are able to overlap on. And everyone has different size hands, different length digits, different strength. It just has to be close enough.

But I wonder, like taking the Tesla bot catching the ball, for instance, because it catches it with a tripod grip. I wonder if the teleoperator kind of caught it like we would with all five fingers, but the bot kind of just got it in this direction because it cannot do that motion as we could. Because both

catches is a tripod grip, the fingers go down to the side of the ball.

But I don't see that, or maybe the teleoperator caught it like that, but I would expect him to catch it like we would with a full palm and all the fingers. So maybe it's OK that it's not exactly one-to-one. At least it's close

the fingers. So maybe it's OK that it's not exactly one-to-one. At least it's close enough, and you would get the same end result. Yeah, and how much of our catching is muscle memory? It's more or less like we just fire that grip, and it just happens to get itself around that shape. So it's not like I was

mentally thinking I want my pinky to come on down there. Just like, that's what kind of happened. And I'm sure the teleoperator was the same thing. you know there was there was a little there was a lot of skill there but there was like a little bit of luck no no doubt about it because i'm sure there are many tries that like completely missed so so finally get in there and happen

to get in the range and everything closed down at the right time it's like hey good we got it and then they tossed it again two in a row all right save that one and then put it out there but you're not seeing the hundreds of tries where they're missing because you know it's very difficult between the lag and everything else the teleoperator is looking through optimist's eyes and probably is not

sure what he's seeing, especially if it's distorted. And so he's got to get that all down. And the trick there was the person throwing the ball was a nice

all down. And the trick there was the person throwing the ball was a nice lob thrown right at the face. So it was like right up there. So there's

no doubt about it being over here. I mean, having to search where the ball is, I could see it come in and he could consistent throw every time, not a fastball, just a nice little lob up there, good hang time. And then, you know, it was like going to be right where it is and then swat it and got it. And then you have to do the anticipation. So if you ever

saw, like, I think it was the launch of Apollo 15 on the moon where the lunar module was taking off. They had the camera out there and they had like a like one and a half second lag time between it. And the operator in Houston was like really nervous looking at the countdown. He had to push the button for that thing to start panning up. way before it happened because he had

to anticipate what was going on. So that was like the world's best teleoperation of a camera to be able to actually track something at the same time. It was

just remarkable. So they had to... Yeah, yeah, exactly. So catching

that ball, I'm sure there was like a lot of anticipation because they had seen it enough and then just hope that hand grasped it.