Project Overview
During the first edition of the Real World Robotics class we were tasked with designing, manufacturing, building and programming a dextrous robotic manipulator - all from scratch (well, mostly). Teams of 4-5 people worked to solve teleoperation and reinforcement learning tasks for a semester. In our team, I was responsible for finger design and teleoperation. It was an ambitious challenge, but it let me train some muscles I hadn’t in a bit: CAD design, manufacturing, and simple Blender animations. Take a look at this video featured on the Soft Robotics Lab youtube channel.
Hand design
The problem statement was quite simply formulated: using 10 servos, can you complete a handful (heh) of teleoperated manipulation tasks, as well as autonomously roll a ball while holding it? Restrictions didn’t exist, so we were free to come up with whatever design would be able to conceivably complete these asks. Certainly, some of the provided starting points were most helpful for a mimetic robot hand, such as the hand pose estimation software, but we were free to explore what avenues we found most promising. To balance utility and power, we decided on an underactuated tendon driven system with 3 fingers and a thumb.
I really like the finished look of the robot hand. But not only does it look pretty, it’s also pretty functional. The fingers have two addressable degrees of freedom: the first joint is independent and the following two are linked. Each degree of freedom corresponds to a single servo that is responsible for both the flexion and extension movements. The first two joints use a rolling contact mechanism, meaning that the two surfaces never slide against eachother, instead rolling and decreasing the nasty friction that would steal the limited power of our servos. Rolling contact joints work by way of two crossing and opposing pieces of string that stabilize the joint. These are notoriously difficult to mount and keep at the desired tightness; so, we designed a simple system that solves both of these problems. It’s the red nubs on the side of the fingers that increase the allowed margin of error during assembly and that can be tightened down the road.
The sleek outside hides a complex internal routing of tendons. See for yourself - you can drag the slider down and reveal a cross-section. The finger design is the same across the hand, even the thumb. As a result the thumb looks somewhat freakishly long, but that really helped in some of the teleoperation tasks.
Control
This is the not-from-scratch part. For teleoperation, we had Oak-D depth cameras with a hand pose reconstruction pipeline running on them, which we used to retarget someone’s hand motion directly to the robot hand. The system tracks the operator’s hand in 3D space and estimates joint angles. We developed a retargeting algorithm to map these human hand motions to our robot’s kinematics.
It’s a surreal feeling to see a robot hand copying your movements.
Finally, to complete the spinning ball task, we used RL in IsaacGym to train a policy. By doing domain randomizations and introducing noise to the finger pose measurements received by the actor, we were able to spin the ball both in sim and on the real robot hand.
In the end, we managed to complete most tasks, save for a few of the more complicated ones, like operating a handheld drill after picking it up. The highlight definitely was getting our hand to use a fidget spinner by pinching it between the thumb and index finger and spinning it with the middle finger.