Advanced manipulation: contact, grasping, and whole-body control

Why this subject matters now

For thirty years the manipulation stack was a tower of assumptions that held only in a laboratory: the object pose is known, the gripper is a parallel jaw, the grasp is a top-down pinch, and the controller runs in position mode against a rigid world. That tower is being dismantled from two directions at once. From below, contact-rich simulation matured: differentiable and contact-accurate physics engines such as MuJoCo and Drake now let a designer optimize through thousands of contacts per second, and GPU-parallel rollouts made it feasible to train policies that actually reason about the friction cone rather than avoiding it. From above, learning arrived: in-hand reorientation that no analytic controller had cracked was demonstrated end-to-end with reinforcement learning and massive domain randomization, and vision-language-action models began to fold grasp selection into a single network.

None of that makes the classical theory obsolete; it makes it load-bearing. A learned policy that reorients a cube is still constrained by Coulomb friction, and if the reward or the simulator gets the friction cone wrong the policy will not transfer. A whole-body controller for a humanoid is a quadratic program whose constraints are exactly the contact-complementarity and friction-cone conditions derived here. A practitioner today is expected to know why contact is nonsmooth, how to test a grasp for force closure and rank it by a quality metric, why you cannot command force and position along the same axis, and how to pose a whole-body task hierarchy as a QP. Those are the durable ideas, and they are the subject of this page. The line of work runs from Nguyen's construction of stable grasps and the Ferrari and Canny quality metric through the operational-space and whole-body control program of Khatib and Sentis to the hierarchical QP solvers of Escande, Mansard and colleagues, and it is being extended, not replaced, by the learning community.

Contact and the friction cone

Everything hard about manipulation descends from one fact: a contact can push but not pull, and it can only exert tangential force up to a limit set by friction. Consider a single point contact between a fingertip and an object, with outward surface normal \( \hat{n} \) (pointing out of the object, into the finger) and the two orthogonal tangent directions \( \hat{t}_1, \hat{t}_2 \) spanning the contact tangent plane. Decompose the contact force the finger applies to the object as \( f = f_n \hat{n}' + f_t \), where \( \hat{n}' = -\hat{n} \) points into the object, \( f_n \ge 0 \) is the normal (compressive) magnitude, and \( f_t \) is the tangential part. The Coulomb model states that the contact can be maintained without slipping if and only if

where \( \mu \) is the coefficient of static friction. The set of admissible forces is therefore a cone: the friction cone

Its half-angle at the apex is \( \alpha = \arctan\mu \). For \( \mu = 0.5 \) that is \( \arctan 0.5 = 26.57^\circ \); for \( \mu = 0.3 \) it is \( 16.70^\circ \). The cone is circular in three dimensions (a right circular cone about \( \hat{n}' \)) and reduces to a planar wedge of half-angle \( \alpha \) on either side of the normal in two dimensions. The unilateral condition \( f_n \ge 0 \) is the source of every difficulty: it makes the admissible set a cone rather than a subspace, and it makes the contact make-or-break rather than a smooth spring.

The polyhedral approximation

The circular friction cone is not a polytope, and optimization over it is a second-order-cone program. Much of grasping and whole-body control instead approximates the cone by an inscribed \( m \)-sided pyramid whose generating edges are unit vectors

so that any force in the pyramid is a nonnegative combination \( f = \sum_k \lambda_k \hat{e}_k \), \( \lambda_k \ge 0 \). This turns cone membership into a linear inequality and lets contact enter a linear program or quadratic program. The price is conservatism: the inscribed pyramid lies strictly inside the true cone, so a force the real contact could sustain may be rejected. The fractional loss is set by how far the pyramid's flat faces dip inside the circular rim. An \( m \)-gon inscribed in a unit circle has apothem \( \cos(\pi/m) \), so the pyramid captures only a \( \cos(\pi/m) \) fraction of the cone's tangential reach along its worst direction: \( 0.707 \) for a 4-sided pyramid, \( 0.866 \) for 6 sides, \( 0.924 \) for 8 sides. Circumscribing instead of inscribing flips the bias to optimistic. In practice 4 to 8 sides per contact is standard, and the planar case needs only two edges, \( \hat{n}' \pm \mu \hat{t} \), which is what the worked grasp problems below use.

Rigid contact as a complementarity problem

Coulomb friction is a static, admissible-set statement. To simulate or plan through contact one needs the dynamics: the contact force and the relative motion at the contact are coupled by conditions that are inherently combinatorial. Let \( \phi(q) \ge 0 \) be the signed gap between two bodies (positive when separated, zero at touch). A contact can only push when the bodies touch, so the normal impulse \( f_n \) and the gap satisfy

This is a complementarity condition, written \( 0 \le \phi \perp f_n \ge 0 \): at least one of the pair is zero at every instant. Either the bodies are separated and the force is zero, or the force is nonzero and the gap is closed. There is no smooth function that enforces this; the graph of admissible \( (\phi, f_n) \) is the L-shaped union of the two positive axes, which has a corner at the origin. Friction adds a second complementarity: along the tangent, either the contact sticks (relative tangential velocity zero, tangential force strictly inside the cone) or it slips (tangential velocity nonzero, tangential force on the cone boundary opposing the slip). Discretizing the equations of motion in time and stacking the normal and (polyhedrally approximated) friction conditions gives, at each step, a linear complementarity problem (LCP): find \( z \ge 0 \) with \( w = Mz + q \ge 0 \) and \( z^{\!\top} w = 0 \). Stewart and Trinkle in 1996 and Anitescu and Potra in 1997 put this time-stepping formulation on a firm footing, and it is the backbone of contact simulators to this day.

The consequence for control and learning is that the map from robot state and action to contact force is nonsmooth: it has kinks where contacts make and break and where a contact transitions between sticking and slipping. Gradients through such a map are either zero (away from events) or undefined (at events), which is why naive gradient-based trajectory optimization stalls on contact-rich tasks and why smoothed or randomized-smoothing surrogates, and sampling-based or learning-based methods, became the practical route. MuJoCo sidesteps the strict LCP by solving a convex relaxation with a soft, invertible contact model, trading a little physical fidelity for a differentiable, always-solvable step; Drake keeps a more rigid model and solves the complementarity problem more faithfully. The reader should carry away that "contact is nonsmooth" is not a numerical nuisance but a structural fact rooted in the unilateral, complementary nature of a push-only constraint.

Grasp analysis: the grasp map and wrenches

A grasp is a set of contacts that together immobilize an object, or hold it firmly enough to move it where the hand goes. The right object of study is not force but wrench: the pair \( F = (f, \tau) \) of a force and the torque it produces about a chosen reference, which lives in the six-dimensional wrench space \( \R^6 \) in three dimensions and in \( \R^3 \) (two force components and one scalar torque) in the plane. A force \( f \) applied at a point \( p \) (measured from the reference) produces the wrench

The cross product \( p \times f \) is the moment arm; in the plane it collapses to the scalar \( p_x f_y - p_y f_x \). This single formula does all the geometric work in what follows: it maps a contact force, applied at a known point, into the wrench it exerts on the object.

Contact types and the grasp map \( G \)

Not every contact transmits every kind of load. Three idealizations cover most hands. A frictionless point contact transmits only a normal force: one generator, \( \hat{n}' \). A point contact with friction (PCWF), the standard hard-fingertip model, transmits a force anywhere in the friction cone but no moment: three force components in 3D (two after the cone is approximated by generators), two in the plane. A soft-finger contact adds a moment about the contact normal, modeling the torsional friction of a deformable fingertip pressed into the surface; it transmits the friction-cone force plus a bounded spin torque. Each contact \( i \) contributes a set of primitive generators, and the map from the stacked contact-force coordinates \( f_c = (f_{c,1}, \dots, f_{c,k}) \) to the total object wrench is linear:

where the columns of the block \( G_i \) are the object-frame wrenches of the primitive forces at contact \( i \). Following Murray, Li and Sastry, \( G \) is the grasp map: it takes the forces the fingers can command, in their own contact frames, and returns the net wrench on the object. For a planar PCWF grasp with contact \( i \) at point \( p_i \), normal \( \hat{n}_i \) and tangent \( \hat{t}_i \), the block is the two-column matrix

The contact forces are not free: each \( f_{c,i} \) must lie in its friction cone \( \mathcal{FC}_i \). The grasp map plus the cone constraints is the complete static description of what the grasp can do. The companion page on robot kinematics and control derives the manipulator Jacobian \( J \) that maps joint torques to the fingertip wrenches \( f_c \); composing them, \( \tau = J^{\!\top} f_c \) with \( F_{\text{obj}} = G f_c \), is the full path from joint torque to object wrench, and the hand Jacobian and grasp map are the two halves of that chain.

Force closure and form closure

A grasp achieves force closure if the fingers can resist any external wrench on the object by choosing contact forces inside their friction cones. Formally, for every disturbance wrench \( F_{\text{ext}} \in \R^d \) (\( d = 3 \) planar, \( 6 \) spatial) there must exist contact forces \( f_c \in \mathcal{FC}_1 \times \cdots \times \mathcal{FC}_k \) with \( G f_c = -F_{\text{ext}} \). Since scaling a friction-cone force by a positive constant keeps it in the cone, the reachable set of object wrenches \( \{ G f_c : f_c \in \prod_i \mathcal{FC}_i \} \) is a cone. Force closure demands that this cone be all of \( \R^d \). Equivalently, replacing each circular friction cone by its polyhedral generators \( \{ \hat{e}_{i,j} \} \) and letting \( w_{i,j} = G_i \hat{e}_{i,j} \) be the corresponding primitive contact wrenches, force closure holds if and only if the \( w_{i,j} \) positively span the wrench space:

There is a clean geometric restatement. Positively spanning \( \R^d \) is equivalent to the origin lying in the interior of the convex hull of the primitive wrenches (after any positive normalization). The forward direction: if \( 0 \) is interior to \( \text{conv}\{w_{i,j}\} \), then for any direction \( u \) a small step \( -\epsilon u \) is still in the hull, so \( -u \) is a nonnegative combination of the \( w_{i,j} \) after clearing the convex-combination denominator, which gives a positive spanning of \( -u \), hence of every direction. The reverse direction is the contrapositive of the separating-hyperplane theorem: if \( 0 \) is not interior to the hull, a hyperplane through the origin has all \( w_{i,j} \) weakly on one side, so no positive combination can reach the wrenches strictly on the other side, and closure fails. This is the operational test: compute the primitive contact wrenches and check whether the origin is strictly inside their convex hull.

Form closure is the stronger, friction-free cousin. A grasp is form-closed if the contacts immobilize the object using normal forces alone, with no reliance on friction at all: the normal wrenches \( G_i \hat{n}_i \) already positively span \( \R^d \). Form closure implies force closure but not conversely, and a classical counting result (Reuleaux; refined by many since) shows that form-closing a rigid body in the plane needs at least four contacts and in space at least seven, whereas force closure of a planar object needs only two frictional contacts and of a spatial object only three. That gap, four versus two in the plane, is exactly why robots rely on friction: friction buys closure with fewer fingers. Nguyen in 1988 turned the geometric picture into a constructive algorithm for synthesizing force-closed (and independently stable) grasps directly from the object edges.

Grasp quality: the ball metric and the minimum singular value

Force closure is a yes-or-no property; engineering needs a scalar that says how good a grasp is, so that a planner can rank candidates. Two families of metric dominate, both due in their modern form to Ferrari and Canny in 1992. The largest-minimum-resisted-wrench or ball metric asks: what is the radius of the largest ball of external wrenches, centered at the origin, that the grasp can resist with unit total contact force? Geometrically, after bounding the total contact effort (an \( L_1 \) or \( L_\infty \) cap on the \( \lambda_{i,j} \)), the set of resistible object wrenches is the convex hull of the primitive wrenches, and the largest centered ball inside it has radius equal to the distance from the origin to the nearest facet of that hull:

where each facet is written \( \{ x : a_k^{\!\top} x + b_k = 0 \} \) with unit normal \( a_k \) and the interior satisfying \( a_k^{\!\top} x + b_k \le 0 \). A larger \( Q_{\text{ball}} \) means the grasp resists a bigger worst-case disturbance for the same effort; \( Q_{\text{ball}} > 0 \) is exactly force closure. This metric is frame-dependent because it mixes force and torque units, and choosing the torque reference (usually the object center of mass) and a length scale to make them commensurate is part of using it honestly.

The second family reads the grasp map \( G \) directly through its singular value decomposition \( G = U \Sigma V^{\!\top} \). The smallest singular value \( \sigma_{\min}(G) \) measures the grasp's weakest direction: it is the smallest object wrench that unit contact forces can produce along some axis, so \( \sigma_{\min}(G) \) large means no direction is starved. Zero \( \sigma_{\min} \) means \( G \) is rank-deficient and the grasp cannot resist wrenches in the lost direction at all. The ratio \( \sigma_{\min}(G)/\sigma_{\max}(G) \) is an isotropy index: \( 1 \) is a perfectly uniform grasp that is equally strong in every direction. These two metrics answer different questions. \( \sigma_{\min}(G) \) ignores the friction cones and grades only the linear map; \( Q_{\text{ball}} \) folds in the cones and the effort budget. A well-engineered grasp planner such as those built on Dex-Net uses a convex-hull quality like \( Q_{\text{ball}} \) (often the Ferrari-Canny metric) rather than \( \sigma_{\min} \) alone.

Antipodal grasps

The simplest force-closed grasp, and the one two-jaw grippers are built to exploit, is the antipodal grasp: two contacts whose line of centers lies within both friction cones. If the contact points are \( p_1, p_2 \) with inward surface normals \( \hat{n}'_1, \hat{n}'_2 \), the grasp is antipodal when the segment direction \( \hat{d} = (p_2 - p_1)/\lVert p_2 - p_1 \rVert \) makes an angle at most \( \arctan\mu \) with \( \hat{n}'_1 \) and \( -\hat{d} \) makes an angle at most \( \arctan\mu \) with \( \hat{n}'_2 \). The reason this gives force closure is exactly Problem 1: when the closing line is inside both cones, each finger can push along that line and also apply tangential friction, and the resulting primitive wrenches surround the origin. Antipodal detection reduces to a geometric test on surface normals, which is why it is the workhorse of learned grasp detectors: a network can be trained to output an antipodal contact pair and gripper angle from a depth image, as in the GraspNet and Contact-GraspNet lines of work, without ever forming the grasp map explicitly. The theory tells the network what a good label looks like.

The wrench space and a note on screw theory

Wrenches and twists are dual six-vectors, and the language that unifies them is screw theory. A twist \( V = (v, \omega) \) packs a linear and an angular velocity; a wrench \( F = (f, \tau) \) packs a force and a torque. Chasles' theorem says any rigid-body velocity is an instantaneous rotation about, plus translation along, a single screw axis, and Poinsot's dual says any system of forces reduces to a single force along a screw axis plus a torque about it. The pairing \( F^{\!\top} V = f^{\!\top} v + \tau^{\!\top} \omega \) is the instantaneous power the wrench does on the twist. Two screws are reciprocal when this pairing vanishes, \( F^{\!\top} V = 0 \): the wrench does no work on the motion. Reciprocity is the exact algebra of a rigid constraint. When a peg slides in a hole, the constraint wrench (the wall pushing) is reciprocal to the freedom twist (the peg sliding), because a frictionless constraint does no work. That single fact, developed next, is what forces the decomposition at the heart of hybrid force/position control. The companion kinematics page develops the Jacobian and the adjoint that transport twists and wrenches between frames; here we only need the pairing and reciprocity.

Hybrid force/position control

Once a robot is in contact it must, in general, control force in some directions and motion in others: press a tool against a surface with a set force while sliding it along the surface at a set speed; insert a peg while regulating the lateral contact forces to zero. The foundational insight, due to Mason in 1981 and turned into a controller by Raibert and Craig the same year, is that at any contact configuration the task space splits into two complementary subspaces, and you get to choose motion in one and force in the other, but never both in the same direction.

Why not both in one direction

Fix a task frame at the contact. The environment imposes natural constraints: along directions where the environment is rigid the robot cannot move, so the velocity is forced to zero there; along directions where the environment is free the robot cannot exert force, so the force is forced to zero there. These natural constraints are reciprocal, as screw theory requires: the natural-constraint velocities and the natural-constraint forces occupy complementary subspaces whose pairing is zero. The controller supplies the remaining, artificial constraints: it commands a force where the environment left velocity free, and commands a velocity where the environment left force free. Suppose one tried to command both a desired force \( f_d \neq 0 \) and a desired velocity \( v_d \neq 0 \) along the same task axis \( \hat{u} \). Two cases. If the environment is rigid along \( \hat{u} \), the natural constraint pins \( v = 0 \), so the commanded \( v_d \) is unachievable and the position loop winds up. If the environment is free along \( \hat{u} \), the natural constraint pins \( f = 0 \), so the commanded \( f_d \) is unachievable and the force loop winds up. Either way one command fights a physical constraint and integrates to saturation. The task is overdetermined along any axis where both are commanded; that is the formal content of "you cannot control force and position in the same direction."

The selection matrix

Encode the choice with a diagonal selection matrix \( S = \operatorname{diag}(s_1, \dots, s_d) \), where \( s_i = 1 \) marks a motion-controlled axis and \( s_i = 0 \) a force-controlled axis (in a general frame \( S \) is a projection, \( S = S^{\!\top} \), \( S^2 = S \)). Its complement is \( \bar S = I - S \), and the two are orthogonal complementary projections:

The hybrid control law, in operational space, drives motion error through \( S \) and force error through \( \bar S \):

where \( J \) is the manipulator Jacobian and \( \tau_{\text{grav}} \) the gravity compensation, both from the kinematics and control page. Because \( S \bar S = 0 \), the motion command lives entirely in the motion subspace and the force command entirely in the force subspace; they never contend for the same axis. The elegance is that \( S \) makes the design a bookkeeping choice: pick, per axis, which of the two loops is closed, and the algebra guarantees they do not interfere. The subtlety Raibert and Craig navigated, and De Luca and others refined, is that \( S \) must be expressed in the instantaneous constraint frame, which moves with the task; a mismatch between the assumed and actual constraint directions leaks force commands into motion axes and vice versa, and that leakage is the practical failure mode of naive hybrid control.

Impedance, admittance, and contact stability

Hybrid control switches an axis between pure force and pure motion. Impedance control, introduced by Hogan in 1985, instead makes the manipulator behave like a programmable mechanical impedance: a mass-spring-damper relating motion to force, without ever switching modes. The kinematics and control page derives the operational-space impedance law; the essential relation renders the endpoint's behavior as

with \( \Lambda(x) = (J M^{-1} J^{\!\top})^{-1} \) the operational-space inertia, \( D \) a desired damping, and \( K \) a desired stiffness. The robot is told what apparent stiffness to present, and contact forces follow from that impedance rather than being commanded directly. Admittance control is the causal dual: it measures the contact force and integrates the impedance model to produce a commanded motion, which an inner high-gain position loop tracks. Impedance suits stiff robots with torque control; admittance suits geared, position-controlled robots with a force sensor at the wrist.

The reason both exist, and the reason contact is where controllers go unstable, is the contact-stability argument. Model the environment along one axis as a stiffness \( k_e \) (a wall of that compliance). Couple it to a robot rendering stiffness \( k \) and damping \( b \). The closed loop is stable only if the loop's effective damping stays positive once the unavoidable delays and the environment stiffness are included. Two facts fall out. First, the harder the environment (\( k_e \to \infty \)), the more the loop gain rises at contact, so a stiff wall destabilizes a controller that was fine in free space; this is why a robot can track beautifully in air and buzz the instant it touches a table. Second, there is a fundamental limit, made precise by Colgate and Hogan in 1988 through the notion of passivity: the range of environments a given controller can contact stably is bounded, and a sampled-data controller can render a passive (hence universally contact-stable) stiffness only up to \( k_{\max} \approx 2 b / T \), where \( b \) is the physical damping and \( T \) the control period. Faster control loops (smaller \( T \)) and more physical damping buy stiffer stable contact. This is the quantitative reason manipulation control runs at kilohertz rates and why torque-controlled, low-friction arms transformed contact-rich manipulation: they raise both \( b \) and the achievable loop rate.

Compliant assembly and the peg-in-hole problem

Peg-in-hole is the fruit fly of manipulation research: simple to state, mechanically rich, and a proxy for most assembly. A cylindrical peg of radius \( r_p \) must enter a hole of radius \( r_h > r_p \) with clearance \( c = r_h - r_p \), typically tens of micrometers. The difficulty is that the peg must be positioned to within the clearance and aligned to within an angular tolerance set by the peg's aspect ratio, and no real robot's absolute accuracy meets those tolerances. Two failure modes dominate. In two-point contact, the peg tips and jams: the reaction forces at the two contact points can lock the peg against insertion no matter how hard it is pushed, a wedging condition governed by the friction angle and the insertion depth. In one-point contact, the peg has entered slightly and needs to be steered to reduce the lateral force.

The classical mechanical solution is the remote center of compliance (RCC), a passive compliant wrist introduced at Draper Laboratory (Whitney and colleagues) in the 1970s. An RCC is a linkage whose compliance center, the point about which applied forces produce pure translation and applied moments produce pure rotation, is projected out to the peg tip. When the peg tip touches the chamfer of the hole, the contact force naturally translates the peg to center it, and a contact moment naturally rotates the peg to align it, with no sensing and no computation: the mechanics does the search. Whitney's 1982 analysis derives the insertion-force and jamming diagrams that show the RCC keeps the peg inside the wedge-free region throughout insertion. The modern, active analog uses force feedback: measure the lateral forces and moments, and command corrective motion through an admittance or hybrid law that regulates the lateral force to zero (exactly the selection matrix of Problem 3) while feeding the peg down. A spiral or Lissajous search pattern finds the hole entrance when the initial position error exceeds the chamfer, then the insertion phase regulates lateral force. The reason peg-in-hole endures as a benchmark is that it exercises the whole stack at once: contact modeling for the jamming analysis, force control for the insertion, compliance for stability, and, increasingly, learning for the search policy under uncertainty.

Whole-body control

A humanoid or a mobile manipulator has far more joints than any single task needs, and it must serve several tasks at once: keep balance, hold a contact, reach a target, avoid a joint limit. Whole-body control, the program of Khatib, Sentis and collaborators through the 2000s, organizes these into a hierarchy and solves for joint torques that respect it. Recall from the kinematics and control page that a task \( x_i = f_i(q) \) has Jacobian \( J_i = \partial f_i / \partial q \) and that the operational-space controller maps a desired task acceleration \( \ddot x_i^\star \) to joint torques. When several tasks compete, a strict priority is enforced by null-space projection: task 1 is served exactly, then task 2 is served only within the null space of task 1, then task 3 within the joint null space of tasks 1 and 2, and so on. Writing \( N_1 = I - J_1^{+} J_1 \) for the null-space projector of the highest task (with \( J_1^{+} \) the pseudo-inverse), the stacked solution is

so that each lower task only uses the freedoms the higher tasks left unused. This is the same null-space idea the kinematics page uses for a single secondary objective, stacked to arbitrary depth. Balance and contact sit at the top of the hierarchy, reaching below them, posture below that.

Quadratic-programming whole-body control

The projection cascade handles priorities but not inequalities: joint limits, torque limits, friction cones, and the unilateral contact condition are all inequality constraints, and a pseudo-inverse cannot express them. The modern formulation, from Escande, Mansard and Wieber (2014) and the controllers built on TSID and Pinocchio, poses whole-body control as a quadratic program solved every control tick. The decision variables are the joint accelerations \( \ddot q \), the actuation torques \( \tau \), and the contact forces \( \lambda \). The QP minimizes a weighted sum of task-acceleration errors subject to the rigid-body dynamics and the physical inequalities:

The equality is the manipulator dynamics with \( M \) the mass matrix, \( h \) the nonlinear (Coriolis plus gravity) term, \( S_a \) the actuation selection (underactuation for a floating base), and \( J_c^{\!\top}\lambda \) the contact-force contribution; the last constraint holds the contact fixed (zero contact acceleration). The friction-cone membership \( \lambda \in \mathcal{FC} \) is imposed with the polyhedral approximation so it becomes a linear inequality and keeps the whole thing a QP. Strict priorities, when needed, are recovered by solving a cascade of QPs, each in the null space of the previous, which is the hierarchical-QP of Escande and colleagues; soft priorities use the task weights \( w_i \). This QP is what runs at 1 kHz on modern humanoids and quadrupeds, and it is the point where the contact theory of the first half of this page becomes a live constraint in a controller. The learned-policy alternative, covered on the embodied foundation models page, replaces the hand-built cost and constraints with a network trained in simulation, but even those policies are frequently deployed on top of a QP that enforces the hard safety constraints.

Task and motion planning

A single grasp and insertion is one action; a real task ("set the table", "assemble the gearbox") is a sequence, and choosing the sequence is symbolic while executing each step is geometric. Task and motion planning (TAMP), the program named by Kaelbling and Lozano-Perez around 2011, integrates the two. The symbolic layer plans over discrete actions (pick A, place A on B, pick C) using logic or PDDL-style operators with preconditions and effects; the geometric layer must find, for each symbolic action, continuous parameters, a grasp pose, a collision-free arm trajectory, a stable placement, that make the action physically realizable. The hardness is the coupling: whether a symbolic plan is feasible depends on continuous choices made several steps earlier. Grasping object A one way may make a later placement of A infeasible because the arm cannot reach; grasping it the other way may collide during pickup. The planner cannot commit to the symbolic plan without checking geometry, and cannot check geometry without a symbolic plan.

The core difficulty is a sampling problem on a space that is part discrete and part continuous with measure-zero constraints. Feasible grasps, stable placements, and valid regrasps live on lower-dimensional manifolds (a grasp must contact the surface; a placement must rest stably), so uniform sampling almost never lands on them. Effective TAMP either samples from these manifolds directly (grasp databases, placement surfaces) or interleaves symbolic search with a motion-feasibility oracle that lazily reports which symbolic transitions are geometrically possible, backtracking when a committed continuous choice dead-ends. Systems in the mold of Garrett, Lozano- Perez and Kaelbling's PDDLStream, and planners built on Drake and MoveIt, formalize this as generating continuous values through streams that the symbolic search calls on demand. The frontier folds learning into the sampling, training networks to propose promising grasps, placements, and subgoals so the continuous search explores the right manifolds first.

Dexterous in-hand manipulation

Everything above assumes the grasp is fixed once acquired. In-hand manipulation deliberately changes the grasp: reorient an object within the hand, walk the fingers to a new configuration, roll the object across the fingertips. The mechanics are governed by the same contact kinematics. Finger gaiting relocates one finger at a time while the others maintain force closure, a discrete search over which fingers hold and which move that must keep the held subset force-closed at every step. Rolling contact exploits the contact kinematic equations (Montana, 1988) that relate finger motion to the motion of the contact point across curved surfaces, letting the object rotate as fingertips roll without slipping. Sliding and pivoting use controlled slip and gravity. A dexterous hand with, say, four fingers and sixteen or more joints is a high-dimensional, contact-rich, underactuated-at-the-object system, and analytic control of it hits a wall: the number of contact modes (each finger sticking, slipping, or free) explodes, and the plan must sequence through them.

This is the task where hand-designed control most clearly gave way to learning. The Dactyl result (OpenAI and colleagues, 2018-2019) trained a policy in simulation with extensive domain randomization to reorient a cube, and later a Rubik's-cube face, in a five-fingered hand from vision and joint sensing alone, a behavior no analytic controller had produced. The lesson was not that the contact theory was wrong but that it was too combinatorial to plan through online, whereas a policy could amortize the reasoning offline across billions of simulated contacts, provided the simulator's friction and the randomization spanned reality. Subsequent work (for example the in-hand reorientation lines at Columbia, MIT, and NVIDIA, and rapid-motor-adaptation approaches) pushed to arbitrary object shapes, touch-only sensing, and real-time reorientation. The embodied foundation models page treats these learned policies in depth; the connection to this page is that the reward, the simulator, and the success test are all written in the contact and grasp language derived here, and a policy trained against a wrong friction cone will not transfer.

Tactile sensing

Vision degrades exactly where manipulation needs information most: at contact, the fingers and object occlude the contact patch, and the sub-millimeter geometry and the incipient slip that decide a grasp are invisible to a camera. Tactile sensing fills that gap. The optical-tactile family, of which GelSight (Adelson's group at MIT; Yuan, Dong and colleagues, 2017) is the archetype, presses a soft, gel-coated elastomer against the object and images the gel's deformation from inside with a camera and grazing colored illumination. Photometric stereo reconstructs the contact surface normal at every pixel, so a single tactile frame recovers a dense height map of the contact patch, including fine texture and the exact contact boundary, at a spatial resolution finer than the object's own machining marks. Painted markers on the gel additionally track the tangential shear field, which reveals incipient slip before the object actually moves: the markers at the edge of the contact patch start to slide while the center still sticks, and that partial-slip signature is a direct, early warning that the grasp force is insufficient.

Tactile servoing closes a control loop on this signal: regulate the contact so that a desired tactile image is maintained, keep the contact centered, hold a target contact force, or arrest incipient slip by increasing grip force, exactly the impedance and force-control laws of this page but with the feedback coming from the skin rather than a wrist sensor. Because a real tactile sensor is expensive to run in the training loop, differentiable tactile simulators such as TACTO render optical-tactile images in simulation so that policies can be trained on touch, echoing how MuJoCo and Drake let policies train on contact dynamics. The broader point is that near contact the state a controller needs is a contact state, normal, tangential shear, slip, and that state is what tactile sensing measures directly and vision only infers.

Implementation

The grasp analysis is small linear algebra and a convex-hull test, entirely runnable in NumPy. The code below computes the primitive contact wrenches for a planar grasp, tests force closure by checking whether the origin is strictly interior to their convex hull, computes the ball-quality metric as the distance from the origin to the nearest hull facet, and forms the grasp map to read off \( \sigma_{\min} \). It reproduces every number in Problems 1 and 2 (\( Q = 0.27217 \) for two fingers, \( 0.40825 \) for three; \( \sigma_{\min} = \sqrt 2 \) and \( \sqrt 3 \)). The SVD is sanity-checked by reconstruction, per the numerical-hygiene note in the authoring spec.

import numpy as np
from scipy.spatial import ConvexHull

def wrench2d(p, f):
    # planar wrench (fx, fy, tau) of force f applied at point p
    return np.array([f[0], f[1], p[0] * f[1] - p[1] * f[0]])

def primitive_wrenches(contacts, mu):
    # contacts: list of (point p, inward-normal n, tangent t)
    # planar friction cone => two edges per contact: n +/- mu*t
    W = []
    for p, n, t in contacts:
        for e in (n + mu * t, n - mu * t):
            W.append(wrench2d(np.asarray(p), np.asarray(e)))
    return np.array(W)                                  # shape (2*k, 3)

def force_closure(W):
    # force closure  <=>  origin strictly interior to conv hull of wrenches
    Wn = W / np.linalg.norm(W, axis=1, keepdims=True)   # normalize generators
    hull = ConvexHull(Wn)
    # hull.equations rows are [a | b] with a.x + b <= 0 inside; at origin value = b
    b = hull.equations[:, -1]
    closed = bool(b.max() < -1e-9)                      # all facets strictly beyond 0
    Q_ball = float(-b.max())                            # dist(origin, nearest facet)
    return closed, Q_ball

def grasp_map(contacts):
    # columns: object-frame wrench of each unit normal and unit tangent force
    cols = []
    for p, n, t in contacts:
        cols.append(wrench2d(np.asarray(p), np.asarray(n)))
        cols.append(wrench2d(np.asarray(p), np.asarray(t)))
    return np.array(cols).T                             # shape (3, 2*k)

# --- two-finger antipodal grasp on the unit disk, mu = 0.5 ---
two = [((1.0, 0.0), (-1.0, 0.0), (0.0, 1.0)),
       ((-1.0, 0.0), (1.0, 0.0), (0.0, 1.0))]
W2 = primitive_wrenches(two, mu=0.5)
print("closed, Q =", force_closure(W2))                # True, 0.27217
G2 = grasp_map(two)
U, S, Vt = np.linalg.svd(G2)
assert np.linalg.norm(U[:, :len(S)] @ np.diag(S) @ Vt[:len(S)] - G2) < 1e-9  # SVD sane
print("sigma =", S, " sigma_min =", S.min())           # all sqrt(2) = 1.41421

# --- three-finger equilateral grasp, mu = 0.5 ---
ang = np.deg2rad([90, 210, 330])
three = [((np.cos(a), np.sin(a)),
          (-np.cos(a), -np.sin(a)),
          (-np.sin(a), np.cos(a))) for a in ang]
print("closed, Q =", force_closure(primitive_wrenches(three, mu=0.5)))  # True, 0.40825
print("sigma_min =", np.linalg.svd(grasp_map(three), compute_uv=False).min())  # sqrt(3)

# --- frictionless two contacts: normals only, rank-deficient, no closure ---
W0 = np.array([wrench2d(np.array(p), np.array(n)) for p, n, _ in two])
print("frictionless rank =", np.linalg.matrix_rank(W0))  # 1  -> cannot span R^3

The whole-body controller is a quadratic program solved each tick. The sketch below sets up the QP of the whole-body section with decision variables \( (\ddot q, \tau, \lambda) \), the floating-base dynamics as an equality, torque bounds and a polyhedral friction cone as inequalities, and a weighted stack of task-acceleration costs, then hands it to a generic QP solver. It is deliberately a skeleton: a production controller (TSID, Drake) uses efficient rigid-body-dynamics libraries for \( M, h, J \), but the constraint structure is exactly this.

import numpy as np

# Whole-body inverse dynamics as one QP over x = [qdd (n), tau (n), lam (m)].
# min  sum_i w_i || J_i qdd + Jdot_i qd - xdd_star_i ||^2
# s.t. M qdd + h = Sa^T tau + Jc^T lam        (floating-base dynamics)
#      tau_lo <= tau <= tau_hi                 (actuation limits)
#      Ffric lam <= 0,  lam_normal >= 0        (polyhedral friction cone)
#      Jc qdd + Jdotc qd = 0                    (contacts do not accelerate)

def build_wbc_qp(M, h, Sa, Jc, Jdotc_qd, tasks, tau_lim, Ffric, qd):
    n = M.shape[0]                     # generalized coords
    m = Jc.shape[0]                    # contact-force dimension
    nx = 2 * n + m
    # ---- quadratic cost 0.5 x^T H x + g^T x from the task stack ----
    H = np.zeros((nx, nx)); g = np.zeros(nx)
    for J_i, Jd_i_qd, xdd_star, w in tasks:
        A = np.zeros((J_i.shape[0], nx)); A[:, :n] = J_i        # acts on qdd block
        b = xdd_star - Jd_i_qd
        H += w * A.T @ A; g += -w * A.T @ b
    H += 1e-8 * np.eye(nx)             # regularize
    # ---- equality: dynamics + contact-acceleration ----
    Aeq = np.zeros((n + m, nx)); beq = np.zeros(n + m)
    Aeq[:n, :n] = M; Aeq[:n, n:2 * n] = -Sa.T; Aeq[:n, 2 * n:] = -Jc.T
    beq[:n] = -h
    Aeq[n:, :n] = Jc; beq[n:] = -Jdotc_qd
    # ---- inequality: torque bounds + friction cone ----
    Gs = []; hs = []
    T = np.zeros((n, nx)); T[:, n:2 * n] = np.eye(n)
    Gs += [T, -T]; hs += [tau_lim, tau_lim]                     # |tau| <= tau_lim
    Fc = np.zeros((Ffric.shape[0], nx)); Fc[:, 2 * n:] = Ffric
    Gs += [Fc]; hs += [np.zeros(Ffric.shape[0])]               # Ffric lam <= 0
    Gineq = np.vstack(Gs); hineq = np.concatenate(hs)
    return H, g, Aeq, beq, Gineq, hineq

# Solve with any QP backend (quadprog / OSQP / qpSWIFT). The point is the
# CONSTRAINTS: friction cone Ffric and the dynamics equality couple contact
# forces, torques, and accelerations into one convex problem solved at 1 kHz.

Where a learned or differentiable component fits, a grasp-quality network scores candidate grasps from geometry, the role Dex-Net plays. The differentiable core is small: encode a grasp candidate, predict a scalar quality, and train the predictor against the analytic ball metric computed above so that the network amortizes the convex-hull test. The PyTorch and JAX versions below share the architecture; the loss regresses the network's score onto the analytic \( Q_{\text{ball}} \) label, which is how a data-driven grasp planner is bootstrapped from the classical metric before being fine-tuned on real grasp outcomes.

import torch
import torch.nn as nn

class GraspQualityNet(nn.Module):
    # maps a grasp descriptor (contact points, normals, mu) -> scalar quality
    def __init__(self, d_in=16, d_hidden=128):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(d_in, d_hidden), nn.GELU(),
            nn.Linear(d_hidden, d_hidden), nn.GELU(),
            nn.Linear(d_hidden, 1),
        )

    def forward(self, x):            # x: (B, d_in)
        return self.net(x).squeeze(-1)   # (B,)  predicted Q_ball

def train_step(model, opt, x, q_analytic):
    # q_analytic: (B,) ground-truth ball metric from the convex-hull test
    pred = model(x)                          # (B,)
    loss = torch.nn.functional.smooth_l1_loss(pred, q_analytic)
    opt.zero_grad(); loss.backward(); opt.step()
    return loss.item()

model = GraspQualityNet()
opt = torch.optim.AdamW(model.parameters(), lr=3e-4)
x = torch.randn(256, 16); q = torch.rand(256)   # stand-in batch
print(train_step(model, opt, x, q))

import jax, jax.numpy as jnp
import flax.linen as fnn
import optax

class GraspQualityNet(fnn.Module):
    d_hidden: int = 128
    @fnn.compact
    def __call__(self, x):           # x: (B, d_in)
        x = fnn.gelu(fnn.Dense(self.d_hidden)(x))
        x = fnn.gelu(fnn.Dense(self.d_hidden)(x))
        return fnn.Dense(1)(x).squeeze(-1)   # (B,) predicted Q_ball

def loss_fn(params, apply, x, q_analytic):
    pred = apply(params, x)                   # (B,)
    return optax.huber_loss(pred, q_analytic).mean()

model = GraspQualityNet()
key = jax.random.PRNGKey(0)
x = jax.random.normal(key, (256, 16)); q = jax.random.uniform(key, (256,))
params = model.init(key, x)
tx = optax.adamw(3e-4); state = tx.init(params)

@jax.jit
def train_step(params, state, x, q):
    loss, grads = jax.value_and_grad(loss_fn)(params, model.apply, x, q)
    updates, state = tx.update(grads, state, params)
    return optax.apply_updates(params, updates), state, loss

params, state, loss = train_step(params, state, x, q)
print(float(loss))

How it is done in practice

The gap between the derivations and a shipped system is mostly about the assumptions that the theory takes for granted. Force closure assumes the contact points, normals, and friction coefficient are known; in the field, object pose comes from a noisy perception stack, the friction coefficient is a guess that drifts with surface contamination, and the "point" contact is a deformable patch. Robust grasp planners therefore optimize a quality metric under uncertainty, integrating the ball metric over a distribution of object poses and friction values rather than trusting a single estimate; this is the probabilistic reframing that Dex-Net's grasp-robustness metric embodies, and it is why a grasp with a slightly lower nominal quality but a flatter sensitivity is often preferred.

Contact simulators are the second practical pillar. Training a whole-body or in-hand policy requires billions of contact resolutions, and the fidelity-versus-speed tradeoff decides whether the trained policy transfers. MuJoCo's soft, convex contact model is fast and always solvable, which is why it dominates policy learning, at the cost of a slightly springy contact; Drake's more rigid complementarity model is slower but closer to the physics, favored for verification and for contact trajectory optimization. The sim-to-real gap is largely a contact-model gap, and the standard mitigation, domain randomization over friction, restitution, mass, and latency, is an admission that the contact parameters cannot be measured precisely enough to trust a single simulator.

Control-rate and hardware realities set the third boundary. The contact-stability limit \( k_{\max} \approx 2b/T \) is not academic: it is why torque-controlled arms with low-friction transmissions and 1 kHz loops (the Franka, the Kuka iiwa, quadruped and humanoid actuators) displaced stiff position-controlled arms for contact work, and why the whole-body QP must solve in well under a millisecond. Efficient rigid-body-dynamics libraries such as Pinocchio compute \( M, h \) and the Jacobians in microseconds so that the QP solver gets its full time budget, and warm-started sparse QP solvers exploit the fact that consecutive control ticks pose nearly identical problems.

The current research frontier

Three threads are active and partly in tension. The first is learned manipulation policies, where vision-language-action models (the RT and OpenVLA lines from Google DeepMind and academic consortia, and diffusion-policy approaches from Columbia, MIT, Toyota Research, and UC San Diego) fold perception, grasp selection, and control into one network trained on large teleoperation or simulation datasets. These excel at the perception and sequencing that the analytic stack handles poorly, and they lean on the classical theory mainly through the simulator and the reward. The embodied foundation models page covers this thread in depth.

The learned thread has moved quickly since 2023. Diffusion Policy made multimodal visuomotor imitation practical, and 3D Diffuser Actor from CMU conditions the denoiser on a 3D scene representation rather than 2D images alone, which pays off on precise tabletop benchmarks. Grasp perception has its own scaled model in AnyGrasp from Shanghai Jiao Tong, which detects stable grasps on cluttered unseen objects at real-time rates from depth input. For bimanual dexterity the Tsinghua RDT line pretrained a 1.2B-parameter diffusion foundation model, RDT-1B (2024), and followed it with RDT-2 (2025), an autoregressive vision-language-action model on a Qwen2.5-VL backbone that predicts 24-step relative action chunks through a residual vector-quantized action tokenizer, trains on large-scale data from the UMI handheld gripper, and aims at zero-shot transfer to arms it never trained on. On the reinforcement side, Berkeley's HIL-SERL showed that human-in-the-loop RL on real hardware reaches near-perfect success on contact-rich insertion and assembly tasks within a few hours of training, precisely the regime where the analytic contact models above are hardest to trust.

The second is differentiable and contact-aware optimization. Contact-implicit trajectory optimization (a line running through Posa, Cantu and Tedrake at MIT, and Manchester, Toussaint and others) plans through the complementarity constraints directly, so that a single optimizer discovers both the contact sequence and the motion, rather than fixing the contacts in advance. Randomized smoothing of the nonsmooth contact dynamics (Pang, Suh, Tedrake) makes gradient-based planning tractable through contact by replacing the kinked dynamics with a smoothed surrogate whose gradient is informative, connecting the classical LCP view to modern optimization. GPU-accelerated motion generation such as cuRobo (NVIDIA) pushes collision- free motion and grasp planning to real-time by parallelizing the search across thousands of seeds.

The third is touch-centric manipulation. High-resolution optical-tactile sensors (GelSight and its descendants at MIT, DIGIT at Meta, and the vision-based tactile efforts at CMU, Berkeley, and ETH Zurich) are moving from perception demonstrations into closed-loop control, and differentiable tactile simulators (TACTO, and the Taxim and TacchNet lines) let policies train on touch in simulation. The open question that unites all three threads is generalization: analytic grasp theory generalizes across objects but not across uncertainty, learned policies generalize across uncertainty but not always across object categories, and the field is converging on hybrids that use the analytic contact and grasp structure as an inductive bias, or as a hard safety layer, around a learned core.

Open source to read

• google-deepmind/mujoco and the mujoco_menagerie model zoo. The reference contact-rich simulator for policy learning. Read src/engine/engine_core_constraint.c to see the convex, soft-contact solver, and the menagerie's hand and arm models to see how friction and contact are specified in practice.
• stack-of-tasks/pinocchio and tsid. Pinocchio is the fast rigid-body- dynamics library (\( M, h \), Jacobians, derivatives); TSID builds the whole-body QP on top of it. Open TSID's formulations/inverse-dynamics-formulation-acc-force.hpp to see the exact task/constraint stack of the whole-body QP.
• RobotLocomotion/drake. The rigorous multibody and contact stack, with faithful complementarity contact and contact-implicit trajectory optimization. Start from the multibody/plant directory and the hydroelastic contact documentation.
• ros-planning/moveit. The mainstream motion- planning framework: sampling-based planners, collision checking, and grasp/pick-place pipelines. Read the pick-place capability to see how grasps become planned trajectories.
• NVlabs/curobo. GPU-parallel collision-free motion generation and grasp planning in real time. The motion-gen example shows the parallel seed-and-optimize structure.
• BerkeleyAutomation/dex-net. Grasp planning with robustness metrics; the code that computes and learns grasp-quality scores. Read the grasp-quality metric module to see the Ferrari-Canny and robust-wrench-resistance implementations next to the theory here.
• facebookresearch/tacto. A fast differentiable simulator for optical-tactile (GelSight/DIGIT-style) sensors, for training touch-based policies. The renderer and the example environments show how a tactile image is produced from contact geometry.

Common misconceptions

"More contact points always means a better grasp." Not necessarily. Adding a redundant contact that duplicates a direction already well covered raises cost and collision risk without improving the ball metric or \( \sigma_{\min} \). What matters is that the primitive wrenches surround the origin uniformly; a well-placed two-finger antipodal grasp can beat a clumsy four-finger one, as the metrics in Problem 2 make quantitative.

"Force closure and form closure are the same thing." They are not. Form closure immobilizes with normal forces alone and needs at least four contacts in the plane; force closure uses friction and needs only two. A frictional antipodal grasp is force-closed but not form-closed; remove friction and it holds nothing.

"You can control both force and position along a contact direction if your gains are high enough." No. The obstruction is not gain but the natural constraint the environment imposes: along any axis the environment fixes either the force (free space) or the motion (rigid contact), so commanding the other is fighting a physical equality. The selection matrix formalizes that exactly one of the two is yours to command per axis.

"A stiffer position controller gives better contact behavior." The opposite is usually true. Higher rendered stiffness against a stiff environment pushes the closed loop toward instability; the passivity bound \( k_{\max} \approx 2b/T \) caps the stiffness a sampled controller can render stably against arbitrary environments. Contact-rich work wants compliance and fast loops, not brute stiffness.

"Contact is smooth if you use a good enough solver." Contact nonsmoothness is structural, not numerical: it comes from the complementarity of a push-only constraint, which has a corner at make/break and at stick/slip. Solvers can smooth it (MuJoCo does), but that trades physical fidelity for differentiability; it does not make the underlying dynamics smooth.

"The grasp map \( G \) alone decides whether a grasp holds." \( G \) is only the linear part. A grasp holds only if the required contact forces also lie inside the friction cones; \( \sigma_{\min}(G) > 0 \) is necessary but not sufficient. The convex-hull force-closure test folds in the cones, which is why it, not the rank of \( G \), is the correct criterion.

"Vision is enough for manipulation if the camera is good." At contact the fingers occlude the patch and the decisive signals, sub-millimeter geometry and incipient slip, are not visible. Tactile sensing measures the contact state directly; the strongest manipulation systems fuse both rather than relying on vision near contact.

Self-check

References

1. Murray, R. M., Li, Z., and Sastry, S. S. A Mathematical Introduction to Robotic Manipulation. CRC Press, 1994. Free online. The reference for the grasp map, contact models, and force closure. mlswiki.
2. Lynch, K. M., and Park, F. C. Modern Robotics: Mechanics, Planning, and Control. Cambridge University Press, 2017. Screw theory, wrenches, and grasping. book site.
3. Mason, M. T. Mechanics of Robotic Manipulation. MIT Press, 2001. Friction, contact mechanics, and the natural/artificial constraint decomposition.
4. Siciliano, B., and Khatib, O., eds. Springer Handbook of Robotics, 2nd ed. Springer, 2016. Chapters on contact modeling, grasping, and force control. DOI.
5. Ferrari, C., and Canny, J. "Planning optimal grasps." IEEE ICRA, 1992. The ball and largest-minimum-resisted-wrench grasp-quality metrics. DOI.
6. Nguyen, V.-D. "Constructing force-closure grasps." International Journal of Robotics Research, 7(3), 1988. Constructive synthesis of force-closed grasps. DOI.
7. Raibert, M. H., and Craig, J. J. "Hybrid position/force control of manipulators." ASME J. Dynamic Systems, Measurement, and Control, 103(2), 1981. The selection-matrix controller. DOI.
8. Mason, M. T. "Compliance and force control for computer controlled manipulators." IEEE Trans. Systems, Man, and Cybernetics, 11(6), 1981. Natural and artificial constraints. DOI.
9. Khatib, O. "A unified approach for motion and force control of robot manipulators: the operational space formulation." IEEE J. Robotics and Automation, 3(1), 1987. DOI.
10. Hogan, N. "Impedance control: an approach to manipulation, parts I-III." ASME J. Dynamic Systems, Measurement, and Control, 107(1), 1985. DOI.
11. Colgate, J. E., and Hogan, N. "Robust control of dynamically interacting systems." International Journal of Control, 48(1), 1988. Passivity and contact stability. DOI.
12. Whitney, D. E. "Quasi-static assembly of compliantly supported rigid parts." ASME J. Dynamic Systems, Measurement, and Control, 104(1), 1982. Peg-in-hole and remote-center compliance. DOI.
13. Bicchi, A., and Kumar, V. "Robotic grasping and contact: a review." IEEE ICRA, 2000. Survey of grasp analysis and contact models. DOI.
14. Montana, D. J. "The kinematics of contact and grasp." International Journal of Robotics Research, 7(3), 1988. Rolling and sliding contact kinematics. DOI.
15. Stewart, D. E., and Trinkle, J. C. "An implicit time-stepping scheme for rigid body dynamics with inelastic collisions and Coulomb friction." International Journal for Numerical Methods in Engineering, 39(15), 1996. The LCP time-stepping formulation. DOI.
16. Anitescu, M., and Potra, F. A. "Formulating dynamic multi-rigid-body contact problems with friction as solvable linear complementarity problems." Nonlinear Dynamics, 14, 1997. DOI.
17. Sentis, L., and Khatib, O. "Synthesis of whole-body behaviors through hierarchical control of behavioral primitives." International Journal of Humanoid Robotics, 2(4), 2005. DOI.
18. Escande, A., Mansard, N., and Wieber, P.-B. "Hierarchical quadratic programming: fast online humanoid-robot motion generation." International Journal of Robotics Research, 33(7), 2014. DOI.
19. Kaelbling, L. P., and Lozano-Perez, T. "Hierarchical task and motion planning in the now." IEEE ICRA, 2011. The TAMP framing. DOI.
20. Garrett, C. R., Lozano-Perez, T., and Kaelbling, L. P. "PDDLStream: integrating symbolic planners and blackbox samplers via optimistic adaptive planning." ICAPS, 2020. arXiv:1802.08705.
21. Yuan, W., Dong, S., and Adelson, E. H. "GelSight: high-resolution robot tactile sensors for estimating geometry and force." Sensors, 17(12), 2017. DOI.
22. OpenAI; Andrychowicz, M., et al. "Learning dexterous in-hand manipulation." International Journal of Robotics Research, 39(1), 2020 (Dactyl). arXiv:1808.00177.
23. OpenAI; Akkaya, I., et al. "Solving Rubik's Cube with a robot hand." 2019. arXiv:1910.07113.
24. Mahler, J., et al. "Dex-Net 2.0: deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics." RSS, 2017. arXiv:1703.09312.
25. Pang, T., Suh, H. J. T., Yang, L., and Tedrake, R. "Global planning for contact-rich manipulation via local smoothing of quasi-dynamic contact models." IEEE Trans. Robotics, 39(6), 2023. arXiv:2206.10787.
26. Fang, H.-S., Wang, C., et al. "AnyGrasp: robust and efficient grasp perception in spatial and temporal domains." IEEE Trans. Robotics, 2023. arXiv:2212.08333.
27. Ke, T.-W., Gkanatsios, N., and Fragkiadaki, K. "3D Diffuser Actor: policy diffusion with 3D scene representations." 2024. arXiv:2402.10885.
28. Chi, C., Xu, Z., Pan, C., et al. "Universal Manipulation Interface: in-the-wild robot teaching without in-the-wild robots." RSS, 2024. arXiv:2402.10329.
29. Liu, S., Wu, L., Li, B., et al. "RDT-1B: a diffusion foundation model for bimanual manipulation." Tsinghua University, 2024. arXiv:2410.07864.
30. RDT Team, Tsinghua University. RDT-2, an autoregressive vision-language-action model with a residual vector-quantized action tokenizer, trained on large-scale UMI data, 2025. github.com/thu-ml/RDT2.
31. Luo, J., Xu, C., Wu, J., and Levine, S. "Precise and dexterous robotic manipulation via human-in-the-loop reinforcement learning" (HIL-SERL). 2024. github.com/rail-berkeley/hil-serl.