1/20
@axisrobotics is a compounding data engine for physical AI. You teleoperate a simulated manipulator, produce trajectories, and those trajectories become the raw material that trains robot policies. If you want to actually contribute instead of just reading about foundation models, this is the walkthrough;
2/20 Physical AI is not bottlenecked by another transformer paper. It is bottlenecked by coverage.
Axis treats diversity as a first class constraint. Scene variants, object layouts, camera viewpoints, embodiments, and language goals get randomized at the task generation layer so the dataset is not a pile of near duplicates.
3/20 The flywheel is simple once you see it.
A task starts in pre-training. Humans teleoperate the end effector and produce demonstrations. Those demos train a base policy. The task then moves into post-training. The policy rolls out in the same simulator. You intervene only when it slips. Those corrections are human-gated DAgger data. The next policy iteration is trained on the mix. Better policies change which tasks get collected next.
That loop is the product.
4/20 Setup is boring and that is the point.
You need a desktop or laptop browser. Chrome, Firefox, Safari, or Edge. Phones are not supported because task space control assumes a keyboard and mouse.
Go to Axis Hub, sign in with Privy (email, Google, X, or a wallet). If you do not connect a wallet, Privy mints one. That wallet is for signing trajectories on-chain, not for paying to play. Completing a run is free. Signing is a small Base gas fee.
5/20 The Hub is the task hall.
Each card shows embodiment, scenario (kitchen, home, office, workshop, bathroom, play), skills (pick, place, stack, rotate, transfer, arrange, insert, separate, reach), difficulty, lifecycle stage, and remaining slots.
Filter hard. Beginners should stay on pre-training, beginner or easy, single skill pick and place. Expert long horizon bimanual insert tasks will eat you alive on day one.
6/20 Every task has a lifecycle: Pre-training, Training, Post-training, then Ended.
Pre-training is demonstration collection. You drive from scratch. One valid episode is one demo.
Training is the aggregation window. Slots close. The pipeline trains a policy. You wait.
Post-training is supervision. The policy attempts the rollout. You take over at failure states, correct, and hand control back.
Do not treat those two collection modes as the same job. The data distributions are different and the scoring logic is different.
7/20 You do not jog individual joints. You command the end effector in task space. An inverse kinematics solver maps your Cartesian target into joint commands. Execution interpolates in joint space so the kinematic chain does not stutter or lock at limits.
That distinction matters. You think in grasp frames and approach vectors. The robot thinks in joint trajectories.
8/20 Three input modes, mixed freely.
Hold and drag the blue ring to slide the gripper along the current view plane.
Double click an object to send the arm into a pre-grasp pose. Use this. Crossing empty workspace is low information data. The informative frames are approach, grasp selection, closure, lift, and place.
Orbit, pan, and zoom the camera so you can actually see contact.
9/20 Keyboard, because you will live here.
Translation is in robot base coordinates: Up/Down: end effector plus or minus X Left/Right: plus or minus Y E / D: plus or minus Z
Rotation is in end effector local coordinates:
Q / W: roll around
EE X A / S: pitch
EE Y Z / X: yaw around
EE Z SPACE holds closed / release open on the gripper. C swaps the active arm on bimanual embodiments. Sensitivity slider under the viewport changes how aggressive the servo feels. Raise it if the arm feels like it is swimming through syrup.
10/20 Episode hygiene keys, memorize these before your first scored run.
N saves a checkpoint of simulator state. B restores the last checkpoint. V completes and saves the current episode. R resets and discards the episode. ESC saves everything and exits.
On multi-stage tasks (open drawer, pick object, place inside, close), checkpoint after every successful subgoal. One late collision should not force you to replay the whole horizon.
11/20 A good pre-training demonstration is not a speedrun with panic corrections.
The scorer cares about success, efficiency, and motion smoothness. High jerk and high acceleration look like a human fighting the interface. The refinement pipeline can clean some of that, but raw teleop that already has low jerk trains better policies and scores better for you.
Plan the grasp before you close the fingers. Approach along the object’s principal axis when you can. Avoid dragging the payload through other rigid bodies. Avoid table scraping.
12/20 Initial conditions are sampled per episode. The goal image is a reference end state, not a pixel perfect replica of your spawn. Read the step list. Left click objects if you are unsure which mesh is the target. The name appears on screen.
If the success checker never fires, you probably placed the object in the wrong region, left a drawer ajar, or failed a contact condition. Reset, do not sit there micro jogging forever.
13/20 Post-training is a different sport.
The policy drives. You watch the rollout. The moment the approach vector is wrong, the grasp is unstable, or it commits to the wrong object, hit Take Over.
Then you are back in the same task space interface. Fix the local failure. Hand control back as soon as the state is recoverable.
Long takeovers collapse the session back into a demonstration. That is already what pre-training is for. Correction data is valuable because it is on-policy and sparse. Intervene late enough that the failure is real, early enough that the scene is still salvageable.
14/20 After a valid run, the trajectory is uploaded: joint states, object poses, control actions, gripper state, timestamps, task metadata.
It sits in your Portfolio. Quality score arrives after verification. To make the contribution count on-chain, sign it. That writes a provenance record (task id, contributor id, data id) so the dataset is auditable.
Failed episodes are not uploaded and do not punish you. Repeat attempts on the same task are averaged, so farming one easy scene with sloppy motion is a bad strategy.
15/20 What happens after you close the browser.
Raw teleop is cleaned. Mean acceleration and jerk get reduced in the refinement stage. Then the IsaacSim backend domain randomizes the episode into many visual and dynamics variants: lighting, textures, camera extrinsics, background, slight layout noise.
Those augmented samples feed two training regimes. Task specific imitation learning (behavior cloning, diffusion policy). And generalist vision language action models conditioned on language goals.
The public Franka sim set already showed the point. Continual pretraining of π0.5 on Axis data lifted LIBERO-Plus success from 83.9 to 88.8 and beat a volume matched RoboCasa control by a wide margin. Crowdsourced sim data is not automatically garbage if coverage is high enough.
16/20 Embodiments you will meet: Franka Research 3 style 7DoF arms, OpenArm, other single arm and bimanual stacks, occasionally more exotic end effectors.
On dexterous hands the interface often collapses high dimensional finger joints into template interpolation (open rest pose toward a pinch, tripod, or power grasp). You are still commanding task space plus a grasp parameter, not puppeteering 16 joints by hand.
Sim to real is the eventual test. Policies trained in this loop get evaluated under randomized initial conditions, then transferred onto real hardware for validation. Your browser episode is not a toy if the asset library and contact model are honest.
17/20 Quality checklist I wish someone had posted on day one.
Watch depth. A top down camera lies about how close the fingertips are to the rim. Commit to a grasp frame and stop hunting. Lift before you translate so you do not plow the table. On place, hover, square the orientation, then descend. Do not fling.
If the policy in post-training is oscillating, that is a sign, not a vibe. Take over at the start of the bad attractor, not after the object is on the floor. Do the newbie practice tasks first. They do not score. They teach the coupling between view plane drag and base frame keys.
18/20 Common failure modes.
Driving in joint space in your head while the UI is Cartesian. You will fight the IK.
Never moving the camera. Occlusion creates false contacts in your brain.
Closing the gripper in free space then hoping the object teleports in.
Ignoring articulated joints. A cabinet door that is 10 degrees short of the success region fails the checker.
Taking over for ten seconds in post-training because it feels safer. You just spent the intervention budget turning a correction into another demo.
19/20 If you care about the research layer, the terms to keep straight are:
Trajectory: the time series of state and action. Policy: the mapping from observation to action. Rollout: executing that policy in the environment. Success checker: the programmatic predicate that labels the episode. Domain randomization: expanding one demo into many visual and dynamics cousins. Human-gated DAgger: the expert takes over only at policy failure states. VLA: a policy that conditions actions on vision and language.
Axis is trying to industrialize those pieces with a global contributor pool instead of a 12 person teleop team in one lab.
start here:
s.kaito.ai/Ks54L4v