2026 Oct Update

TL;DR

• Configure kernel selection and Humming quantization through one YAML policy, with INT4, INT8, FP8, MXFP4, and NVFP4 options.
• Scale independent requests across complete model replicas, using the Engine for local DP and a gateway for multiple hosts and LeRobot clients.
• Run π0.5 PPO rollouts with actor weight updates. The PhyAI runtime is merged; RLinf integration is under review.
• Run MolmoAct2 continuous-action inference and Qwen-Image 2.1 image generation and editing.

This update covers changes merged through October 10, 2026. Two companion articles explain the systems work in more detail: kernel dispatch and Humming quantization, and data-parallel inference across GPUs and hosts.

1. Kernel dispatch and Humming quantization

Choosing a kernel depends on the workload. The GPU matters, but so do batch size, tensor layout, precision, and the particular stage of a model. We moved those decisions into a shared dispatcher so each model can reuse the same selection machinery.

Layers describe the operation they need, and the dispatcher checks candidate implementations against their capabilities. A YAML policy can then prefer a backend, restrict a workload to a measured implementation, or define a fallback. The system covers GEMM, attention, normalization, convolution, and other recurring operations. It caches prepared choices for repeated execution and supports optional autotuning for operations that its built-in benchmark can measure.

Humming quantization is a practical use of this design. One policy chooses which layers to quantize and which kernels execute the resulting weights. PhyAI can load an original checkpoint and quantize selected layers to INT4, INT8, FP8, MXFP4, or NVFP4, subject to each scheme’s hardware and shape requirements.

The supplied π0.5 W4A16 preset, for example, uses INT4 weights for the language model and action expert’s attention/MLP projections while keeping BF16 or FP16 activations. The vision tower and input/output heads retain their configured precision. Those quantized projections use compatible Humming kernels; other operations keep their own selected implementations. We also provide Cosmos3 presets.

Teams can keep sensitive layers at their original precision and evaluate lower-precision projections on their own checkpoint and tasks. The kernel dispatcher and Humming article explains how to specify those choices and includes a runnable preset example.

2. Data-parallel inference and multi-host serving

Many robots or environments can request actions from the same policy at once. Data-parallel (DP) inference handles that independent work by running complete model replicas on multiple GPUs. Each replica processes its own input through the full model path.

PhyAI’s unified Engine now manages this replica pool. A deployment configuration sets the number of copies and their GPU placement. Applications submit concurrent requests through Engine.submit(), and the local dispatcher routes each whole request to a healthy replica with the fewest requests in flight. Each submission has its own future; the caller can collect results in the required order.

This keeps DP outside the model implementation. π0.5 can run as several independent single-GPU replicas. A model such as Cosmos3 can use a supported TP/CFG group for each replica and then replicate that group. Applications that want to distribute a large batch prepare separate requests and assemble the results themselves.

The gateway extends the pool across hosts. For π0.5, each network replica runs in its own model server process, and the gateway routes requests among healthy servers registered under the same model name. This gives separate replicas the concurrency to serve different clients. The LeRobot adapter connects its asynchronous clients to that pool through a shared address.

The first benefit is more capacity for simultaneous inference, with less queueing when there is enough independent work to use the replicas. The Data-parallel inference in PhyAI article explains the local and multi-host paths, provides a concurrent submission example, and covers the current server-adapter constraints.

3. π0.5 rollouts for reinforcement learning

PPO needs more from inference than a final action. The training system needs the sampled trajectory and its probabilities, and the rollout model needs to receive updated actor weights as training progresses. PhyAI now provides those pieces for π0.5.

The rollout runtime returns actions, the denoising chain, per-transition log probabilities, and optional values. An external controller supplies the sampling schedule and owns the training algorithm. PhyAI executes the model and returns the data needed for training. It also accepts actor weight updates without rebuilding the engine, then refreshes weight-dependent state and recaptures the affected CUDA Graphs.

The RLinf integration PR reports measured π0.5 PPO results on H800 GPUs. In its controlled four-GPU profile, PhyAI more than halved prediction time on the critical path. The improvement in the complete RL step is smaller because environment execution, actor updates, and other work still take time.

A separate run on eight H800 GPUs completed 500 PPO steps from a fresh start on LIBERO-Spatial. The PR reports mean success_once of 0.8797 for PhyAI and 0.8684 for its stitched HF reference aligned by global step. This is a training-run comparison; the timing results above come from the separate four-GPU profile.

These measurements describe the integration versions used for those experiments and predate the final cleanup commits. The PR documents the software stack, training configuration, and separate focused validation. PhyAI’s rollout and weight-update support is merged; the RLinf integration remains under upstream review on October 10. The current gateway’s bundled π0.5 server is a separate serving path and needs a matching adapter for RLinf’s SGLang-style payloads.

For teams building a rollout controller, the π0.5 integration guide describes the request fields and weight-update lifecycle. The backend work gives training systems access to PhyAI’s inference runtime while keeping sampling and PPO decisions in the training framework.

4. New model support

MolmoAct2

MolmoAct2 support landed on October 10. The model uses camera images, a task instruction, and robot state to predict a sequence of continuous actions. PhyAI runs its vision-language backbone and action expert in BF16 and supports CUDA Graph execution for the repeated action-velocity computation.

The integration includes checkpoint loading, observation processing, action postprocessing, and an example using the franka_droid configuration. That example takes three RGB camera views and an eight-dimensional state, then returns 15 actions with eight values each. The base checkpoint is intended for task adaptation; the example validates the inference path rather than establishing a robot task success rate. Depth reasoning and the action-expert depth gate are outside the currently supported configuration.

Qwen-Image 2.1

Qwen-Image 2.1 adds text-to-image generation and image editing to PhyAI. Editing accepts up to ten reference images. The execution path includes conditioning, diffusion sampling, and VAE decoding, with examples that save the generated result as a PNG.

The model supports tensor parallelism and optional parallel CFG branches through the shared Engine deployment layer. A user can start with one GPU or configure more devices for the supported topology without replacing the model entry point. The implementation entered main on October 5.

August also brought the GR00T N1.7 integration with mixed-length batching, CUDA Graph profiles, and LIBERO evaluation tools. π0 gained additional LeRobot alignment and benchmarking tools, and the MiniCPM-Robot Day-0 work merged into the main branch. The two newly added model families in this October update are MolmoAct2 and Qwen-Image 2.1.

All four areas are available to inspect in the PhyAI repository. Start with the installation guide for source and nightly packages, then use the model and deployment guides for the workload you want to run.

Next

1. We are tuning kernels for NVIDIA Thor (SM110) using a KDA-like approach. Early results look promising, and we plan to merge these optimizations into PhyAI in a future update.
2. Support for Lingbot-VLA and Xiaomi VLA is also in progress.