REINFORCE Zero
Kernel alignment and simple importance sampling for 20,000 stable asynchronous RL updates.
I work with NVIDIA Research on reinforcement learning, large language models, and algorithm–system co-design. I am interested in advancing artificial general intelligence (AGI) through scalable learning, with a current focus on RL for LLMs and efficient large-scale model training.
Previously, I worked in NVIDIA’s compute architecture team on scalable training for RLVR. Before that, my research focused on multi-agent reinforcement learning in game environments and theory.
Selected work
Research
Nemotron 3.5
Open hybrid Mamba–Transformer models for efficient reasoning and agentic AI.
FlashREINFORCE
Critic-free, single-rollout asynchronous RL for agentic language models.
REINFORCE++
Simple, stable critic-free policy optimization, adopted in verl and NVIDIA NeMo RL.
ProRL V2.1
Prolonged reinforcement learning and training recipes for sustained reasoning gains.
BroRL
Scaling reinforcement learning through broader exploration. ICML 2026.
Skill2Env
Turning collective Agent Skills into executable training environments.
PyMARL2
Cooperative multi-agent RL, optimized QMIX, and studies of implementation and monotonicity.
Open source
OpenRLHF
Scalable RLHF and agent training with Ray and vLLM. Project lead and core maintainer.
Molt
A lightweight, Hugging Face-native framework for large-scale agentic RL. Development lead.
vLLM
RL API design and integration, with runtime weight synchronization and extensible workers.
Awesome LLM Strawberry
A curated collection of papers, blogs, and resources on LLM reasoning.