Publications

Robot Utility Models: General Policies for Zero-Shot Deployment in New Environments

Haritheja Etukuru, Norihito Naka, Zijin Hu, Seungjae Lee, Julian Mehu, Aaron Edsinger, Chris Paxton, Soumith Chintala, Lerrel Pinto, Nur Muhammad Mahi Shafiullah

AnySkin: Plug-and-play Skin Sensing for Robotic Touch

Raunaq Bhirangi, Venkatesh Pattabiraman, Enes Erciyes, Yifeng Cao, Tess Hellebrekers, Lerrel Pinto

Bridging the Human to Robot Dexterity Gap through Object-Oriented Rewards

Irmak Güzey, Yinlong Dai, Georgy Savva, Raunaq Bhirangi, Lerrel Pinto

DynaMem: Online Dynamic Spatio-Semantic Memory for Open World Mobile Manipulation

Peiqi Liu, Zhanqiu Guo, Mohit Warke, Soumith Chintala, Chris Paxton, Nur Muhammad Mahi Shafiullah, Lerrel Pinto

P3-PO: Prescriptive Point Priors for Visuo-Spatial Generalization of Robot Policies

Mara Levy, Siddhant Haldar, Lerrel Pinto, Abhinav Shirivastava

Training Language Models on Synthetic Edit Sequences Improves Code Synthesis

Ulyana Piterbarg, Lerrel Pinto, Rob Fergus

DynaMo: In-Domain Dynamics Pretraining for Visuo-Motor Control

Zichen Jeff Cui, Hengkai Pan, Aadhithya Iyer, Siddhant Haldar, Lerrel Pinto

BAKU: An Efficient Transformer for Multi-Task Policy Learning

Siddhant Haldar, Zhuoran Peng, Lerrel Pinto

OPEN TEACH: A Versatile Teleoperation System for Robotic Manipulation

Aadhithya Iyer, Zhuoran Peng, Yinlong Dai, Irmak Güzey, Siddhant Haldar, Soumith Chintala, Lerrel Pinto

OK-Robot: What Really Matters in Integrating Open-Knowledge Models for Robotics

Peiqi Liu, Yaswanth Orru, Jay Vakil, Chris Paxton, Nur Muhammad Mahi Shafiullah, Lerrel Pinto

VQ-BeT: Behavior Generation with Latent Actions

Seungjae Lee, Yibin Wang, Haritheja Etukuru, H Jin Kim, Nur Muhammad Mahi Shafiullah, Lerrel Pinto

Adaptive Sampling of k-space in Magnetic Resonance for Rapid Pathology Prediction

Chen-Yu Yen, Raghav Singhal, Umang Sharma, Rajesh Ranganath, Sumit Chopra, Lerrel Pinto

Hierarchical State Space Models for Continuous Sequence-to-Sequence Modeling

Raunaq Bhirangi, Chenyu Wang, Venkatesh Pattabiraman, Carmel Majidi, Abhinav Gupta, Tess Hellebrekers, Lerrel Pinto

diff History for Neural Language Agents

Ulyana Piterbarg, Lerrel Pinto, Rob Fergus

On Bringing Robots Home

Nur Muhammad Mahi Shafiullah, Anant Rai, Haritheja Etukuru, Yiqian Liu, Ishan Misra, Soumith Chintala, Lerrel Pinto

See to Touch: Learning Tactile Dexterity through Visual Incentives

Irmak Güzey, Yinlong Dai, Ben Evans, Soumith Chintala, Lerrel Pinto

NetHack is Hard to Hack

Ulyana Piterbarg, Lerrel Pinto, Rob Fergus

Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Play

Irmak Güzey, Ben Evans, Soumith Chintala, Lerrel Pinto

That Sounds Right: Auditory Self-Supervision for Dynamic Robot Manipulation

Abitha Thankaraj, Lerrel Pinto

Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrations

Siddhant Haldar, Jyothish Pari, Anant Rai, Lerrel Pinto

CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memory

Mahi Shafiullah, Chris Paxton, Lerrel Pinto, Soumith Chintala, Arthur Szlam

From play to policy: Conditional behavior generation from uncurated robot data

Jeff Cui, Yibin Wang, Mahi Shafiullah, Lerrel Pinto

Holo-Dex: Teaching Dexterity with Immersive Mixed Reality

Sridhar Pandian Arunachalam, Irmak Güzey, Soumith Chintala, Lerrel Pinto

Behavior Transformers: Cloning k modes with one stone

Mahi Shafiullah, Jeff Cui, Ariuntuya Altanzaya, Lerrel Pinto

Watch and Match: Supercharging Imitation with Regularized Optimal Transport

Siddhant Haldar, Vaibhav Mathur, Denis Yarats, Lerrel Pinto

Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulation

Sridhar Pandian Arunachalam, Sneha Silwal, Ben Evans, Lerrel Pinto

Don't Change the Algorithm, Change the Data: Exploratory Data for Offline Reinforcement Learning

Denis Yarats, David Brandfonbrener, Hao Liu, Michael Laskin, Pieter Abbeel, Alessandro Lazaric, Lerrel Pinto

The Surprising Effectiveness of Representation Learning for Visual Imitation

Jyothish Pari, Nur Muhammad Shafiullah, Sridhar Pandian Arunachalam, Lerrel Pinto

Context is Everything: Implicit Identification for Dynamics Adaptation

Ben Evans, Abitha Thankaraj, Lerrel Pinto

Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learning

Denis Yarats, Rob Fergus, Alessandro Lazaric, Lerrel Pinto

Reinforcement Learning with Prototypical Representations

Denis Yarats, Rob Fergus, Alessandro Lazaric, Lerrel Pinto

Task-Agnostic Morphology Evolution

Donald J. Hejna III, Pieter Abbeel, Lerrel Pinto

Self-Supervised Policy Adaptation during Deployment

Nicklas Hansen, Rishabh Jangir, Yu Sun, Guillem Alenya, Pieter Abbeel, Alexei A. Efros, Lerrel Pinto, Xiaolong Wang

Learning Cross-Domain Correspondence for Control with Dynamics Cycle-Consistency

Qiang Zhang, Tete Xiao, Alexei A. Efros, Lerrel Pinto, Xiaolong Wang

Visual Imitation Made Easy

Sarah Young, Dhiraj Gandhi, Shubham Tulsiani, Abhinav Gupta, Pieter Abbeel, Lerrel Pinto

Robust Policies via Mid-Level Visual Representations

Bryan Chen, Alexander Sax, Francis E. Lewis, Silvio Savarese, Jitendra Malik, Amir Zamir, Lerrel Pinto

Learning Predictive Representations for Deformable Objects Using Contrastive Estimation

Wilson Yan, Ashwin Vangipuram,, Pieter Abbeel, Lerrel Pinto

Reinforcement Learning with Augmented Data

Michael Laskin, Kimin Lee, Adam Stooke, Lerrel Pinto, Pieter Abbeel, Aravind Srinivas

Generalized Hindsight for Reinforcement Learning

Alexander Li, Lerrel Pinto, Pieter Abbeel

Automatic Curriculum Learning through Value Disagreement

Yunzhi Zhang, Pieter Abbeel, Lerrel Pinto

Swoosh! Rattle! Thump! - Actions that Sound

Dhiraj Gandhi, Abhinav Gupta, Lerrel Pinto

Learning to Manipulate Deformable Objects without Demonstrations

Yilin Wu, Wilson Yan, Thanard Kurutach, Lerrel Pinto, Pieter Abbeel

Hierarchically Decoupled Imitation for Morphological Transfer

Donald J. Hejna III, Pieter Abbeel, Lerrel Pinto

PyRobot: An Open-source Robotics Framework for Research and Benchmarking

Adithya Murali, Tao Chen, Kalyan Vasudev Alwala, Dhiraj Gandhi, Lerrel Pinto, Saurabh Gupta, Abhinav Gupta

Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation

Pratyusha Sharma, Lekha Mohan, Lerrel Pinto, Abhinav Gupta

Robot Learning in Homes: Improving Generalization and Reducing Dataset Bias

Abhinav Gupta, Adithya Murali, Dhiraj Gandhi, Lerrel Pinto

Asymmetric Actor Critic for Image-Based Robot Learning

Lerrel Pinto, Marcin Andrychowicz, Peter Welinder, Wojciech Zaremba, Pieter Abbeel

CASSL: Curriculum Accelerated Self-Supervised Learning

Adithyavairavan Murali, Lerrel Pinto, Dhiraj Gandhi, Abhinav Gupta

Learning to Fly by Crashing

Dhiraj Gandhi, Lerrel Pinto, Abhinav Gupta

Robust Adversarial Reinforcement Learning

Lerrel Pinto, James Davidson, Rahul Sukthankar, Abhinav Gupta

Supervision via Competition: Robot Adversaries for Learning Tasks

Lerrel Pinto, James Davidson, Abhinav Gupta

Learning to Push by Grasping: Using multiple tasks for effective learning

Lerrel Pinto, Abhinav Gupta

The Curious Robot: Learning Visual Representations via Physical Interactions

Lerrel Pinto, Dhiraj Gandhi, Yuanfeng Han, Yong-Lae Park, Abhinav Gupta

Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hours

Lerrel Pinto, Abhinav Gupta