Dhruv Shah

Dhruv Shah
rsvha.uedeeh@.dyrulkbeh

I am a Senior Research Scientist at Google DeepMind, working foundation models of and for robotics. Previously, I obtained my PhD in EECS at UC Berkeley, where I was advised by Sergey Levine. My research was supported by the Berkeley Fellowship for Graduate Study, and has been nominated for (and received) several Best Paper Awards at leading robotics conferences, including RSS and ICRA.

Earlier, I graduated with honors from IIT Bombay, where I received the Undergraduate Research Award and the Institute Academic Prize. I have also been fortunate to spend time at Meta AI (FAIR), Google DeepMind (Brain Robotics), Carnegie Mellon University, Imperial College London and the University of Sydney.

I will join Princeton next academic year as an Assistant Professor in ECE and Robotics! If you are interested in working with me, please apply to the centralized admission portal and mention my name.

CV / Scholar / Twitter / LinkedIn

Updates

[Aug 2024] I defended my PhD thesis at Berkeley!
[July 2024] I started full-time at Google DeepMind Robotics
[May 2024] NoMaD and RT-X both won the Best Conference Paper award at ICRA 2024!
[Mar 2024] I was named a 2024 Microsoft Future Leader in Robotics and AI by the University of Maryland
[Feb 2024] I will be giving invited talks at MIT, Princeton, UCSD, UCLA, Michigan, Purdue, and Columbia
[Jan 2024] NoMaD and RT-X were accepted to ICRA 2024 with 6 best paper award nominations!
[Dec 2023] I will be speaking at the ML4AD workshop at NeurIPS 2023
[Nov 2023] I am on the job market! Please reach out if you think I would be a good fit for your team/department.
[Oct 2023] Our work on grounding language models for robotic tasks was accepted to NeurIPS 2023!
[Oct 2023] Our work on data collection for social navigation was accepted to RA Letters and will be presented at ICRA 2024!
[Sep 2023] We are organizing the 2nd Workshop on Language and Robot Learning @ CoRL 2023: consider submitting
[Sep 2023] Three papers to appear at CoRL 2023: a foundation model for visual navigation, autonomous high-speed off-road driving, and coming soon!
[Aug 2023] We are organizing the 6th Robot Learning Workshop @ NeurIPS 2023: consider submitting
[Jan 2023] Two papers on generalizable navigation policies across robots to accepted to ICRA 2023: GNM and Ex-Aug!
[Jan 2023] We are organizing the 2nd Workshop on Learning from Diverse, Offline Data at ICRA 2023 — consider submitting
[Dec 2022] We published an opinion and survey paper on data-driven navigation!
[Oct 2022] LM-Nav was featured in a video by Two Minute Papers!
[Sep 2022] We are organizing the Workshop on Language and Robot Learning at CoRL 2022 — consider submitting
[Sep 2022] Two papers accepted to CoRL 2022: LM-Nav and ReViND!
[Apr 2022] Our paper on kilometer-scale visual navigation was accepted to RSS 2022 as a Best Systems Paper Finalist
[Mar 2022] We are organizing the Workshop on Learning from Diverse, Offline Data at RSS 2022 — consider submitting
[Mar 2022] ViKiNG was featured in the news — see coverage by IEEE Spectrum and ZDNet!
[Jan 2022] Our paper on skill-centric state abstractions was accepted to ICLR 2022
[Jan 2022] Our paper on combining geometric costmaps with learned models was accepted to ICRA 2022
[Oct 2021] Check out this blog post by Sergey on data-driven robot learning, featuring some of my recent research
[Sep 2021] Our paper on goal-directed exploration on mobile robots was accepted as an Oral Presentation to CoRL 2021
[Aug 2021] I'm teaching the Berkeley Deep RL class (CS285) as Head GSI this semester
[May 2021] I'll be spending the summer at Google Brain Robotics, interning with Brian Ichter and Alex Toshev
[Feb 2021] Our paper on navigating to visual goals in the real-world was accepted to ICRA 2021
[May 2020] Our paper on hybrid control for aerial manipulation was accepted to RSS 2020
[Jan 2020] Our paper on analyzing ingredients of real-world RL for robots was accepted to ICLR 2020

Blog Posts

Scaling up learning across many different robot types: massive inter-university collaboration on cross-embodiment learning
Extracting Skill-Centric State Abstractions from Value Functions: our work on obtaining state abstractions that are informed by capabilities of low-level skills
Learning to Explore the Real World with a Ground Robot: our work on goal-directed exploration with a mobile robot
The Ingredients of Real World Robotic Reinforcement Learning: our work on robot learning in the real-world without instrumentation

Publications

	Open X-Embodiment: Robotic Learning Datasets and RT-X Models Open X-Embodiment Collaboration Best Conference Paper Award Best Student Paper Award (Finalist) Best Paper Award in Robot Manipulation (Finalist) International Conference on Robotics and Automation (ICRA), 2024 CoRL 2023 Workshop Towards Generalist Robots (Oral Presentation) arXiv / Blog Post / Dataset / Code
	NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration Ajay Sridhar, Dhruv Shah, Catherine Glossop, Sergey Levine Best Conference Paper Award Best Student Paper Award (Finalist) Best Paper Award in Cognitive Robotics (Finalist) International Conference on Robotics and Automation (ICRA), 2024 CoRL 2023 Workshop on Pre-Training for Robot Learning (Oral Presentation) NeurIPS 2023 Workshop on Foundation Models for Decision-Making (Oral Presentation) arXiv / Summary Video / Dataset / Code
	SACSoN: Scalable Autonomous Data Collection for Social Navigation Noriaki Hirose, Dhruv Shah, Ajay Sridhar, Sergey Levine Robotics and Automation Letters (RA-L), 2023 International Conference on Robotics and Automation (ICRA), 2024 Conference on Robot Learning (CoRL), 2023 (Live Demo) arXiv / Summary Video / Dataset
	Grounded Decoding: Guiding Text Generation with Grounded Models for Robot Control Wenlong Huang, Fei Xia, Dhruv Shah, Danny Driess, Andy Zeng, Yao Lu, Pete Florence, Igor Mordatch, Sergey Levine, Karol Hausman, Brian Ichter Advances in Neural Information Processing Systems (NeurIPS) 2023 arXiv / Summary Video
	ViNT: A Foundation Model for Visual Navigation Dhruv Shah^†, Ajay Sridhar^†, Nitish Dashora^†, Kyle Stachowicz, Kevin Black, Noriaki Hirose, Sergey Levine Conference on Robot Learning (CoRL), 2023 (Oral Presentation & Live Demo) Bay Area Machine Learning Symposium (BayLearn) 2022 (Oral Presentation) arXiv / Summary Video / Code
	FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicing Kyle Stachowicz^†, Dhruv Shah^†, Arjun Bhorkar, Ilya Kostrikov, Sergey Levine Conference on Robot Learning (CoRL), 2023 arXiv / Summary Video / Code / Media Coverage ^{1, 2, 3, 4}
	Navigation with Large Language Models: Semantic Guesswork as a Heuristic for Planning Dhruv Shah^†, Michael Equi^†, Blazej Osinski, Fei Xia, Brian Ichter, Sergey Levine Conference on Robot Learning (CoRL), 2023 arXiv / Summary Video / Code / Interactive Colab
	ExAug: Robot-Conditioned Navigation Policies via Geometric Experience Augmentation Noriaki Hirose, Dhruv Shah, Ajay Sridhar, Sergey Levine International Conference on Robotics and Automation (ICRA), 2023 arXiv / Summary Video
	GNM: A General Navigation Model to Drive Any Robot Dhruv Shah^†, Ajay Sridhar^†, Arjun Bhorkar, Noriaki Hirose, Sergey Levine International Conference on Robotics and Automation (ICRA), 2023 arXiv / Summary Video / Code / Media Coverage
	Learning Robotic Navigation from Experience: Principles, Methods, and Recent Results Sergey Levine, Dhruv Shah Philosophical Transactions of the Royal Society B, 2022 (Invited Paper) arXiv
	Offline Reinforcement Learning for Visual Navigation Dhruv Shah^†, Arjun Bhorkar^†, Hrish Leen, Ilya Kostrikov, Nick Rhinehart, Sergey Levine Conference on Robot Learning (CoRL), 2022 (Oral Presentation) arXiv / Talk @ CoRL / Code
	LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action Dhruv Shah^†, Blazej Osinski^†, Brian Ichter, Sergey Levine Conference on Robot Learning (CoRL), 2022 Foundation Models for Decision Making Workshop at NeurIPS 2022 (Oral Presentation) Bay Area Machine Learning Symposium (BayLearn) 2022 (Oral Presentation) arXiv / Summary Video / Code / Interactive Colab / 2MP Feature / Spotlight @ CoRL / Media Coverage
	ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hints Dhruv Shah, Sergey Levine Best Systems Paper Finalist Robotics: Science and Systems (RSS), 2022 (Oral Presentation) arXiv / Summary Video / Talk @ RSS / Media Coverage ^{1, 2, 3}
	Hybrid Imitative Planning with Geometric and Predictive Costs in Off-road Environments Nitish Dashora^†, Daniel Shin^†, Dhruv Shah, Henry Leopold, David Fan, Ali Agha-Mohammadi, Nicholas Rhinehart, Sergey Levine International Conference on Robotics and Automation (ICRA), 2022 arXiv / Talk @ ICRA
	Value Function Spaces: Skill-Centric State Abstractions for Long-Horizon Reasoning Dhruv Shah, Peng Xu, Yao Lu, Ted Xiao, Alexander Toshev, Sergey Levine, Brian Ichter International Conference on Learning Representations (ICLR), 2022 Blog Post / arXiv / Talk @ ICLR
	Rapid Exploration for Open-World Navigation with Latent Goal Models Dhruv Shah, Benjamin Eysenbach, Nicholas Rhinehart, Sergey Levine Conference on Robot Learning (CoRL), 2021 (Oral Presentation) Workshop on Never-Ending Reinforcement Learning at ICLR 2021 (Oral Presentation) Blog Post / arXiv / Talk @ CoRL / Talk @ ICLR / Dataset / Media Coverage
	ViNG: Learning Open-World Navigation with Visual Goals Dhruv Shah, Benjamin Eysenbach, Gregory Kahn, Nicholas Rhinehart, Sergey Levine International Conference on Robotics and Automation (ICRA), 2021 arXiv / Summary Video
	Aerial Manipulation Using Hybrid Force and Position NMPC Applied to Aerial Writing Dimos Tzoumanikas, Felix Graule, Qingyue Yan, Dhruv Shah, Marija Popovic, Stefan Leutenegger Robotics: Science and Systems (RSS), 2020 arXiv / Talk @ RSS / Cool Demos
	The Ingredients of Real World Robotic Reinforcement Learning Henry Zhu^†, Justin Yu^†, Abhishek Gupta^†, Dhruv Shah, Kristian Hartikainen, Avi Singh, Vikash Kumar, Sergey Levine International Conference on Learning Representations (ICLR), 2020 (Spotlight Presentation) Blog Post / arXiv / Talk / Virtual Presentation
	Swarm Aggregation Without Communication and Global Positioning Dhruv Shah, Leena Vachhani Robotics and Automation Letters (RA-L), 2019 International Conference on Robotics and Automation (ICRA), 2019
	Projection Design for Compressive Source Separation using Mean Errors and Cross-Validation Dhruv Shah, Ajit Rajwade International Conference on Image Processing (ICIP), 2019
	Designing Constrained Projections for Compressed Sensing: Mean Errors and Anomalies with Coherence Dhruv Shah^†, Alankar Kotwal^†, Ajit Rajwade Global Conference on Signal and Information Processing (GlobalSIP), 2018

Of course, I did not design this page!