Multimodal Learning Intern - Large Behavior Models, Pretraining

At Toyota Research Institute (TRI), we’re on a mission to improve the quality of human life. We’re developing new tools and capabilities to amplify the human experience. To lead this transformative shift in mobility, we’ve built a world-class team in Energy & Materials, Human-Centered AI, Human Interactive Driving, Large Behavioral Models, and Robotics.

This is a summer 2025 paid 12-week internship opportunity. Please note that this internship will be a hybrid in-office role.

The Mission

We are working to create general-purpose robots capable of accomplishing a wide variety of dexterous tasks. To do this, our team is building general-purpose foundation models for dexterous robot manipulation. These models, which we call Large Behavior Models (LBMs), use generative AI techniques to produce robot action from sensor data and human request. To accomplish this, we are training large multimodal models from a combination of internet-scale text, image, and video data, as well as a curriculum of embodied robot demonstration data. 

The Team

The Pretraining team focuses on training foundation models that can effectively bridge visual, language, and robotic domains. We combine large-scale model training with empirical validation in simulation and on physical robots, emphasizing both fundamental research advances and practical capabilities. Our work spans computer vision, multimodal learning, and robotic control, with particular focus on scaling up model architectures, training data, and training approaches that generalize effectively to various downstream tasks.

The Internship

As a Research Scientist Intern, you will conduct research in robot foundation model pretraining alongside our core technical team. You'll work on developing and implementing large-scale multimodal models that bridge visual, language, and robotic domains and validate them on our simulated and physical robot fleet.

Responsibilities

• Advance the state of the art in training large-scale robot foundation models, and validate the impact of that research on real-world benchmarks and robots.
• Work as part of a dynamic, closely-knit research team.
• Implement high-performance machine-learning pipelines and optimize data and learning stacks for scalability, efficiency, and performance.
• Present results in verbal and written communications at international conferences, internally, and via open-source contributions to the community.
• Collaborate with internal research scientists, our engineering team, and our partner labs at top academic research universities including MIT, Stanford, Berkeley, CMU, Columbia, and Princeton to drive pioneering research at scale.

Qualifications

• Currently pursuing a Ph.D. in Machine Learning, Robotics, or related fields.
• Publications at high-impact conferences/journals (e.g., NeurIPS, ICLR, ICML, ACL, EMNLP, CVPR, CoRL, RSS, etc.) on some of the aforementioned topics.
• Passionate about large-scale challenges in ML grounded in physical systems, especially in the space of robotics.
• Proficiency with one or more coding languages and systems, preferably Python, Unix, and a Deep Learning framework (e.g., PyTorch).
• Ability to work in collaboration with other researchers and engineers to invent and develop interesting research ideas.
• Experience training large-scale foundation models (VLMs, text-to-video models, etc) is desirable.

The pay range for this position at commencement of employment is expected to be between $45 and $65/hour for California-based roles; however, base pay offered may vary depending on multiple individualized factors, including market location, job-related knowledge, skills, and experience. Note that TRI offers a generous benefits package including vacation and sick time. Details of participation in these benefit plans will be provided if an employee receives an offer of employment.

Please reference this Candidate Privacy Notice to inform you of the categories of personal information that we collect from individuals who inquire about and/or apply to work for Toyota Research Institute, Inc. or its subsidiaries, including Toyota A.I. Ventures GP, L.P., and the purposes for which we use such personal information.

TRI is fueled by a diverse and inclusive community of people with unique backgrounds, education and life experiences. We are dedicated to fostering an innovative and collaborative environment by living the values that are an essential part of our culture. We believe diversity makes us stronger and are proud to provide Equal Employment Opportunity for all, without regard to an applicant’s race, color, creed, gender, gender identity or expression, sexual orientation, national origin, age, physical or mental disability, medical condition, religion, marital status, genetic information, veteran status, or any other status protected under federal, state or local laws.

It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability. Pursuant to the San Francisco Fair Chance Ordinance, we will consider qualified applicants with arrest and conviction records for employment.