Machine Learning Researcher (PhD)

Who we are

We’re redefining how humans interact with massive amounts of imagery & video.

What that means

We’re building a near future where people can get information out of imagery at scale without building their own ML models. Collaborating with users around the world, our AI research and products change how people monitor and protect our oceans.

Our core technology: Find Anything & Video Search. We make it possible to find anything in imagery/video, without having seen it in training. Advancing the field of computer vision, pioneered in-house at OnDeck and published at one of the world’s most distinguished AI conferences (NeurIPS).

Backed by National Geographic, the Environmental Defense Fund, Canada’s Ocean Supercluster, and many more, we stand at the forefront of AI for ocean conservation & protection. Our software has been deployed by governments, Indigenous nations, scientists, NGOs, and industry around the world. 

What you’ll be doing

We’re excited to be pushing the boundaries of what is possible in computer vision: creating grounded visual reasoning & fine-grained classification, and searching for anything in thousands of hours of video footage with natural language search. What you’ll be doing is extending our NeurIPS workshop paper across several domains, scaling it to thousands of classes, and bridging the gap between video understanding models and our methods. We’ll provide the dedicated cloud GPUs and research autonomy for you to keep pushing the boundaries of visual reasoning.

You’ll have the opportunity to work alongside our engineering team to bring this applied research into reality, watching as your research turns into real products in just a few weeks. And you’ll get to publish to conferences and journals too.

Impact and scope of work: 

You will have ownership over mission-critical research, propelling us towards the frontier of ocean tech and our goal of end-to-end visual automation. In this mix of machine learning research and research engineering, you will

• Develop state-of-the-art machine learning models that can reason about visual data and retrieve relevant answers from external knowledge sources. 

• Research, implement, and optimize for various tasks, including object detection, species classification, and multi-object tracking for scalable ocean vision systems and environmental monitoring.

• Devise better data-driven models for information retrieval, hierarchical classification, multi-modal fusion, generation and media understanding (CV & NLP).

• Collaborate with engineers to translate research into products, as well as  communicating research plans, progress, results, and impact to the whole organization and in conference/journal papers.

• Collaborate and work across teams to develop concepts that advance the entire product pipeline, all the way to production (infrastructure, software, data collection, machine learning, etc.).

• Conduct research that enables learning the semantics of data across multiple modalities (images, video, text, physical sensors, and other modalities).

You will also get visibility of the business and product side, and see how OnDeck is commercializing machine learning systems and infrastructure to deliver recurring value to customers and real change to ocean sustainability.

Qualifications

• Ph.D. degree in Computer Science, Machine Learning, Artificial Intelligence, or relevant technical field.

• Experience as a machine learning researcher, including internships and full-time.

• Proficiency in programming and implementing machine learning workflows using Python (experience with C, C++, and JavaScript/TypeScript is an asset)

• Research experience with vision language models/large vision models, multi-modal encoder-decoder design, domain adaptation of open-vocabulary models, information retrieval problems, or other areas related to NLP and CV.

• Proven track record of achieving significant results as demonstrated by awards, fellowships, patents, as well as first-authored publications at leading journals or conferences such as NeurIPS, ICLR, ICML, JMLR, AAAI, KDD, CVPR, ECCV, ICCV, ACL, NAACL, EACL, or similar.

• Experience communicating research for public audiences of peers.

• Experience manipulating and analyzing complex, large scale, potentially high-dimensionality data from varying sources.

• Experience in utilizing theoretical and empirical research to solve problems.

• Authorized to work in Canada (work permit or other).

• Strong technical communication skills in English, both written and verbal

• Enthusiasm for research that revolutionizes ocean tech and visual reasoning

Preferred qualifications:

• Previous work experience designing and implementing ML models in production environments

• Extensive experience in machine learning products, computer vision-as-a-service, serving ML models, and/or optimizing model performance

• Experience in software engineering including cloud/edge development, Unix/Linux environments, distributed and parallel systems, and data engineering

• Proficiency with PyTorch, TensorFlow, and multiple machine learning frameworks/tools

Benefits

OnDeck rewards extraordinary work with extraordinary benefits. Get unparalleled career development and exposure with ownership of critical projects. We balance our commitment to excellence with VIP events, hybrid work, travel, and days on the water. We encourage flexible vacation and schedules.

Specific benefits to help everyone work at their best include:

• Health benefits: Health spending account that covers any medical expense in your life.

• Wellness benefits: Dedicated wellness spending account to cover any additional mental or physical wellness expenses.

• Hybrid work, remote flexible: Want to work from a surf beach for a week? Go for it.

• Team lunches & happy hours, office pastries, snacks, coffees and beers. 

• Team sailing days + offsites to remote Pacific beaches & rainforests.

• Startup events and dinners such as NeurIPS and more.

• Base salary of $100,000 - $170,000