Infrastructure Engineer, Data Acquisition

Posted 14 Days Ago
Be an Early Applicant
San Francisco, CA
Mid level
Artificial Intelligence • Machine Learning • Generative AI
The Role
The Infrastructure Engineer will design, implement, and maintain scalable data pipelines and systems, improving developer productivity and service reliability in a fast-paced environment.
Summary Generated by Built In

About the DAQ Team

The Data Acquisition (DAQ) organization is responsible for building and operating the data pipelines, crawling, storage systems, and post-processing platforms that fuel OpenAI’s research and product development. Our Infrastructure team’s mission is to enable DAQ developers to move fast with minimal friction, providing “just enough” reliability, observability, and tooling in a highly dynamic environment. We value broad impact and rapid iteration over polished perfection. As our data needs grow by orders of magnitude, this small but scrappy team ensures that our foundational services can keep pace.

We operate across a wide range of infrastructure concerns, including:

  • Scaling web crawler (from modest usage to many multiples of that in the coming year)

  • Managing storage and compute for large-scale indexing, embedding, and search workloads

  • Evolving observability (metrics, logs, traces) and automation in a flexible, 80/20 manner

  • Adopting best-of-breed tooling and security from other parts of the organization (e.g., Terraform stacks, cloud platform practices)

Rather than enforcing rigid SLAs or designing monolithic infrastructure, we focus on empowering DAQ teams to build, deploy, and run their own services. Our environment can be fluid and ad hoc— if something solves the problem quickly and reliably enough, that is usually the correct approach.

About the Role

We’re looking for a hands-on Infrastructure Engineer with a strong bias toward action. You’ll be part of a small group of generalists responsible for everything from ad-hoc shell scripts to cluster provisioning automation. You will both design and implement systems: we do create architecture, but it’s rarely in the form of a lengthy design doc that goes stale—rather, we value prototyping, iterating, and shipping quickly.

In this role, you will:

  • Scale and maintain our data pipelines and compute clusters as DAQ grows by large multiples in the next year

  • Build out “just enough” observability (metrics, logs, tracing) to support developer troubleshooting and performance insights

  • Help design on-call processes for the infra we own, balancing developer velocity with service reliability

  • Collaborate directly with DAQ teams on deployment approaches, ephemeral or ad hoc workloads, network/security integrations, and more

  • Prototype and implement solutions for caching, load balancing, job scheduling, and cluster scaling, with an emphasis on iteration speed

  • Improve developer productivity by reducing friction—through better tooling, automated provisioning, and simplified environment setups

  • Adopt and integrate Infrastructure-as-Code (IaC), CI, and security best practices from other teams, tailoring them to DAQ’s dynamic needs

You might thrive in this role if you are a broad generalist who enjoys a scrappy, results‑oriented culture, can dive into anything from container orchestration to networking, and loves to unblock fellow engineers by building reliable infrastructure that supports massive growth. You’re comfortable with cloud infrastructure, automation, observability, and are eager to work in a dynamic and fast‑moving team environment.

Qualifications

  • Proven experience in an infrastructure backend role, ideally in a fast-paced environment

  • Comfort with at least one major cloud platform and its associated tooling

  • Familiarity with containerization and orchestration technologies (Kubernetes or similar)

  • Some background with IaC (e.g., Terraform, CloudFormation)

  • Hands-on experience with monitoring/observability stacks

  • Strong scripting/coding ability to build and automate solutions (your language of choice)

  • Ability to balance scrappiness and speed with robust design when needed

  • Effective communication and collaboration skills; you enjoy enabling others

We are a small team tackling big scaling challenges with lean resources—this role is pivotal to enabling the next wave of DAQ’s growth. If you’re excited by a high-impact, hands-on position where you’ll have broad autonomy and creative freedom to shape infrastructure, we’d love to hear from you!

About OpenAI

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity. 

We are an equal opportunity employer and do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, veteran status, disability or any other legally protected status. 

OpenAI Affirmative Action and Equal Employment Opportunity Policy Statement

For US Based Candidates: Pursuant to the San Francisco Fair Chance Ordinance, we will consider qualified applicants with arrest and conviction records.

We are committed to providing reasonable accommodations to applicants with disabilities, and requests can be made via this link.

OpenAI Global Applicant Privacy Policy

At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.

Top Skills

Cloud Platforms
Infrastructure-As-Code
Kubernetes
Monitoring Stacks
Scripting Languages
Terraform
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: San Francisco, CA
224 Employees
On-site Workplace
Year Founded: 2015

What We Do

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. AI is an extremely powerful tool that must be created with safety and human needs at its core. OpenAI is dedicated to putting that alignment of interests first — ahead of profit.

To achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity. Our investment in diversity, equity, and inclusion is ongoing, executed through a wide range of initiatives, and championed and supported by leadership.

At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.

Similar Jobs

Palo Alto, CA, USA
265 Employees
180K-225K Annually

Benchling Logo Benchling

Software Engineer, Procedures

Cloud • Healthtech • Social Impact • Software • Biotech
Hybrid
San Francisco, CA, USA
697 Employees

The Aerospace Corporation Logo The Aerospace Corporation

Software Tools and Assurance Department (STAD) Section Manager

Aerospace • Artificial Intelligence • Cloud • Machine Learning • Software • Cybersecurity • Defense
Hybrid
El Segundo, CA, USA
4600 Employees
148K-221K Annually

Metropolis Technologies Logo Metropolis Technologies

Senior Software Engineer, Backend

Artificial Intelligence • Computer Vision • Machine Learning • Payments • Real Estate • PropTech
Easy Apply
Los Angeles, CA, USA
23100 Employees

Similar Companies Hiring

HERE Technologies Thumbnail
Software • Logistics • Internet of Things • Information Technology • Computer Vision • Automotive • Artificial Intelligence
Amsterdam, NL
6000 Employees
True Anomaly Thumbnail
Software • Machine Learning • Hardware • Defense • Artificial Intelligence • Aerospace
Colorado Springs, CO
131 Employees
Caliola Engineering Thumbnail
Software • Machine Learning • Hardware • Defense • Data Privacy • App development • Aerospace
Colorado Springs, CO
53 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account