Director of Data Engineering, Data and Systems

About Pendulum

Pendulum® is leading a revolution that is occurring around the world to improve physical and mental health by first understanding, then restoring and enhancing the human microbiome. 

Studies have shown that our microbiome (the bacterial communities in and on our bodies) is linked to everything from metabolism and diabetes, to longevity, weight loss, healthy immune systems, cancer prevention, feelings of well-being, inflammatory bowel disease, and even healthy skin. We have just scratched the surface on understanding the impact that our microbiome has on our lives. During early life we develop a diverse and balanced microbiome that plays a critical role in shaping our long-term health. Over our lives, a combination of diet, lifestyle, antibiotics, and aging can decrease the effectiveness of our microbiome.

Pendulum recognized the enormous impact they could have on people’s lives if they were able to address the imbalances in the microbiome. To accomplish this, Pendulum created proprietary probiotic pipelines and a unique discovery platform to identify key, novel bacterial strains and the prebiotics that feed them. The company has also built and developed the world’s first manufacturing technology to produce bacteria in an anaerobic (oxygen-free) environment at scale.

The medical probiotics that Pendulum has formulated have transformed the consumer probiotics market into a new category of therapeutic offerings that deliver the power and efficacy of a pharmaceutical with the safety and accessibility of a natural probiotic. Due to Pendulum’s explosive revenue and customer growth over the last two years, the company earned a spot on Forbes Magazine’s exclusive “The Next Billion Dollar Startups” list.

If you’re interested in improving the lives of people globally and you love working in a cross-functional, collaborative, inspiring environment, please continue reading.

Position Summary:

We are seeking a Director of Data Engineering to lead the development, optimization, and scaling of our data platform and infrastructure. This role will be pivotal in building and maintaining robust data pipelines, managing cloud-based data environments, and driving data platform innovations to support AI/ML initiatives. As a key leader, you will oversee the development of scalable, real-time data architectures, ensuring data reliability, accessibility, and compliance with data governance standards. The ideal candidate will have a strong technical foundation, proven leadership in managing data engineering teams, and the ability to align data strategies with business goals. The role will be fully hands-on with a couple members on the team, with plan to build out over the years.

What You'll Do:

• Lead, mentor, and grow a high-performing data engineering team, fostering collaboration, innovation, and continuous learning to meet evolving business needs. 




• Define and drive the data infrastructure strategy, ensuring alignment with business goals, scalability, and adaptability to support data-driven decision-making and AI/ML initiatives.




• Own the management and optimization of data warehouses and lakehouse (e.g., Snowflake), ensuring data is reliable, performant, and accessible for stakeholders. 




• Collaborate with cross-functional leaders (e.g., Data Science, Product, Analytics) to build data foundations that support machine learning, analytics, and business intelligence efforts. 




• Ensure best practices in data governance, security, and compliance (GDPR, CCPA), managing data quality, lineage, and security frameworks across all platforms. 




• Develop efficient data models and schema designs to support data integration and querying, ensuring scalability and performance across business functions. 




• Design and maintain scalable ETL pipelines using tools like dbt, Fivetran, and Airflow, with a focus on real-time processing and automation. 




• Leverage Docker, Kubernetes, and automation tools to streamline data processing workflows and ensure efficient resource utilization across the data platform. 




• Establish a robust operational framework, including monitoring, alerting, and incident management, to ensure the stability and performance of data platforms in production environments.

Knowledge Requirements:

• MSc/PhD in Computer Science or a related field. 
• 10+ years of experience building and managing data infrastructure, with expertise in data pipelines, cloud platforms, and big data technologies. 
• Expert in Python, SQL, and big data tools (Kafka, Spark, Hive/Iceberg), with significant experience in cloud platforms (AWS, GCP, Azure). 
• Proven experience with containerization (Docker, Kubernetes) and orchestration tools like Airflow to scale and optimize workflows.
• Strong understanding of data quality management, governance, and regulatory compliance, with experience in GDPR and CCPA requirements.
• Demonstrated ability to collaborate with cross-functional teams, including Data Science and ML teams, to deliver data solutions aligned with AI/ML and business objectives. 
• Leadership experience managing data engineering teams, driving strategic decisions, and adapting to the latest advancements in data technologies.

Salary & Benefits

•  $175,000-$225,000
• Medical, Dental, and Vision
• Commuter Benefits
• Life & STD Insurance
• Company match on 401 (k)
• Flexible Time Off (FTO)
• Equity