Lead Databricks Data Engineer

Posted 16 Days Ago
Saint Paul, MN
130K-195K Annually
Senior level
Biotech
The Role
Design, develop, and maintain data pipelines on Databricks while ensuring data quality, security, and compliance. Collaborate with teams to optimize performance and document processes.
Summary Generated by Built In

Ecolab is looking for a Data Engineer to be part of a dynamic team that's at the forefront of technological innovation. We're leveraging cutting-edge AI to create novel solutions that optimize operations for our clients, particularly within the restaurant industry. Our work is transforming how restaurants operate, making them more efficient and sustainable.

What’s in it For You:  

  • As a key player in our new division, you'll have the unique opportunity to shape its culture and direction.  
  • Impact the success of our innovative projects and help define the future of our product offerings 
  • Experience the best of both worlds with this team at Ecolab: the agility and creativity of a startup paired with the stability and resources of a global leader. 
  • Our collaborative environment fosters innovation while providing the support and security you need to thrive.  

  

What You Will Do: 

  • Design, develop, and maintain scalable and robust data pipelines on Databricks (Spark SQL, PySpark, Delta Lake).
  • Collaborate with data scientists and analysts to understand data requirements and deliver solutions.
  • Optimize and troubleshoot existing data pipelines for performance and reliability.
  • Ensure data quality and integrity across various data sources.
  • Implement data security and compliance best practices.
  • Monitor data pipeline performance, implement data quality checks, and conduct necessary maintenance and updates.
  • Document data pipeline processes and technical specifications.
  • Implement robust pipeline orchestration using tools like Databricks Workflows, dbt, or similar.
  • Generate and maintain data quality reports and dashboards.
  • Implement Infrastructure as Code (IaC) principles for managing Databricks infrastructure.

Minimum Qualifications 

  • Bachelor’s degree and 8 years work experience; or no degree and 12 years combined education and equivalent work experience.
  • 3 years of experience (work or educational) with a data engineering focus.
  • Proven experience in Databricks (Delta Lake, Workflows, Asset Bundles).
  • Proven experience in distributed data processing technologies (Spark SQL, PySpark).
  • Strong knowledge in designing and developing ETL pipelines.
  • Experience with data quality monitoring and reporting.
  • Experience working in a collaborative environment with data scientists and software engineers.
  • Immigration sponsorship not available for this role

Preferred Qualifications

  • Master’s degree (MS) in Computer Science or related engineering field.
  • Proficiency in Databricks (Delta Lake, Workflows, Asset Bundles).
  • Proficiency in distributed data processing technologies (Spark SQL, PySpark).
  • Experience with pipeline orchestration tools (Databricks Workflows, dbt, etc.).
  • Experience with data visualization tools (e.g., Tableau, Power BI).
  • Working experience with machine learning platforms and tools.

Experience with real-time data streaming technologies (e.g., Kafka, Kinesis

Annual or Hourly Compensation Range

The base salary range for this position is $130,100.00 - $195,100.00. This position is eligible for annual bonus pay based on performance, per plan terms. Many factors are taken into consideration when determining compensation, such as experience, education, training, geography, etc. We comply with all minimum wage and overtime laws.

Benefits 

Ecolab strives to provide comprehensive and market-competitive benefits to meet the needs of our associates and their families. Click here to see our benefits. 

If you are viewing this posting on a site other than our Ecolab Career website, view our benefits at jobs.ecolab.com/working-here. 

Potential Customer Requirements Notice

To meet customer requirements and comply with local or state regulations, applicants for certain customer-facing roles may need to:

- Undergo additional background screens and/or drug/alcohol testing for customer credentialing.

- Be fully vaccinated for COVID-19, including a booster if eligible, unless a religious or medical accommodation is requested by the applicant and approved by Ecolab.

 

Americans with Disabilities Act (ADA) 

Ecolab will provide reasonable accommodation (such as a qualified sign language interpreter or other personal assistance) with our application process upon request as required to comply with applicable laws. If you have a disability and require accommodation assistance in this application process, please visit the Recruiting Support link in the footer of each page of our career website.  

Top Skills

Databricks
Dbt
Delta Lake
Kafka
Kinesis
Power BI
Pyspark
Spark Sql
Tableau
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: St. Paul, MN
29,154 Employees
On-site Workplace

What We Do

A trusted partner at nearly three million customer locations, Ecolab (ECL) is the global leader in water, hygiene and infection prevention solutions and services. With annual sales of $12 billion and more than 44,000 associates, Ecolab delivers comprehensive solutions, data-driven insights and personalized service to advance food safety, maintain clean and safe environments, optimize water and energy use, and improve operational efficiencies and sustainability for customers in the food, healthcare, hospitality and industrial markets in more than 170 countries around the world.

For more Ecolab news and information, visit www.ecolab.com, or follow us on twitter.com/ecolab, facebook.com/ecolab or instagram.com/ecolab_inc.

Similar Jobs

JumpCloud Logo JumpCloud

Business Systems Analyst - United States

Cloud • Information Technology • Security • Software
Easy Apply
Remote
15 Locations
800 Employees
80K-95K Annually

Calm Logo Calm

Data Engineer

Healthtech • Kids + Family • Mobile • Music • Software
Hybrid
4 Locations
280 Employees

PwC Logo PwC

Data Architect- Manager

Artificial Intelligence • Professional Services • Business Intelligence • Consulting • Cybersecurity • Generative AI
Remote
Hybrid
67 Locations
370000 Employees
100K-232K Annually

PwC Logo PwC

Data Architect- Senior Associate

Artificial Intelligence • Professional Services • Business Intelligence • Consulting • Cybersecurity • Generative AI
Remote
Hybrid
67 Locations
370000 Employees
84K-202K Annually

Similar Companies Hiring

SOPHiA GENETICS Thumbnail
Software • Healthtech • Biotech • Big Data • Artificial Intelligence
Boston, MA
450 Employees
Pfizer Thumbnail
Pharmaceutical • Natural Language Processing • Machine Learning • Healthtech • Biotech • Artificial Intelligence
New York, NY
121990 Employees
Takeda Thumbnail
Software • Pharmaceutical • Manufacturing • Healthtech • Biotech • Analytics
Cambridge, MA
50000 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account