Engineer - Big Data

Posted Yesterday
Be an Early Applicant
Gurgaon, Gurugram, Haryana
3-5 Years Experience
eCommerce • Retail
The Role
The Big Data Engineer will design and implement scalable architecture for data processing, utilizing expertise in SQL, Pyspark, and Python with FastAPI. The role involves managing big data solutions like Spark and Hive, orchestrating workflows with Airflow, and ensuring code quality and testing. Additionally, familiarity with cloud databases and CI/CD practices is advantageous. Strong problem-solving and communication skills are crucial for effective collaboration.
Summary Generated by Built In

dunnhumby is the global leader in Customer Data Science, empowering businesses everywhere to compete and thrive in the modern data-driven economy. We always put the Customer First.

 

Our mission: to enable businesses to grow and reimagine themselves by becoming advocates and champions for their Customers. With deep heritage and expertise in retail – one of the world’s most competitive markets, with a deluge of multi-dimensional data – dunnhumby today enables businesses all over the world, across industries, to be Customer First.

 

dunnhumby employs nearly 2,500 experts in offices throughout Europe, Asia, Africa, and the Americas working for transformative, iconic brands such as Tesco, Coca-Cola, Meijer, Procter & Gamble and Metro.

Most companies try to meet expectations, dunnhumby exists to defy them. Using big data, deep expertise and AI-driven platforms to decode the 21st century human experience – then redefine it in meaningful and surprising ways that put customers first. Across digital, mobile and retail. For brands like Tesco, Coca-Cola, Procter & Gamble and PepsiCo.

Preferrable 3-6 years of experience as Big data engineer backend Engineer

  • Prior experience with scalable architecture and distributed processing
  • Strong Programming expertise in  SQL, Pyspark
  • Strong programming skills in Python, with experience in FastAPI or any API framework
  • Proficiency in unit testing and code quality assurance.
  • Expertise on major big data solutions like Spark,  Hive
  • Experience in big data orchestrator like Airflow
  • Good hands-on experience on GIT or other code versioning tools
  • Experience on cloud-based data stores like Redshift and Big Query is an advantage.
  • Experience in cloud solution architecture preferrable GCP and Azure.
  • Excellent problem-solving and communication skills.
  • Ability to work effectively in a collaborative team environment.

Nice To Have

  • Good experience on microservices architecture, docker and Kubernetes as orchestration.
  • ML ops and ML experience using Spark
  • Understanding of logging, monitoring, and alerting of production big data pipelines
  • Familiarity with GitLab CI/CD is a plus.

What you can expect from us

 We won’t just meet your expectations. We’ll defy them. So you’ll enjoy the comprehensive rewards package you’d expect from a leading technology company. But also, a degree of personal flexibility you might not expect.  Plus, thoughtful perks, like flexible working hours and your birthday off.

You’ll also benefit from an investment in cutting-edge technology that reflects our global ambition. But with a nimble, small-business feel that gives you the freedom to play, experiment and learn.

And we don’t just talk about diversity and inclusion. We live it every day – with thriving networks including dh Gender Equality Network, dh Proud, dh Family, dh One and dh Thrive as the living proof. Everyone’s invited.


Our approach to Flexible Working

At dunnhumby, we value and respect difference and are committed to building an inclusive culture by creating an environment where you can balance a successful career with your commitments and interests outside of work.

We believe that you will do your best at work if you have a work / life balance. Some roles lend themselves to flexible options more than others, so if this is important to you please raise this with your recruiter, as we are open to discussing agile working opportunities during the hiring process.

For further information about how we collect and use your personal information please see our Privacy Notice which can be found (here)

What you can expect from us

We won’t just meet your expectations. We’ll defy them. So you’ll enjoy the comprehensive rewards package you’d expect from a leading technology company. But also, a degree of personal flexibility you might not expect.  Plus, thoughtful perks, like flexible working hours and your birthday off.

You’ll also benefit from an investment in cutting-edge technology that reflects our global ambition. But with a nimble, small-business feel that gives you the freedom to play, experiment and learn.

And we don’t just talk about diversity and inclusion. We live it every day – with thriving networks including dh Gender Equality Network, dh Proud, dh Family, dh One and dh Thrive as the living proof.  We want everyone to have the opportunity to shine and perform at your best throughout our recruitment process. Please let us know how we can make this process work best for you. For an informal and confidential chat please contact [email protected] to discuss how we can meet your needs. 

Our approach to Flexible Working

At dunnhumby, we value and respect difference and are committed to building an inclusive culture by creating an environment where you can balance a successful career with your commitments and interests outside of work.

We believe that you will do your best at work if you have a work / life balance. Some roles lend themselves to flexible options more than others, so if this is important to you please raise this with your recruiter, as we are open to discussing agile working opportunities during the hiring process.

For further information about how we collect and use your personal information please see our Privacy Notice which can be found (here)

Top Skills

Python
The Company
Chicago, IL
0 Employees
Hybrid Workplace
Year Founded: 1989

What We Do

Dunnhumby is a customer data platform that provides models and insights into how customers engage with retail and e-commerce spaces.

Jobs at Similar Companies

Optimum Logo Optimum

Lead IT Compliance

AdTech • Digital Media • Internet of Things • Marketing Tech • Mobile • Retail • Software
Hybrid
Bethpage, NY, USA
9000 Employees
123K-203K Annually

Spokeo Logo Spokeo

Account Executive, Spokeo for Business

Big Data • Consumer Web • eCommerce • Enterprise Web • Software • Analytics • Big Data Analytics
Remote
United States
160 Employees

Grocery TV Logo Grocery TV

District Manager - Des Moines, IA

AdTech • Digital Media • Hardware • Marketing Tech • Retail • Software
Easy Apply
Remote
Des Moines, IA, USA
45 Employees

Similar Companies Hiring

Spokeo Thumbnail
Software • Enterprise Web • eCommerce • Consumer Web • Big Data Analytics • Big Data • Analytics
Pasadena, CA
160 Employees
Optimum Thumbnail
Software • Retail • Mobile • Marketing Tech • Internet of Things • Digital Media • AdTech
Long Island City, NY
9000 Employees
Grocery TV Thumbnail
Software • Retail • Marketing Tech • Hardware • Digital Media • AdTech
Austin, TX
45 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account