Senior Data Engineer

Posted 5 Days Ago
Be an Early Applicant
Bengaluru, Bengaluru Urban, Karnataka
Senior level
Artificial Intelligence • Big Data • Machine Learning
The Role
The Senior Data Engineer will design, implement, and test scalable AWS-based big data ETL/ELT pipelines using PySpark and SQL. Responsibilities include working with transactional databases, data warehousing, and migrating data to the cloud while ensuring effective architecture and design best practices are followed. Knowledge of data orchestration and AWS services is essential.
Summary Generated by Built In

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.
If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!

Requirement Description:

  • Experienced in designing, implementation & testing of highly scalable and cost-efficient AWS based big data ETL/ELT pipeline using PySpark and SQL

  • Well-experienced in data warehousing

  • Well versed in working with transactional databases

Must have

  • Big data engineering technologies like Spark (PySpark)

  • Good knowledge of working with Python Programming

  • Good knowledge of advanced SQL concepts, Optimization techniques

  • Good knowledge of data warehousing, data modeling and concepts like OLAP and OLTP

  • Working knowledge with SQL databases like  Postgresql, etc. is mandatory

  • Good knowledge of Data Migration like bringing on premise database to cloud or one cloud to another

  • Understand Architecture Requirements and ensure effective design, development, validation, and support activities.

  • Understanding of data based AWS services like Glue, EMR, Lambda, DMS, RDS, S3 etc and basic AWS architecture best practices.

  • Knowledge of data pipeline orchestration using Step function or Airflow

  • Ability to use the AWS service APIs, AWS CLI, and SDKs to build applications

Good to have

  •  Knowledge of NoSQL databases like Dynamo can be add on

  • Knowledge of BI tools

  • Knowledge of working with streaming pipelines 

If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!

Top Skills

Pyspark
Python
SQL
The Company
HQ: Marlborough, MA
3,494 Employees
On-site Workplace
Year Founded: 2013

What We Do

Quantiphi is an award-winning AI-first digital engineering company driven by the desire to solve transformational problems at the heart of business.
Quantiphi solves the toughest and complex business problems by combining deep industry experience, disciplined cloud, and data-engineering practices, and cutting-edge artificial intelligence research to achieve quantifiable business impact at unprecedented speed.

Similar Jobs

Cisco Meraki Logo Cisco Meraki

Senior Data Engineer

Hardware • Information Technology • Security • Software • Cybersecurity • Conversational AI
Easy Apply
Remote
Bengaluru, Karnataka, IND
3000 Employees

Atlassian Logo Atlassian

Senior Data Engineer

Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Remote
Bengaluru, Karnataka, IND
11000 Employees

Takeda Logo Takeda

Senior Data Engineer

Healthtech • Software • Analytics • Biotech • Pharmaceutical • Manufacturing
Hybrid
Bengaluru, Karnataka, IND
50000 Employees

EchoStar Logo EchoStar

Senior Data Engineer

Aerospace • Cloud • Digital Media • Information Technology • Mobile • News + Entertainment • Retail
Bengaluru, Karnataka, IND
14500 Employees

Similar Companies Hiring

RunPod Thumbnail
Software • Infrastructure as a Service (IaaS) • Cloud • Artificial Intelligence
Charlotte, North Carolina
53 Employees
HERE Thumbnail
Software • Logistics • Internet of Things • Information Technology • Computer Vision • Automotive • Artificial Intelligence
Amsterdam, NL
6000 Employees
True Anomaly Thumbnail
Software • Machine Learning • Hardware • Defense • Artificial Intelligence • Aerospace
Colorado Springs, CO
131 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account