Senior Data Engineer (GCP)

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.
If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!

Key Responsibilities:

• Responsible to oversee the migration and modernization of the customer’s platform and database services from Google Cloud Platform (GCP) to Amazon Web Services (AWS)

• Experience in BigQuery, DataProc, DataFlow, Looker, Spanner, CloudSQL, GCS, GKE

• Lead should provide technical guidance and mentorship to the data engineering team

• Design a robust AWS architecture for the database services, utilizing services such as Amazon RDS, Amazon Redshift, Amazon Aurora, Amazon Athena and Amazon DynamoDB

• Implement migration strategies and modernization processes, ensuring compatibility, performance, and adherence to best practices

• Collaborate closely with application and infrastructure teams to facilitate seamless integration of database services

• Identifies and implements opportunities for optimization and performance improvements during the migration

• Ensure compliance with data security, governance, and regulatory requirements during the migration

Must have skills:

• Proven experience in cloud migration and modernization projects, with a strong understanding of AWS and GCP services.

• Strong understanding of cloud architecture principles and best practices

• Experience with data migration tools and strategies for AWS

• Familiarity with GCP services like Google Cloud Storage, BigQuery, Cloud SQL, and Dataflow

• Skills in designing data models for both GCP and AWS environments.

• Proficiency in ETL (Extract, Transform, Load) processes and tools, including AWS Glue and Apache Beam.

• Knowledge of both SQL and NoSQL databases

• Proficient in scripting and automation tools using Python, Bash, or similar languages to automate data transfer and to streamline migration processes

• Strong ability to work effectively with cross-functional teams

• Proficient in documenting migration processes and providing training/support to stakeholders

• Experience in project planning, including defining timelines, milestones, and deliverables

• Skilled in scoping, planning, and executing data-related projects while adhering to timelines and budgets

• Ability to convey data requirements and migration benefits to non-technical stakeholders

• Strong documentation skills for creating architectural documents and communicating complex concepts to non-technical stakeholders

• Certifications: 

  • For Lead Architects at least 2 Professional level GCP certification 

  • For Associate Architects at least 1 Professional level GCP Certification 

  • For Senior Data Engineers at least 1 Associate level GCP certification 

Good to have skills:

• Certifications: Relevant GCP Certifications (e.g., GCP Cloud Architect) are advantageous

• Knowledge of big data technologies like Hadoop, Spark and Kafka for handling large datasets