Senior Data Engineer

Job Description

We are seeking a senior data engineer with hands-on experience in building and optimising data pipelines, working with large-scale data processing and implementing cloud-based solutions. The ideal candidate has expertise in modern data platforms and a proven ability to develop robust, scalable and secure data workflows. This role is central to our mission of delivering data-driven solutions that create business impact for our clients.

Your key responsibilities will include:

• Designing and building data pipelines: Leading the development and optimisation of ETL/ELT workflows, ensuring seamless data integration and efficient processing across diverse data sources.
• Cloud-based data engineering: Designing, deploying and managing data platforms in cloud environments such as Azure, AWS or GCP, leveraging cloud-native tools and best practices.
• Data modelling and architecture: Developing and maintaining scalable data architectures, creating robust data models and ensuring high performance and reliability for analytics and reporting systems.
• Data quality and governance: Implementing data validation processes, monitoring pipeline health and enforcing data governance policies to ensure accuracy, consistency and compliance.
• Automation and orchestration: Utilising workflow orchestration tools like Airflow, Prefect or Azure Data Factory to automate data workflows and optimise processing.
• Collaboration with cross-functional teams: Partnering with data scientists, analysts and business stakeholders to understand data needs and deliver solutions that align with organisational objectives.
• Performance tuning and optimisation: Analysing and optimising data pipelines for speed, scalability and cost efficiency, ensuring the system meets SLAs for performance and availability.
• Mentorship and team development: Guiding junior engineers, sharing best practices and fostering a culture of continuous learning and innovation within the data engineering team.

Key requirements are:

• Educational background: A master’s degree in computer science, engineering, economics or a related field.
• Experience: A minimum of 3–5 years in data engineering, with proven experience in designing and building data pipelines in professional environments.
• Technical expertise:
  • Proficiency in SQL and at least one programming language such as Python, Java or Scala.
  • Hands-on experience with cloud platforms (Azure, AWS, GCP) and their data tools (e.g., Azure Synapse, AWS Glue, Google BigQuery).
  • Experience with modern data processing frameworks like Apache Spark, Databricks or Snowflake.
  • Knowledge of data lake and data warehouse concepts, including Delta Lake and other lakehouse architectures.
  • Familiarity with CI/CD pipelines and infrastructure-as-code tools like Terraform or CloudFormation.
• Problem-solving and analytical skills: Ability to design solutions for complex data challenges with a focus on scalability and maintainability.
• Communication skills: Strong written and verbal communication skills to explain technical concepts to technical and non-technical stakeholders.
• Leadership and mentorship: Demonstrated ability to lead projects and mentor junior team members, driving excellence in data engineering practices.
• Certifications: Relevant certifications in cloud platforms (e.g., Azure Data Engineer, AWS Certified Data Analytics or GCP Professional Data Engineer) are a plus.

We will be conducting interviews on an ongoing basis and encourage you to apply as soon as possible, as applications will close once the position is filled.

We look forward to hearing from you!

About us

The Tech Collective is a fast-growing consulting company powered by Implement Consulting Group. We help organisations unlock the immense potential of data, AI and analytics, ensuring that they are fit for the digital future. Through co-creation and passionate advisory, we aim to make our clients’ data easy to handle and provide useful insights. In our collective, you will find people with a mix of nerdiness and infectious energy. We are passionate about technology, but we also have a genuine interest in people.