Senior Data Engineer ( Portfolio Companies: Sitecore)

Company Description

IGT 1 Outsourcing Lanka (Private) Limited, hereafter referred to as ‘IGT 1 Lanka’, is a Port City registered offshore company owned by three of the largest private equity companies, and a sister company of the largest Sri Lanka technology company, IFS.

We are committed to reinventing company success via offshore growth, expansion, diversity, and an unwavering pursuit of quality. As a leading provider of technology and employee offshore services, we help organizations all over the world navigate the complexities of the modern business environment. Our goal is to provide our customers with an operation that maximize operations, spur growth, allows them to develop and deliver world-class SaaS platforms, and create long-term value.

At IGT1 Lanka we believe that our people are the key to our collective success. We have developed a workplace culture that promotes diversity, teamwork, and ongoing education. We are presently a team of 300+ employees with a plan to double this capacity in the next 12 months.

As such, we are always on the lookout for talented individuals who share our passion for innovation and excellence. Joining IGT1 Lanka means becoming part of a forward-thinking organization that is shaping the future of business within the vibrant new Port City. Together, we can drive change, push boundaries, and build a smarter, more connected world through our offshore operation.

About Sitecore:

Sitecore delivers a composable digital experience platform that empowers the world’s smartest and largest brands to build lifelong relationships with their customers. A highly decorated industry leader, Sitecore is the leading company bringing together content, commerce, and data into one connected platform that delivers millions of digital experiences every day. Thousands of blue-chip companies including American Express, Porsche, Starbucks, L’Oréal, and Volvo Cars rely on Sitecore to provide more engaging, personalized experiences for their customers.

Job Description

About the position:

We are looking for a Senior Data Engineer to join our Data Platform group. In this position, you will work in a small, dynamic team to build data infrastructure and manage the overall data pipeline. You will be responsible for expanding and optimizing data and data pipeline architecture, as well as optimizing dataflow and collection of data from cross functional teams.

Responsibilities

• Design, develop, and maintain a generic ingestion framework capable of processing various types of data (structured, semi-structured, unstructured) from customer sources.
• Implement and optimize ETL (Extract, Transform, Load) pipelines to ensure data integrity, quality, and reliability as it flows into the centralized datastore like Elasticsearch.
• Ensure the ingestion framework is scalable, secure, efficient and capable of handling large volumes of data in real-time or batch processes.
• Continuously monitor and enhance the data ingestion process to improve performance, reduce latency, and handle new data sources and formats.
• Develop automated testing and monitoring tools to ensure the framework operates smoothly and can quickly adapt to changes in data sources or requirements.
• Provide documentation, support, and training to other team members and stakeholders on using the ingestion framework.
• Implement large-scale near real-time streaming data processing pipelines.
• Design, support and continuously enhance the project code base, continuous integration pipeline, etc.
• Build analytics tools that utilize the data pipeline to provide actionable insights into key business performance metrics.
• Perform POCs and evaluate different technologies and continue to improve the overall architecture.

Qualifications

Qualifications

• Experience building and optimizing Big Data data pipelines, architectures and data sets.
• Strong proficiency in Elasticsearch, its architecture and optimal querying of data.
• Strong analytic skills related to working with unstructured datasets.
• Experience supporting and working with cross-functional teams in a dynamic environment.
• Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data systems.
• One plus years of experience contributing to the architecture and design (architecture, design patterns, reliability and scaling) of new and current systems.
• Candidates must have 2-4 years of experience in a Data Engineer role with a Bachelors or Masters Degree in Computer Science or Information Systems or equivalent field. Candidate should have knowledge of using following technologies/tools:
  • Experience working on Big Data processing systems like Hadoop, Spark, Spark Streaming, or Flink Streaming.
  • Experience with SQL systems like Snowflake or Redshift
  • Direct, hands-on experience in two or more of these integration technologies; Java/Python, React, Golang, SQL, NoSQL (Mongo), Restful API.
  • Versed in Agile, APIs, Microservices, Containerization etc.
  • Experience with CI/CD pipeline running on GitHub, Jenkins, Docker, EKS.
  • Knowledge of at least one distributed datastores like MongoDb, DynamoDB, HBase.
  • Experience using batch scheduling frameworks like Airflow (preferred), Luigi, Azkaban etc is a plus.
  • Experience with AWS cloud services: EC2, S3, DynamoDB, Elasticsearch

Additional Information

We believe that coming together as a community, in person, is important for innovation, connection and fostering a sense of belonging. Our roles have the right balance of remote and in-office working to enable flexibility for managing your life along with ensuring a real connection with your colleagues and the broader IFS community. #Li-Hybrid