ETL Developer

If you are interested kindly send the following documents to [email protected] at your earliest convenience.
Job Title: ETL Developer
Client: One of the Ministry in Ontario
Work Location: Toronto, Ontario, Hybrid
Hours per day or Week: 7.25 hours per day
Security Level: CRJMC
Must haves

• Design, Develop and Implement data pipelines using Azure Data Factory (ADF), stored procedure, and Informatica PowerCenter performing extraction, transformation, and loading activities
• Develop Azure CI/CD pipeline to automate ADF release.
• Extensive experience in design, development, and implementation with Azure Data Factory, Databricks, Informatica including - Data movement, ETL pipelines
• Experience with CI/CD (DevOps) pipelines and concepts, including Azure Resource Management, GitHub Repo, Source code version control
• Extensive experience in designing and developing SQL Server Stored Procedure, Oracle PL/SQL programs.
• Knowledge of Databases - Oracle, Microsoft SQL Server database and tools.

Description
Deliverables section:

• Scope of Services and Deliverables - The Services and Deliverables to be provided by the Vendor will include the following:
• Participate in the review of project, work request and maintenance release artefacts - Use Case, System Requirement Specification, etc.
• Deliver detailed system design documents and any other supporting documentation conforming to Ministry standards.
• Design logical and physical data models using Sybase Power Designer.
• Produce transformation mapping document.
• Implement data pipelines using Azure Data Factory (ADF), stored procedure, and Informatica PowerCenter performing extraction, transformation, and loading activities.
• Implement solutions using Logic App and Function App.
• Generate structured JSON file using ADF/ SQL.
• Develop Azure CI/CD pipeline to automate ADF release.
• Develop complex Oracle PL/SQL program to fulfill project requirements.
• Implement complex data conversion, e.g., binary to character, EBCDIC to UTF.
• Implement complex data transformation(s) for derived and calculated values.
• Integrate data sets from diverse source systems.
• Implement ADF for initial data load and incremental load.
• Promote ADF and Informatica ETL/ELT through all Ministry environments including Development, Integration Testing, QA, UAT, Production.
• Resolve and troubleshoot ADF pipeline, stored procedure, and Informatica workflow technical problems.
• Monitor and analyze incident, provide timely resolution.
• Collaborate with IT Professionals throughout the Software Development Life Cycle.
• Document information from diverse business area stakeholders and subject matter experts.
• Optimize performance of ADF pipeline, Azure SQL and Synapse databases, and Informatica workflow.
• Monitor application functionality and performance on daily basis.
• Provide effective knowledge transfer to Ministry staff at all stages of this assignment.

Experience and Skill Set Requirements
ETL Development-Data Integration - 40%

• Extensive experience in gathering requirements and business process knowledge in order to design correct and high-quality data transformation.
• Extensive experience in design, development, and implementation with Azure Data Factory, Databricks, including:
• Self-Hosted Integration Runtime
• Data movement, ETL pipelines
• Performance tuning on large data volumes
• Complex data transformations
• ADF Debugger to validate data transformation.
• Azure Roles
• Experience with Informatica, including:
• Workflow Manager
• Repository Manager
• Designer
• Workflow Monitor
• Experience with developing solutions using Logic App and Function App.
• Experience in rapid application development (RAD) methodologies.
• Experience in maintaining and improving existing ETL processes.
• Experience in investigating data to identify potential issues within ETL pipelines.
• Experience with Business Intelligence tools IBM Cognos and Microsoft PowerBI.

Azure Technologies - 40%

• Experience with CI/CD (DevOps) pipelines and concepts, including:
• Azure Resource Management
• GitHub Repo
• Source code version control
• Branching, pull requests, build, release, multiple parallel development repositories merge.
• Experience with Azure DevOps, including:
• Azure Boards
• Azure Test Plans
• Azure Pipelines
• Azure Repos
• Azure Artefacts

Oracle PL/SQL Development - 10%

• Extensive experience in designing and developing SQL Server Stored Procedure, Oracle PL/SQL programs.

Database Technologies - 10%

• Experience with Oracle, Microsoft SQL Server database and tools.
• Experience with relational and hierarchical database technologies.
• Extensive knowledge of Azure SQL DB and Synapse platforms, including:
• Performance tuning on large data volume

Note:
This position is currently listed as "Hybrid"; however, the consultant will be remote due to COVID-19 related Work from Home (WFH) direction. Once the I&IT cluster staff are required to return to the office, the resource under this request will be required to work onsite as well.

• Hybrid: 3 days onsite / 2 days remote.