Lead Data Engineer

More than a career - a chance to make a difference in people's lives.

Build an exciting, rewarding career with us – help us make a difference for millions of people every day. Consider joining the Duke Energy team, where you'll find a friendly work environment, opportunities for growth and development, recognition for your work, and competitive pay and benefits.

Job Summary:

This position is responsible for development and consulting activities related to analytics within various Duke Energy IT and business units.

• Support or collaborate with application developers, database architects, data analysts and data scientists to ensure optimal data delivery architecture throughout ongoing projects/operations.

• Design, build, and manage analytics infrastructure that can be utilized by data analysts, data scientists, and non-technical data consumers, which enables functions of the big data platform for Analytics.

• Develop, construct, test, and maintain architectures, such as databases and large-scale processing systems that help analyze and process data in the way the Analytics organization requires.

• Develop highly scalable data management interfaces, as well as software components by employing programming languages and tools.

• Work closely with data subject matter experts to determine what data management systems are appropriate and data scientists to determine which data is needed.

• Work with stakeholders to understand the information needs and translate these into technical solutions.

• Work closely with a team of Data Science staff to take existing or new models and convert them into scalable analytical solutions.

• Design, document, build, test and deploy data pipelines that assemble large complex datasets from various sources and integrate them into a unified view.

• Identify, design, and implement operational improvements: automating manual processes, data quality checks, error handling and recovery, re-designing infrastructure as needed.

• Create data models that will allow analytics and business teams to derive insights about customer behaviors

• Build new data pipelines, identify existing data gaps and provide automated solutions to deliver analytical capabilities and enriched data to applications.

• Responsible for obtaining data from the System of Record and establishing batch or real-time data feed to provide analysis in an automated fashion.

• Develop techniques supporting trending and analytic decision making processes

• Apply technologies for responsive front-end experience

• Ensure systems meet business requirements and industry practices

• Research opportunities for data acquisition and new uses for existing data

• Develop data set processes for data modeling, mining and production

• Integrate data management technologies and software engineering tools into existing structures

• Employ a variety of languages and tools (e.g. scripting languages) for integration

• Recommend ways to improve data reliability, efficiency and quality

Responsibilities:

• Develop components required for optimal extraction, transformation, and loading of data from a wide variety of data sources using a set of tools or languages (e.g. SQL)

• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. Implement big data analytics solutions, as well as integrate new data management technologies and software engineering tools into existing structures

• Work with stakeholders to assist with data-related technical issues and support their data infrastructure needs. Solve problems for Stakeholder, construct problem frames in order to understand the analysis context and scope that will provide timely, useful results

• Create data tools for analytics and data scientist team members that assist them in building and optimizing products. Work in Project Teams and participate in multidisciplinary analytics project teams, translating complex functional and technical requirements into detailed architecture, design, and high performing software/technical components

• Work with data and analytics experts to strive for greater functionality in data systems. Partner with business clients, establish professional relationship, and communicate with analytics clients in order to understand business needs.

• Employees in this role will also:
  • Translate complex functional and technical requirements into detailed architecture, design, and high performing software.
  • Drive user ideation analysis and elaboration, design and development of software applications, and build automation tools.
  • Select data solution software and define hardware requirements.
  • Help build a next-generation Big Data analytics framework with a set of core technologies.
  • Code, test, and document new or modified data systems to create robust and scalable applications for data analytics.
  • Ensure all automated processes preserve data by managing the alignment of data availability and integration processes.
  • Collaborate with data architects, modelers and IT team members on project goals.
  • Assist with extracting standard sets of features from important datasets that will be leveraged by Data Science teams.
  • Convert code from applications.
  • Build processes supporting data transformation, data structures, metadata, dependency and workload management.

Basic/Required Qualifications:

• Masters Degree in Business Administration, Managing Information Strategies (MIS), Mathematics, Computer Science, or Engineering

• Minimum 5 Years Required Related Experience

• In lieu of Masters degree(s) AND 5 year(s) related work experience listed above, Bachelors Degree AND 8 year(s) related work experience

Preferred Qualifications:

• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.

• Excellent verbal and written communication skills

• Self-starter who works with minimal supervision and the ability to work in a team of diverse skill sets

• Ability to comprehend customer requests and provide the correct solution

• Strong analytical mind to help take on complicated problems

• Desire to resolve issues and drive into potential issues

• Familiarity with the Agile Methodology Strong analytic skills related to working with unstructured datasets.

• Excellent verbal and written communication skills.

• Experience supporting and working with cross-functional teams in a dynamic environment.

• Experience with big data tools: Hadoop, Spark, Kafka, etc.

• Experience with relational SQL and NoSQL databases, including SQL Server and Cassandra.

• Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.

• Understanding of distributed computing principles.

Working Conditions:

• Hybrid Mobility Classification – Work will be performed from both remote and onsite locations after the onboarding period. However, hybrid employees should live within a reasonable daily commute to a Duke Energy facility.

Travel Requirements

5-15%

Relocation Assistance Provided (as applicable)No

Represented/Union PositionNo

Visa Sponsored PositionNo

Posting Expiration Date

Tuesday, November 26, 2024

All job postings expire at 12:01 AM on the posting expiration date.

Please note that in order to be considered for this position, you must possess all of the basic/required qualifications.

Privacy

Do Not Sell My Personal Information (CA)

Terms of Use

Accessibility