Big Data Engineer

Posted Yesterday
Edison, NJ
Junior
Software
The Role
The Big Data Engineer will design and implement big data pipelines, develop Hadoop batch jobs, and ensure data processing and performance optimization. Responsibilities also include client interaction for architectural requirements, and contributing to the setup of automated build processes using Jenkins and Ansible.
Summary Generated by Built In

Orion Innovation is a premier, award-winning, global business and technology services firm.  Orion delivers game-changing business transformation and product development rooted in digital strategy, experience design, and engineering, with a unique combination of agility, scale, and maturity.  We work with a wide range of clients across many industries including financial services, professional services, telecommunications and media, consumer products, automotive, industrial automation, professional sports and entertainment, life sciences, ecommerce, and education.

Job Responsibilities:

  • Requirement analysis, requirement gathering, design, business impact analysis, gap analysis, estimation, development, testing, coding, code review, unit testing and deployment of the application.
  • Design, build, install, configure, and support Hadoop (Big Data). 
  • Design and implement big data pipelines to ingest and process data in real-time and monitor for data miss or error in progress to ensure that data reaches the end system.
  •   Measure the performance of various APIs and implement optimization on slow services to enhance the responsiveness of the system. 
  • Deliver functional enhancements to the existing big data applications. 
  • Provide inputs on solution architecture based on evaluation/understanding of solution alternatives, frameworks, and products. 
  • Develop Hadoop batch jobs for data extraction from multiple unstructured and structured data sources for populating into various repositories (Hadoop, Neo4J, MongoDB, Apache SOLR). 
  • Interact with clients to elicit architectural and non-functional requirements such as performance, scalability, reliability, availability, maintainability. 
  • Participate in designing the data model structured data in RDBMS into a graph database solution and NoSQL database solution. 
  • Develop near real-time data processing solutions using Kafka and Spark Streaming. 
  • Participate in designing Spark architecture with Databricks and Structured Streaming.
  • Set up Microsoft Azure with Databricks, and Databricks Workspace for business analytics. 
  • Contribute to the automate build process by using Jenkins and Ansible to achieve continuous integration and continuous deployment.

Requirements:

Master’s in Computer Science/Applications, Information Technology/Systems or Electronics/Electrical Engineering + minimum 1 year experience as Big Data Engineer, Data Engineer, Data Engineering Specialist, Data Warehousing Analyst or related occupation.

Must be willing to travel/relocate to anywhere in the US.

Edison, NJ and unanticipated locations throughout US


Orion is an equal opportunity employer, and all qualified applicants will receive consideration for employment without regard to race, color, creed, religion, sex, sexual orientation, gender identity or expression, pregnancy, age, national origin, citizenship status, disability status, genetic information, protected veteran status, or any other characteristic protected by law.

Candidate Privacy Policy

Orion Systems Integrators, LLC and its subsidiaries and its affiliates (collectively, “Orion,” “we” or “us”) are committed to protecting your privacy. This Candidate Privacy Policy (orioninc.com) (“Notice”) explains:

  • What information we collect during our application and recruitment process and why we collect it;
  • How we handle that information; and
  • How to access and update that information.

Your use of Orion services is governed by any applicable terms in this notice and our general Privacy Policy.


Top Skills

Hadoop
Kafka
Spark
The Company
HQ: Edison, NJ
3,410 Employees
On-site Workplace
Year Founded: 1993

What We Do

Orion is a leading digital transformation and product development services firm. Headquartered in Edison, NJ, we have a global team of 6,200+ associates, with engineers in 14 major delivery centers across North America, Europe, Asia Pacific and Latin America.

For over 25 years, Orion has been solving complex business problems for our clients. Our transformative business solutions are rooted in digital strategy, experience design, and engineering, empowering our clients to operate with agility at scale.​

Our mission is to serve as an agile and trusted partner for business transformation initiatives, providing deep emerging technology, experience design, and domain expertise.​

Our business has more than tripled over the last three years. ​

We have grown aggressively both organically and inorganically, adding new clients, complementary skills, domain expertise, and strengthening our global footprint.

Similar Jobs

MetLife Logo MetLife

Stable Value Controller - Sr Analyst

Fintech • Information Technology • Insurance • Financial Services • Big Data Analytics
Hybrid
Bridgewater, NJ, USA
43000 Employees

PwC Logo PwC

AI & GenAI Data Scientist-Manager

Artificial Intelligence • Professional Services • Business Intelligence • Consulting • Cybersecurity • Generative AI
Hybrid
Florham Park, NJ, USA
364000 Employees
100K-232K Annually

PwC Logo PwC

Cloud Data & Analytics Manager (Financial Services-Insurance)

Artificial Intelligence • Professional Services • Business Intelligence • Consulting • Cybersecurity • Generative AI
Remote
Hybrid
Florham Park, NJ, USA
364000 Employees
100K-232K Annually
Hybrid
Jersey City, NJ, USA
289097 Employees

Similar Companies Hiring

TrainingPeaks (A Peaksware Company) Thumbnail
Software • Fitness
Louisville, CO
69 Employees
bet365 Thumbnail
Software • Gaming • eSports • Digital Media • Automation
Denver, Colorado
6100 Employees
Jobba Trade Technologies, Inc. Thumbnail
Software • Professional Services • Productivity • Information Technology • Cloud
Chicago, IL
45 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account