Orion Innovation is a premier, award-winning, global business and technology services firm. Orion delivers game-changing business transformation and product development rooted in digital strategy, experience design, and engineering, with a unique combination of agility, scale, and maturity. We work with a wide range of clients across many industries including financial services, professional services, telecommunications and media, consumer products, automotive, industrial automation, professional sports and entertainment, life sciences, ecommerce, and education.
We are seeking a big data engineer for one of our clients. The successful candidate will be responsible for overseeing the creation and maintenance of our database infrastructure, including collecting and maintaining data, ensuring the integrity of our data, and creating and training data models.
Responsibilities:
Below are some of the responsibilities of a big data engineer:
- Design the architecture of our big data platform
- Build lakehouse PoC and make lakehouse technology decisions (based on our requirements)
- Build a scalable and performant data model satisfying our existing use cases and future plans using object storage (S3-compatible)
- Build code architecture and deployment infrastructure for data pipelines development using orchestrator (Airflow) and Spark
- Prepare documentation, knowledge transfer sessions and run books for data lake and data platform
- Mentor engineering team during work process
- Maintain our data pipeline and data lake platform
- Customize and oversee integration tools, warehouses, databases, and analytical systems
- Configure and provide availability for data-access tools
Job Qualifications and Skill Set:
Below are the qualifications expected of a big data engineer:
- 3 to 5 years of relevant data engineering experience
- Experience designing data model which utilizes object (S3-like) storage and used for massive parallel processing using Spark
- Experience running lake house technologies (DeltaLake, Iceberg) in production on big datasets (TB of data)
- Experience with pipeline orchestrators (Airflow/Dagster).
- Experience writing and operating Spark pipelines at scale (hundreds of cores, TBs or RAM)
- Experience with JVM based languages (Kotlin would be a plus)
- Experience with Python
- Excellent command of English both verbal and written
Orion is an equal opportunity employer, and all qualified applicants will receive consideration for employment without regard to race, color, creed, religion, sex, sexual orientation, gender identity or expression, pregnancy, age, national origin, citizenship status, disability status, genetic information, protected veteran status, or any other characteristic protected by law.
Candidate Privacy Policy
Orion Systems Integrators, LLC and its subsidiaries and its affiliates (collectively, “Orion,” “we” or “us”) are committed to protecting your privacy. This Candidate Privacy Policy (orioninc.com) (“Notice”) explains:
- What information we collect during our application and recruitment process and why we collect it;
- How we handle that information; and
- How to access and update that information.
Your use of Orion services is governed by any applicable terms in this notice and our general Privacy Policy.
Top Skills
What We Do
Orion is a leading digital transformation and product development services firm. Headquartered in Edison, NJ, we have a global team of 6,200+ associates, with engineers in 14 major delivery centers across North America, Europe, Asia Pacific and Latin America.
For over 25 years, Orion has been solving complex business problems for our clients. Our transformative business solutions are rooted in digital strategy, experience design, and engineering, empowering our clients to operate with agility at scale.
Our mission is to serve as an agile and trusted partner for business transformation initiatives, providing deep emerging technology, experience design, and domain expertise.
Our business has more than tripled over the last three years.
We have grown aggressively both organically and inorganically, adding new clients, complementary skills, domain expertise, and strengthening our global footprint.
