Senior Software Engineer-Data Platform Compute

At Lyft, our mission is to improve people’s lives with the world’s best transportation. To do this, we start with our own community by creating an open, inclusive, and diverse organization.

 The Batch Compute team handles about 75M queries per week across all offline compute systems. This means there is about 150 PB of data read every week which is used to create datasets and ML features that power daily use of the Lyft platform. We are seeking a talented Software Engineer to join our team. We are in search of individuals who have experience working with large scale distributed data systems such as hive, presto, spark etc. In this role, you'll be instrumental in owning and working with these systems, designing, developing and expanding the vision of offline compute at Lyft. 

As a Software Engineer, with your technical expertise you will manage project priorities, deadlines, and deliverables. You will design, develop, test, deploy, maintain, and enhance the platform offerings. Your work will have a major impact on several areas of the business.

We are looking for candidates who are self starters and have a proven track record of delivering software solutions that can solve critical business needs. The candidate should be able to dive deep into any problems with lots of ambiguity and build a technical solution to solve it. You will have the opportunity take ownership of a project or a feature and be able to drive it from design to implementation.

Responsibilities:

• Design, develop, deploy, monitor, operate and maintain scalable and robust systems to support cross-org customers such as data engineers, data scientists, data analysts etc
• Work with technologies such as Spark, Presto, Clickhouse etc. to enhance our data platform's capabilities
• Improve the stability, reliability and performance of our platform components and tools
• Analyze our internal systems and processes and locate areas for improvement/automation
• Collaborate with product org stakeholders to address and prioritize custom edge cases
• Help lead large projects from inception to positive execution

Experience:

• 5+ years of experience building, developing, maintaining and growing large-scale infrastructure, distributed systems or networks, and/or experience with compute technologies
• BSc/MSc in Computer Engineering, Computer Science, or equivalent hands-on expertise
• Experience with distributed data solutions like Apache Spark, Apache hive etc
• Experience working with and scaling ETL solutions
• Experience working with cataloging and table format solutions like Apache iceberg, delta lake etc
• Experience working with kubernetes and container technologies (e.g. Docker, cri-o, etc)
• Familiar with a cloud-based environments such as AWS/GCP/Azure

Benefits:

• Great medical, dental, and vision insurance options with additional programs available when enrolled
• Mental health benefits
• Family building benefits
• Child care and pet benefits
• 401(k) plan to help save for your future
• In addition to 12 observed holidays, salaried team members have discretionary paid time off, hourly team members have 15 days paid time off
• 18 weeks of paid parental leave. Biological, adoptive, and foster parents are all eligible
• Subsidized commuter benefits
• Lyft Pink - Lyft team members get an exclusive opportunity to test new benefits of our Ridership Program

Lyft is an equal opportunity/affirmative action employer committed to an inclusive and diverse workplace. All qualified applicants will receive consideration for employment without regards to race, color, religion, sex, sexual orientation, gender identity, national origin, disability status, protected veteran status or any other basis prohibited by law. We also consider qualified applicants with criminal histories consistent with applicable federal, state and local law.

This role will be in-office on a hybrid schedule — Team Members will be expected to work in the office 3 days per week on Mondays, Wednesdays, and Thursdays. Additionally, hybrid roles have the flexibility to work from anywhere for up to 4 weeks per year. #Hybrid

The expected base pay range for this position in the San Francisco area is $176,000 - $220,000 USD. Salary ranges are dependent on a variety of factors, including qualifications, experience and geographic location. Range is not inclusive of potential equity offering, bonus or benefits. Your recruiter can share more information about the salary range specific to your working location and other factors during the hiring process.