The Role
The Senior Software Architect will lead data engineering initiatives, focusing on building and managing data lakes and warehouses, utilizing cloud services and open-source technologies for seamless data integration and analytics. They will guide development teams, ensuring optimal performance, scalability, and security of the data platform while staying updated with emerging technologies.
Summary Generated by Built In
Job Title: Senior Software Architect – Cloud Data Platform
About the Role: We are seeking a seasoned Senior Architect with over 18 years of experience in technical, solutioning, and analytical roles to lead our data engineering initiatives. The ideal candidate will have extensive experience in building and managing data lakes, data warehouses, and BI/AI solutions, with a strong preference for utilizing open-source technologies alongside AWS, Azure or GCP platforms.
Key Responsibilities:
- Design and Development: Lead the design and implementation of a complex data platform for streaming real-time data from diverse environmental monitoring devices across various industries and tenants.
- Architecture Leadership: Architect scalable web applications leveraging the data platform to ensure optimal scalability, performance, and security.
- Strategic Integration: Utilize predominantly open-source technologies for data integration, processing, and analytics, integrating them with cloud-native services where necessary.
- Team Collaboration and Leadership: Collaborate with product managers, UX/UI designers, and other stakeholders to convert business requirements into robust technical solutions. Mentor and guide development teams, promoting a culture of excellence and continuous improvement.
- Innovation: Stay updated with emerging technologies and industry trends to assess their impact and integration into our projects, focusing on enhancing efficiency and performance.
Required Skills and Qualifications:
- Professional Experience: Minimum 18 years in relevant technical roles, including at least 5 years in data architecture and management.
- Education Qualification: Bachelor’s degree in computer science or related fields
Technical Expertise:
- Data Management and Integration: Expertise in data lakes, data warehouses, and complex data integrations using open-source tools such as Apache Spark, Pyspark/Scala, and Hadoop. Proficient in ETL processes and data pipelines with tools like Apache Airflow, Dagster and Kafka.
- Cloud and Database Technologies: Deep knowledge of various databases (RDS, Aurora, Oracle, Teradata, SQL Server) and integration with cloud services (AWS, Azure, GCP). Skilled in leveraging open-source databases like PostgreSQL and MySQL for data operations.
- Data Processing and Analytics: Strong experience in building data processing frameworks such as Spark SQL and Presto, and using data pipeline technologies like S3, Athena, Iceberg and Glue. Familiar with BI tools such as Tableau and Power BI.
- Machine Learning and AI: Experience with machine learning frameworks like TensorFlow and Pytorch, integrating AI into data solutions to enhance business insights.
- Cloud Architecture and Security: Understanding of cloud architecture (IaaS, PaaS, SaaS) and security protocols across cloud platforms. Ability to optimize data security and performance in cloud environments.
Other Attributes:
- Ability to provide high-level strategic thinking and fine-grained technical solutions.
- Excellent problem-solving skills and ability to work in dynamic, fast-paced environments.
- Strong communication skills and ability to act as a trusted advisor to clients and internal teams.
Top Skills
Spark
Pyspark
Scala
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Resume Uploaded Successfully
The Company
What We Do
Empowering the world through timely, trusted and actionable data through enhanced optical spectroscopy
.png)
