Senior Software Developer, Big Data

Position Overview 
 

Autodesk is seeking a Senior Software Developer with experience building software solutions in support of data platforms to join the Enterprise Data Engineering & Visualization (EDEV) team.  Our team is chartered with building data solutions that enable our users to seamlessly interact with the Autodesk data platform to enable ML and analytic use cases that unlock business insights.  These solutions serve as the building blocks for all data engineers, data scientists and data analysts throughout the organization.   In addition, this team establishes and promotes data engineering best practices across Autodesk, leading efforts to make data an integral part of decision-making.    

This role collaborates with Product Managers, Program/Project managers, Architects, Operations, and the Engineering Team to ensure data solutions leverage modern, scalable, and reliable methods. Communication skills are critical, and you must be comfortable distilling complex technical information to non-technical audiences as well as technical leaders. 

You will report to Senior Software Development Manager based in the US. This role is based in Toronto, in a hybrid mode. Autodesk supports FLEX working model. 

Responsibilities 

• Design, architect and implement secure and scalable data solutions enabling data scientists and analysts 

• Develop Micro-front ends and APIs that interact with each other to provide automation of complex processes 

• Leverage programming languages for data manipulation, including SQL, Python, Spark, PySpark, Spark SQL, Java 

• Apply CI/CD processes to orchestrate automated batch and streaming pipelines running PySpark and Flink  

• Design, develop, execute, and document software solutions to address complex data collection, processing, transformation, testing and publishing 

• Collaborate with peer organizations, DevOps, and Support n technical issues and help troubleshoot code level problems and performance issues 

• Recommend architectural standards, best practices and quality assurance processes for data related systems and applications 

• Build data quality and durability tracking mechanisms to provide visibility into and address changes in data ingestion, processing, and storage 

• Lead and/or help design data schemas to ensure common protocols and storage mechanisms are used across all data services 

• Partner with data and software architects to design data models, APIs or other architectural elements 

• Develop, refine, and educate the data community on coding standards and best practices. 

• Automate data transformations against data sourced from a variety of systems including desktop, web, and mobile product usage logs and business systems data 

Minimum Qualifications:

• 5 years of experience in cloud ETL, ELT and near-real time data collection, transport, and processing technologies 

• 3+ years of experience with data modeling, including the design of schemas optimized for data retrieval 

• 5+ year of experience in programming with SQL, Python, Spark, PySpark, Spark SQL, Java, Jinja, dbt and related languages and technologies 

• 3+ years of experience working with a big data environment including Hadoop, Hive, Spark and Presto 

• Experience architecting and implementing data testing solutions 

• Experience working with a big data environment including Hadoop, Hive, Spark and Presto 

• Experience with workflow management tools, like Airflow and Temporal 

• Proven expertise with algorithms, distributed systems design and the software development lifecycle 

• Experience designing and implementing RESTful APIs in Python, preferably with Flask 

• Experience deploying and maintaining cloud infrastructure, preferably on AWS 

• Experience with Infrastructure as Code frameworks, preferably Terraform 

• Experience developing CI/CD pipelines, preferably with Jenkins or Spinnaker 

• Familiarity with data governance frameworks, SDLC (Software Development Life Cycle), and Agile methodology 

• Strong technical skills and interest in learning best of class technologies around data warehousing, data wrangling, data quality, data governance and data ethics