Software Engineer II, Data

About the Position

FiscalNote seeks an experienced Software Engineer to join our Data Engineering team. You will help scale and improve our data processing capabilities by designing, building, and supporting services that handle one of the largest collections of policy and policymaker data, across many countries, levels of government, and commercial sectors, including large unstructured content in the form of video, audio, and text. Your role involves creating reliable systems that can provide this data in a timely and accurate way at scale, ultimately making it accessible and actionable. You will design systems that integrate and work closely with partners in applications and data science to provide our customers with an unprecedented level of automated analysis and reporting.

About the Engineering Team

The Data Engineering Team is responsible for the full lifecycle of open data in our Public Policy and Transcriptions products. We identify the datasets that we want to offer, then ingest, process, augment, store, and make them accessible to our clients and applications. We maintain the quality and usability of a rapidly growing collection of unstructured and structured data from frequently-brittle and obsolete external sources, mostly relating to policy and government action in the legislative and regulatory space. Our data powers multiple client-facing products and internal tools, and serves as the core foundation of FiscalNote’s success.

About You

You take pride in ambitious projects that push your limits, and enjoy collaborating with team members to overcome technical challenges. You are committed to seeing projects through to completion and prioritizing the best decisions for the team. Along the way, you’re capable of communicating what you’re thinking to both technical and non-technical audiences, taking in others’ ideas as well as expressing your own. You’re comfortable around ambiguity and with a high degree of autonomy, and are excited about solving complex, open-ended problems.

What To Expect In This Position

• Design, communicate, and develop solutions to complex problems
• Evaluate technical choices and their tradeoffs objectively, ensuring the best path forward for the team
• Write robust, maintainable, and readable code
• Identify opportunities to improve our codebase or reduce the complexity of our architecture
• Build data services for batch and real-time data processing
• Enhance monitoring tools for data pipelines’ health and data quality
• Create and enhance internal tools used for data review and augmentation
• Work with a full stack of technologies, including API services, databases, scheduled workers, queuing infrastructure, AI models, UI frontends, ELT pipelines, text search infrastructure, and more!

What Sets You Apart

• 3-5+ years of professional experience as an individual contributor in the software development industry, with a focus on backend or data engineering
• Strong SQL skills and understanding of relational databases (especially PostgreSQL or MySQL)
• Significant experience designing and developing within a networked architecture, using APIs and/or message-based communication
• Proficiency with a version control system, especially git
• Proven ability to both work independently and as part of a collaborative team
• Have experience with a successful working relationship with Product + UI/UX teams
• Experience working with automated test and/or build systems
• Proficiency in Python preferred
• Experience running production services on Linux preferred
• Familiarity with a Cloud computing platform (AWS, GCP, Azure, etc) preferred
• Experience with containerization tools (especially Docker, Kubernetes) is a plus
• Experience with document-based stores (NoSQL, JSONB, Elasticsearch) is a plus
• Experience with Airflow is a plus
• Experience writing and maintaining code in PHP is a plus
• Familiarity with video and audio conversion tools like ffmpeg, yt-dlp is a plus