Senior Data Engineer

Posted 19 Hours Ago
Be an Early Applicant
Brazil
Senior level
Software • Analytics
The Role
As a Senior Data Engineer, you will architect and implement scalable data pipelines and infrastructures to support advanced Generative AI models. Collaborate with software engineers and product teams to design real-time processing systems for political data analysis, ensuring efficiency and security in a cloud-based environment primarily using AWS.
Summary Generated by Built In

At the intersection of innovation and influence, Quorum is pioneering the development of Generative AI technology that revolutionizes data analysis and utilization in government affairs. Our latest product, Quorum Copilot, is designed to navigate the vast expanse of political data - including 120,000 bills from 50 state legislatures, over 100,000 social media posts related to your issues, 10,000 documents, and more than 100 hours of testimony from Capitol Hill.
In today's fast-paced world, there's too much information and not enough time to process it all. Quorum Copilot aims to ensure that professionals in public affairs and policy advocacy never miss a mention or misinterpret vital information. By boosting productivity, transforming workflows, and saving time on daily activities, Quorum Copilot empowers teams to focus on what truly matters, enhancing creativity and decision-making.

As a Senior Data Engineer on the Quorum Copilot project, you will lead the effort to build and scale the data infrastructure that underpins our AI analysis tool. Working alongside a team of software engineers, product managers, and product designers you'll ensure our data systems are scalable, efficient, and primed for supporting advanced Generative AI and machine learning models at an unprecedented scale.

Key Responsibilities:

  • Architect and implement highly scalable advanced Retrieval-Augmented Generation (RAG) data pipelines to support the aggregation and analysis of legislative bills, social media posts, documents, and testimonial content.
  • Design robust data pipelines that enable real-time processing and analysis of vast datasets to inform public affairs strategies and decision-making.
  • Design and implement data cleansing and transformation pipelines to power the AI product.
  • Lead cloud-based deployments, primarily in AWS, ensuring optimal performance, security, and cost-efficiency.
  • Innovate and iterate on data architecture to support Quorum Copilot's dynamic needs, ensuring the product remains at the cutting edge of AI-driven public affairs tools.
  • Drive build vs buy, tool selection, and solution trade-off analysis using standard engineering principles.

About You

  • Bachelor's or Master's degree in Computer Science, Engineering, or related field.
  • 5+ years in data engineering with a track record of developing and scaling data-driven products
  • Expertise in building data pipelines, architectures, and datasets, with experience in AWS cloud services (EC2, EMR, RDS, Redshift).
  • Proficient in big data tools (Hadoop, Spark, Kafka) and machine learning frameworks (TensorFlow, PyTorch).
  • 3+ years experience with Python
  • Deep knowledge of SQL and NoSQL databases, workflow management tools (Azkaban, Luigi, Airflow), and an understanding of the machine learning model deployment cycle.
  • Top candidates will have experience or working knowledge of vector databases and advanced Retrieval-Augmented Generation (RAG) systems (Langchain, Pinecone, OpenAI/ChatGPT).
  • You care deeply about the code you write and either have significant technical skills solving real-world problems or the drive to quickly develop them.
  • You have an interest in learning new development languages, tools, and platforms used by the wider team.
  • You're a highly motivated self-starter who wants to make valuable contributions to a team from day 1.
  • You're an engaged learner—you want to expand your skills, get your hands dirty, and find solutions to challenging problems that require tradeoffs.
  • You want to work in a fun environment where you can form real friendships with other professionals and feel a strong sense of belonging at a growing tech company.
  • You're excited to pitch in wherever needed, from helping in recruiting to supporting customers.
  • You are looking to join a dynamic team immediately.

About the Software Development Team

  • Our fast-growing remote-first team is dedicated to improving our people, processes, and platform. The 100+ team comprises different functions, including software engineering, Product Management, Product Design, QA, DevOps/SRE, and an Information Security team.
  • We support many different products but internally think of them as one system with a single unified codebase. We aim to build centralized systems that handle complex problems for multiple features and products.
  • We’re focused on doing the important things correctly -- we write solid documentation, test our code well, design for scale, and follow defined software development processes.
  • We follow best-in-class security practices to safeguard our data and applications against bad actors. We regularly test and update processes and technology to fight complacency because we know good security is a continuous effort.
  • We curate high-quality datasets essential for effective mapping and tracking of government and public affairs spaces. Our datasets underpin our promise to give users the information necessary to make smart decisions that influence policy.
  • We set rigorous quality standards and monitor them with a mix of programmatic and manual verifications to ensure users maintain a high degree of trust in our systems.
  • We’re very close as a team and invest not only in each others’ skills and careers but also in building real relationships with one another: product development is a team sport and we believe that it’s better (and more fun) to work on a team of people that you know well and care about.

Our Work Environment
We are a remote-first team with flexible work options: work remotely, or you may choose to come into our office in Belo Horizonte.
Do you want to learn what it's like to have a real impact at a fast-growing company that is changing the way the advocacy process works? If so, drop us a line. We'd love to talk to you!

Benefits

  • Standard Brazil Holidays
  • Monthly Work from Home Stipend
  • Mental and Dental Care by SulAmérica
  • Monthly Professional Development/Mental Health stipend
  • Alelo Food Voucher
  • Incompany English classes
  • Free Subscription to the Calm app
  • Flexible Paid Time Off
  • Inclusion & Diversity Affinity Groups to support belonging
  • 12 weeks of paid parental leave

This is a CLT opportunity
For any questions regarding any open roles on our team, please reach out to [email protected].

Quorum Is Working to Advance Pay Equity: What Does That Mean For You?
In an effort to continue to build a diverse and inclusive work environment that advances pay equity, Quorum has implemented a “No Negotiation” policy for base salary for new hires for roles that are currently staffed by three or more team members. This means that candidates for the Fullstack Software Engineer II role cannot negotiate Quorum’s base salary offer.
Here’s our promise to you:
We will not ask you what you are currently earning.
We will consider years of relevant experience, relevant professional certifications/education, and performance expectations in setting what we believe is a competitive, fair base salary offer.
We will be transparent about our compensation structure (see above) so that all candidates have equal access to compensation information and can make an informed decision about whether or not Quorum is the right workplace for them.
If you are interested in learning more about how negotiation impacts pay equity and/or why other start-ups have decided to implement a “No Negotiation” policy of some type, here are a few resources: Project Include, AAUW, Ministry for Women, Magoosh.
Note: Quorum does not endorse or verify any of the information provided in the resources on the impact of negotiation on pay equity. We provide these resources simply to increase awareness of a topic we believe is important for both employers and candidates to consider.  
We comply with all requirements for US government federal contractors issued by the OFCCP, IFR, and the terms of our government contracts.
EEO/AA/F/M/Vet/Disabled

Top Skills

Python
SQL
The Company
HQ: Washington, DC
172 Employees
On-site Workplace
Year Founded: 2014

What We Do

Quorum is public affairs software that helps you work smarter and move faster.
Thousands of public affairs professionals use Quorum for their work in Congress, all 50 state legislatures, major U.S. cities, the European Union, and several additional countries. Founded in 2014, Quorum is headquartered in Washington, D.C. with an international office in Brussels.

Quorum works with Fortune 500 companies, trade associations, advocacy groups, embassies, nonprofits, and government agencies including Coca-Cola, Walmart, General Motors, U.S. Travel Association, Human Rights Campaign, the United Nations Foundation, and the Air Force.

Similar Jobs

Adyen Logo Adyen

Senior Data Engineer

Fintech • Payments • Financial Services
Easy Apply
Hybrid
São Paulo, BRA
4196 Employees
Remote
12 Locations
266 Employees

Encora Logo Encora

Senior Data Engineer

Software • Consulting
Remote
12 Locations
7456 Employees
Brazil
39 Employees

Similar Companies Hiring

InCommodities Thumbnail
Renewable Energy • Machine Learning • Information Technology • Energy • Automation • Analytics
Austin, TX
234 Employees
RunPod Thumbnail
Software • Infrastructure as a Service (IaaS) • Cloud • Artificial Intelligence
Charlotte, North Carolina
53 Employees
Hedra Thumbnail
Software • News + Entertainment • Marketing Tech • Generative AI • Enterprise Web • Digital Media • Consumer Web
San Francisco, CA
14 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account