Data Engineer

Posted 7 Days Ago
Hiring Remotely in United States
Remote
Senior level
Software
The Role
The Data Engineer will design and maintain data infrastructure and pipelines, optimize performance, develop ML workflows, and ensure data governance while collaborating with various teams.
Summary Generated by Built In

Who We Are

Vividly is the leading modern trade promotion management (TPM) solution built by and for the consumer packaged goods industry. From campaign creation to deductions management and promotion measurement, we help brands manage every element of trade spend. With our easy-to-use platform and industry-leading experience, we pride ourselves on our ability to bring businesses to the next level.

Not familiar with trade promotion spend?  That’s OK.  Most people aren’t.  It’s a $8 trillion dollar problem that is ripe for disruption by an innovative SaaS startup.  In order to grow, consumer packaged goods (CPG) brands spend 20% of their topline revenue to get into retailers and promote their products within those retailers.  The problem is that today 70% of those promotions are not ROI positive.  This is because ROI is hard to track.  The industry operates on emails, pdfs, and excel spreadsheets, and other sources of unstructured data. At Vividly, we’re solving this massive problem using state of the art machine learning and AI methods to process this unstructured data and drive value for everyone in the chain, from brands to retailers to end consumers. Some of our customers include brands you may use and love - Liquid Death, Amy’s, and Laird Superfoods.

You Are

As a Data Engineer at Vividly, you will be responsible for designing, building, and maintaining the data infrastructure and pipelines that power our platform. You thrive on solving complex data challenges, ensuring that our systems are scalable, reliable, and efficient. You understand how to structure data to support analytics, reporting, and product development, working closely with software engineers, product managers, and business stakeholders to provide high-quality, actionable data. You’re passionate about data modeling, performance optimization, and automation, and you take pride in building robust ETL/ELT pipelines that support both real-time and batch processing needs.

What You Will Do

  • Design and develop scalable data pipelines to collect, process, and store large volumes of structured and unstructured data.
  • Optimize and maintain data infrastructure, ensuring reliability, efficiency, and cost-effectiveness.
  • Develop end to end machine learning workflows.
  • Work closely with product, engineering, and business teams to understand data needs and translate them into technical solutions.
  • Implement and enforce data governance best practices, including security, privacy, and compliance.
  • Develop and maintain data models that support analytics, reporting, and machine learning initiatives.
  • Monitor and troubleshoot data pipelines, proactively identifying and resolving performance bottlenecks.
  • Drive architectural decisions related to data storage, processing frameworks, and orchestration tools.
  • Ensure high data quality through validation, testing, and monitoring strategies.
  • Continuously evaluate and introduce new technologies to improve our data platform’s scalability and performance.

What We Look For

  • 5+ years of experience as a Data Engineer or in a similar role, designing, building, and maintaining scalable data infrastructure and pipelines.
  • Experience with our tech stack (GCP, Python, SQL, dbt, Airflow, BigQuery, Terraform) or a similar modern data stack.
  • Experience with machine learning libraries such as scikit-learn, tensorflow, keras, and pytorch.
  • Experience with BI tools such as Looker or Sigma.
  • Proficiency in SQL and data modeling to support analytical and operational use cases.
  • Experience with ETL/ELT pipelines, data orchestration frameworks (e.g., Airflow, Dagster), and data warehousing solutions (e.g., BigQuery, Snowflake, Redshift).
  • Familiarity with event-driven architectures and real-time data processing frameworks (e.g., Kafka, Pub/Sub, Flink).
  • Strong analytical and problem-solving skills, with the ability to debug complex data issues and optimize data pipelines for performance and cost efficiency.
  • Knowledge of data governance best practices, including security, privacy, compliance, data quality management, AI modeling, and data infrastructure.
  • Experience working cross-functionally with product, engineering, and business teams to deliver data-driven solutions.
  • Excellent communication and documentation skills, ensuring that data infrastructure and processes are well understood across teams.
  • A strong sense of ownership and initiative, with the ability to work autonomously and drive projects forward in a fast-paced environment

Additional Info

  • 100% Remote position - Open to anywhere in the USA/Canada so long as you have internet
  • Salary and equity - We base our salary bands off of current market data to ensure you’re getting compensated fairly
  • Flexible PTO - and we really mean it! We want folks to utilize time off to help prevent burn out!  
  • Healthcare, Dental and Vision - We cover 100% of your monthly premiums and 50% of all dependents
  • 401K through Betterment - Where we contribute 3% 
  • Weekly $20 Sharebite Stipend - For every meal you purchase using your Sharebite stipend, they donate a meal
  • Monthly $50 Wellness Reimbursement - physical wellness, mental wellness, learning & development - we’ve got you!
  • Monthly $200 Remote Work Stipend - Use it for your phone bill and internet
  • Vividly Days Off - For every holiday, you get an additional day off! Whenever a holiday falls on a Monday or Friday, we turn it into a 4 day weekend! That means Vividly employees have 20 days of holiday right off the bat

Vividly is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, religion, sex, sexual orientation, gender identity, national origin, age, disability, genetic information, political affiliation or belief.

Top Skills

Airflow
BigQuery
Dbt
Flink
GCP
Kafka
Keras
Looker
Pub/Sub
Python
PyTorch
Redshift
Scikit-Learn
Sigma
Snowflake
SQL
TensorFlow
Terraform
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: San Francisco, CA
99 Employees
On-site Workplace
Year Founded: 2017

What We Do

Streamline your trade promotion management, reconcile your deductions, and increase ROI with real-time forecasting and in-depth analytics. Vividly is the trade promotion partner CPG brands love

Similar Jobs

Arcadia Logo Arcadia

Data Engineer (Claims)

Big Data • Fitness • Healthtech • Software • Analytics • Energy
Remote
4 Locations
370 Employees

Capital One Logo Capital One

Distinguished Data Engineer - Capital One Software (Remote)

Fintech • Machine Learning • Payments • Software • Financial Services
Remote
Hybrid
2 Locations
55000 Employees
240K-274K Annually

Movable Ink Logo Movable Ink

Principal Data Engineer - Data Platform & Ingress

Artificial Intelligence • Marketing Tech • Software
Easy Apply
Remote
Hybrid
United States
625 Employees

Arcadia Logo Arcadia

Data Engineer (Clinical)

Big Data • Fitness • Healthtech • Software • Analytics • Energy
Remote
4 Locations
370 Employees

Similar Companies Hiring

True Anomaly Thumbnail
Software • Machine Learning • Hardware • Defense • Artificial Intelligence • Aerospace
Colorado Springs, CO
131 Employees
Caliola Engineering Thumbnail
Software • Machine Learning • Hardware • Defense • Data Privacy • App development • Aerospace
Colorado Springs, CO
53 Employees
Red 6 Thumbnail
Virtual Reality • Software • Hardware • Defense • Aerospace
Orlando, Florida
113 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account