Machine Learning Modeler, Model Risk Management

Company
Block is one company built from many blocks, all united by the same purpose of economic empowerment. The blocks that form our foundational teams - People, Finance, Counsel, Hardware, Information Security, Platform Infrastructure Engineering, and more - provide support and guidance at the corporate level. They work across business groups and around the globe, spanning time zones and disciplines to develop inclusive People policies, forecast finances, give legal counsel, safeguard systems, nurture new initiatives, and more. Every challenge creates possibilities, and we need different perspectives to see them all. Bring yours to Block.
Team
The Model Risk Management team ensures our machine learning models are safe, reliable, and compliant with regulatory requirements. We focus on building automated validation tools while maintaining high standards for model assessment. Our work helps prevent errors and bias while making sure models are used appropriately across Block. We work closely with teams throughout the company to establish practical governance frameworks, assess model performance, and share best practices. As a core part of the compliance organization, we support all key areas of Block regardless of product or market.
The Role
We're looking for a Machine Learning Modeler to help us make our model validation process both more efficient and more thorough. You'll be validating a diverse portfolio of machine learning models that support Block's financial services, with current focus on:

• Production-scale gradient boosted decision trees for financial decisioning systems
• Hybrid ML systems that leverage LLMs in the training data pipeline
• Emerging applications of LLMs in financial services
• Additional ML models supporting Block's growing suite of financial products

You'll split your time between hands-on model validation work and developing tools to automate these processes. Your experience validating ML models will be key in building practical solutions that help our team work more effectively.
A key aspect of the role will be developing validation frameworks that can handle both traditional ML models (like XGBoost) and newer AI technologies (like LLMs), ensuring robust governance and reliable performance across different model architectures and use cases.
We're looking for someone with strong technical skills in both machine learning and software engineering. You should be detail-oriented and familiar with relevant regulations and industry standards. Clear communication and the ability to work both independently and as part of a team are essential.
You Will
Primary Focus:

• Build scalable validation frameworks for tree-based models, focusing on feature importance analysis, stability, and performance metrics
• Develop validation approaches for hybrid systems where LLMs support the ML pipeline
• Create governance frameworks for LLM applications, including:
  • Reliability and consistency assessment methodologies
  • Prompt engineering validation approaches
  • Output quality control mechanisms
  • Drift detection for both traditional ML and LLM components
• Design testing frameworks that can adapt to different model types and use cases
• Create tools that help generate clear validation reports
• Set up systems to continuously monitor model performance
• Build tools that make model validation faster and more consistent
• Create validation components we can reuse across different projects
• Develop automated approaches for common tasks like:
  • Checking model performance
  • Running statistical tests
  • Verifying data quality
  • Testing model assumptions
  • Tracking performance changes over time

Immediate Responsibilities:

• Assess machine learning models using rigorous validation methodologies
• Develop automated validation tools that can scale across different model types
• Set up comprehensive model monitoring systems
• Maintain clear validation documentation aligned with regulatory requirements
• Partner with model development teams to understand validation needs
• Build effective relationships while maintaining independent assessment standards

You Have
Required:

• Advanced degree in Computer Science, Machine Learning, or related quantitative field
• 5+ years experience in model validation or risk management, with focus on machine learning models; or 3+ years and a graduate degree
• Strong software engineering practices and experience building maintainable, well-documented code
• Strong understanding of tree-based models, particularly gradient boosted decision trees and XGBoost
• Understanding of LLM capabilities, limitations, and validation requirements
• Experience with or strong understanding of:
  • Feature importance analysis for tree-based models
  • Model interpretability techniques
  • Validation of training data quality, especially in cases of automated labeling
  • Performance metric selection and validation for different model types
  • Model governance in financial services
• Expertise in Python for building robust validation frameworks and automation tools
• Advanced SQL skills for data analysis and validation automation
• Experience with test automation and software testing frameworks
• Strong quantitative skills with the ability to identify patterns in validation processes
• Experience building modular, reusable code and tools
• High ethical standards with a commitment to integrity and professionalism

Preferred:

• Experience developing internal tools or validation frameworks
• Knowledge of software development best practices (version control, unit testing, CI/CD)
• Experience validating models in regulated domains
• Knowledge of model governance platforms and frameworks
• Experience with model inventory management systems
• Knowledge of emerging technology validation approaches
• Experience with data visualization tools (e.g., Looker) for monitoring and reporting

Technical Focus Areas:
Key Validation Considerations:

• Model architecture and implementation review
• Feature engineering and data pipeline validation
• Performance assessment across different use cases
• Validation of automated labeling pipelines
• Assessment of model generalization
• Evaluation of model refresh requirements and drift detection
• Governance frameworks for hybrid ML systems
• Testing methodologies for LLM-based components
• Compliance with regulatory requirements for model risk management

Why Block?
At Block, you'll be at the forefront of transforming how we approach model risk management through automation. Your work will not only ensure the reliability and safety of our machine learning systems but will also create lasting impact by building the tools and frameworks that will shape the future of model validation. This is a unique opportunity to combine technical expertise in machine learning, validation, and software development to create innovative solutions in the fintech space.
We value diversity of perspectives because they fuel creativity and innovation. We offer competitive compensation packages, benefits, and perks, along with opportunities for career growth and professional development. As a Data Scientist in Model Risk Management at Block, you will be part of a team that is committed to providing reliable and accurate decision-making support while minimizing risk and delivering long-term value to the organization and its stakeholders.
If you are passionate about using your quantitative and technical skills to make an impact on economic empowerment while building the future of model validation, join us at Block!
We're working to build a more inclusive economy where our customers have equal access to opportunity, and we strive to live by these same values in building our workplace. Block is an equal opportunity employer evaluating all employees and job applicants without regard to identity or any legally protected class. We also consider qualified applicants with criminal histories for employment on our team, and always assess candidates on an individualized basis.
We believe in being fair, and are committed to an inclusive interview experience, including providing reasonable accommodations to disabled applicants throughout the recruitment process. We encourage applicants to share any needed accommodations with their recruiter, who will treat these requests as confidentially as possible. Want to learn more about what we're doing to build a workplace that is fair and square? Check out our I+D page .
Block will consider qualified applicants with arrest or conviction records for employment in accordance with state and local laws and "fair chance" ordinances.
Block takes a market-based approach to pay, and pay may vary depending on your location. U.S. locations are categorized into one of four zones based on a cost of labor index for that geographic area. The successful candidate's starting pay will be determined based on job-related skills, experience, qualifications, work location, and market conditions. These ranges may be modified in the future.
To find a location's zone designation, please refer to this resource . If a location of interest is not listed, please speak with a recruiter for additional information.
Zone A:
$228,700 - $343,100 USD
Zone B:
$217,300 - $325,900 USD
Zone C:
$205,900 - $308,900 USD
Zone D:
$194,500 - $291,700 USD
Every benefit we offer is designed with one goal: empowering you to do the best work of your career while building the life you want. Remote work, medical insurance, flexible time off, retirement savings plans, and modern family planning are just some of our offering. Check out our other benefits at Block.
Block, Inc. (NYSE: XYZ) builds technology to increase access to the global economy. Each of our brands unlocks different aspects of the economy for more people. Square makes commerce and financial services accessible to sellers. Cash App is the easy way to spend, send, and store money. Afterpay is transforming the way customers manage their spending over time. TIDAL is a music platform that empowers artists to thrive as entrepreneurs. Bitkey is a simple self-custody wallet built for bitcoin. Proto is a suite of bitcoin mining products and services. Together, we're helping build a financial system that is open to everyone.