AI/ML Test Engineer

Ushur is transforming the way enterprises communicate and engage with customers. Fueled by consumer’s self-service demands, enterprises are modernizing customer engagement and experience models. Ushur is fast becoming the platform of choice for Customer Experience Automation™, enabling these enterprises to leapfrog their digital native counterparts and deliver delightful customer and employee experiences. With cutting-edge Conversational AI, Machine Learning and Intelligent Process Automation technologies, Ushur has enabled Fortune 100 enterprises including some of the world’s most well known brands in healthcare, insurance, banking and financial services sectors to automate their customer engagement. Cloud-native, 100% no-code and purely workflow-driven, Ushur empowers citizen developers within business operations teams to build AI-powered, fully-automated and omni-channel experience to digitally transform customer journeys end-to-end.About UshurIUshur XOSlUshur GenA I

The Role and Responsibilities

• Role: AI/ML Test Engineer
• Exp: 5-8 Years




• Test Planning and Strategy
• Develop and execute comprehensive test plans and test cases for AI/ML models.
• Define testing strategies that cover various aspects of model performance, including accuracy, robustness, scalability, and fairness.
• Ability to create test cases for various use cases such as document extraction, classification and conversational use cases
• Good Python programming skills to create samples for each category (classification, extraction and conversational)
• Some familiarity with annotation/ground truth creation tools to help with test case creation
• Model Validation and Verification
• Validate the correctness and performance of ML models using appropriate metrics.
• Perform rigorous testing to ensure models meet functional and non-functional requirements.
• Automation and Scripting
• Automate repetitive testing tasks and develop test scripts.
• Maintain and improve test automation frameworks.
• Data Quality and Integrity
• Ensure the quality and integrity of training and testing datasets.
• Verify data pre-processing and feature engineering pipelines.
• Performance Testing
• Conduct performance testing to assess the scalability and efficiency of AI systems.
• Monitor and analyse the runtime performance of models.
• Documentation and Reporting
• Document test cases, results, and any identified issues.
• Generate reports and provide feedback to the development team.
• Collaboration
• Work closely with data scientists, developers, and other stakeholders to understand model requirements and design.
• Provide support during the deployment and maintenance phases.
• Continuous Improvement
• Stay updated with the latest AI/ML testing tools and techniques advancements.
• Continuously improve testing processes and methodologies.

Qualifications

• Educational Background
• Bachelor’s or Master’s degree in Computer Science, Engineering, Mathematics, or a related field.
• Advanced degrees or AI/ML or Data Science certifications are advantageous.
• Technical Skills
• Proficiency in programming languages such as Python
• Experience with Natural Language Processing (NLP) and Chatbot testing
• Understanding different ML algorithms and their applications (supervised, unsupervised, reinforcement learning).Knowledge of data processing and feature engineering techniques.
• Familiarity with software testing tools and methodologies (e.g., unit testing, integration testing, regression testing).Experience with automated testing tools and frameworks.
• Understanding/exposure to LLMs, Prompt Engineering and Prompt Tuning
• Machine Learning and AI Testing
• Knowledge of model evaluation metrics and validation techniques (accuracy, precision, recall, F1-score, ROC-AUC).Ability to develop test strategies for AI/ML models, including black-box and white-box testing approaches.
• Experience with A/B testing, cross-validation, and hyperparameter tuning.
• Software Development and DevOps
• Understanding of software development lifecycle (SDLC) and agile methodologies.
• Experience with version control systems (e.g., Git).Familiarity with CI/CD pipelines and tools like Jenkins, Docker, Kubernetes.
• Analytical and Problem-Solving Skills
• Strong analytical and problem-solving skills to identify and troubleshoot issues in ML models and AI systems.
• Ability to interpret and analyse complex data sets.
• Soft Skills
• Good communication skills to collaborate with data scientists, developers, and other stakeholders.
• Attention to detail and a proactive approach to identifying potential issues.

Benefits

• Great Company Culture. We pride ourselves on having a values-based culture that is welcoming, intentional, and respectful. Our internal NPS of over 65 speaks for itself - employees recommend Ushur as a great place to work! 
• Bring your whole self to work. We are focused on building a diverse culture, with innovative ideas where you and your ideas are valued. We are a start-up and know that every person has a significant impact!
• Rest and Relaxation. 20 days of flexible leaves per year, Monthly Wellness Day (aka a day off to care for yourself) and more!
• Health Benefits. Preventive health checkups, Medical Insurance covering the dependents, wellness sessions, and health talks at the office
• Keep learning. One of our core values is Growth Mindset - we believe in lifelong learning. Certification courses are reimbursed. Ushur Community offers wide resources for our employees to learn and grow.
• Flexible Work. In-office or hybrid working model, depending on position and location. We seek to create an environment for all our employees where they can thrive in both their profession and personal life.

Why join us?

• We are passionate about Ushur, our product, and helping our employees grow and develop in their career in a caring, collaborative environment. We offer a very competitive compensation plan & stock options for the ideal candidates.