Master Thesis in Visual Representation of Technical Diagrams for LLMs - remote possible

Company Description

At Bosch, we shape the future by inventing high-quality technologies and services that spark enthusiasm and enrich people’s lives. Our promise to our associates is rock-solid: we grow together, we enjoy our work, and we inspire each other. Join in and feel the difference.

The Robert Bosch GmbH  is looking forward to your application!

Job Description

• During your master thesis, you will research optimal methods for processing and representing technical diagrams available solely as graphics for integration with Large Language Models (LLMs). You will explore the challenges and limitations of current multi-modal LLMs when handling visual data and propose novel approaches for diagram comprehension.
• You will design and implement a pipeline for creating a synthetic dataset that links technical diagrams to textual descriptions in both directions (diagram-to-text and text-to-diagram). In addition you will investigate how variations in synthetic dataset quality and structure influence model training and performance.
• Furthermore, you will conduct rigorous testing of the trained model on real-world technical diagrams from diverse domains, such as engineering, biology or software development. On top of that, you will develop metrics to assess the model's ability to interpret, explain, and generate meaningful outputs based on diagram inputs.
• Moreover, you will perform a comparative analysis of state-of-the-art multi-modal models to determine their strengths and limitations in handling technical diagram as well as identify key factors (e.g., model architecture, training data) that influence performance on diagram-related tasks.
• Finally, you will investigate the potential of an abstract markup syntax to serve as a universal intermediate representation for technical diagram as well as evaluate how such a syntax could improve model interpretability, training efficiency, and generalization across domains. You will prototype and test possible syntaxes, assessing their practicality and alignment with existing diagramming standards.

Qualifications

• Education: Master studies in Computer Science, Data Science or comparable
• Experience and Knowledge: proficiency in Python, ideally some prior experience in building RAG systems or chatbots
• Personality and Working Practice: a self-driven, proactive and solution-oriented individual with an independent work ethic
• Enthusiasm: strong passion for developing the best possible solutions
• Languages: fluent in English

Additional Information

Start: according to prior agreement
Duration: 6 months

You want to work flexibly from your home in Germany or prefer working at the Bosch location in Abstatt? For positions with the addition “remote possible”, you can agree on the appropriate collaboration for your task together with your manager and your team within the framework of Smart Work.

Requirement for this thesis is the enrollment at university. Please attach your CV, transcript of records, examination regulations and if indicated a valid work and residence permit.

Diversity and inclusion are not just trends for us but are firmly anchored in our corporate culture. Therefore, we welcome all applications, regardless of gender, age, disability, religion, ethnic origin or sexual identity.

Need further information about the job?
Simon Klug (Functional Department)
+49 174 7403108

#LI-DNI