ML & AI ART Test Manager & Quality Engineer At Julius Baer, we celebrate and value the individual qualities you bring, enabling you to be impactful, entrepreneurial, empowered, and to create value beyond wealth. Join our ML & AI ART as a Test Manager & Quality Engineer responsible for testing AI & ML solutions throughout their life‑cycle.
Key Responsibilities Ensure delivery quality and proper testing of AI use cases.
Define and evolve the technical testing approach and framework architecture for the ML & AI ART, aligned with the Bank’s Test Strategy and Test Policy.
Design reusable, scalable testing patterns (page objects, API clients, test data builders) for adoption across squads, ensuring technical consistency.
Analyse and evaluate requirements, features, and stories for testability during PI Planning, Backlog Refinement, and Iteration Planning.
Derive test cases from technical and risk analysis of both functional and non‑functional requirements (reliability, performance, security, usability, robustness).
Automate identified test cases using Python‑based frameworks—Playwright‑Python for UI, requests+pytest for APIs, Behave or pytest‑bdd for BDD/Gherkin—applying clean code principles, reusability, readability, and stability.
Design and implement AI/ML‑specific test cases, including evaluation pipelines for LLM outputs.
Integrate and orchestrate automated tests in GitLab CI/CD pipelines, including merge request pipelines and GitLab runner, scheduling executions across DEV, INT, UAT, pre‑PROD, regression suites, smoke tests, release executions, and on‑demand runs tied to merge requests and PI milestones.
Triage execution results, raise defects in Jira with evidence (logs, traces, screenshots, videos), and communicate quality signals to the squad and Product Owner.
Contribute actively to PI Planning, System Demos, Inspect & Adapt, and other SAFe ceremonies as part of the ML & AI ART.
Required Qualifications Hands‑on experience integrating and executing tests for AI solutions and/or large‑scale data projects.
Solid grasp of Git and version control workflows, clean code principles, and code review culture.
Working knowledge of Docker; familiarity with Kubernetes basics (jobs, namespaces).
Exposure to testing AI/ML systems or strong motivation to develop this expertise: evaluation of LLM outputs, handling non‑deterministic responses, evals for RAG and agentic workflow.
Understanding of API design, microservices, event‑driven architectures, and authentication layers.
Sound understanding of SAFe and DevOps principles; experience operating in an Agile Release Train is a plus.
Experience with Jira for story/feature tracking and test management integration (Xray, Agile Hive).
Demonstrated end‑to‑end thinking—connecting user journeys, data flows, authentication layers, and system boundaries—and collaboration with Test Managers on execution planning, reporting, and compliance.
Collaborative team player with strong ownership, taking automation problems from analysis to execution to resolution with minimal supervision.
Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
Minimum 5–7 years of experience with data & ML platforms and QA/testing, substantial hands‑on Python experience, including demonstrated framework design, ownership, and test execution at scale.
Experience in a regulated environment (financial services, healthcare, pharma) strongly preferred.
Certifications in ISTQB (Foundation) and SAFe (SP, SSM, or equivalent), or DevOps disciplines are a plus.
Strong communicator able to work effectively with engineers, Product Owners, and Scrum Masters.
Good organisational skills, structured and reliable.
Fluency in English; German is a plus.
#J-18808-Ljbffr