Minimum qualifications:
* Bachelor's degree or equivalent practical experience
* 8 years of experience working with Large Language Models (LLMs) and building agents.
Preferred qualifications:
* Understanding of core machine learning (ML) concepts, training algorithms, best practices for evaluation.
* Understanding of data, familiarity with basic statistical analysis concepts (e.g., variance, p-value, bias), and data science classics.
* Understanding of essential concepts such as tokens, context, Retrieval-Augmented Generation (RAG) and function calling.
* Understanding of core ML concepts, training algorithms, best practices for evaluation.
* Familiarity with running quality interactions, which involves understanding business goals, defining aligned technical metrics, implementing evaluation frameworks, designing experiments, analyzing results, performing RCAs, formulating hypotheses for improvements, and conducting ablation or live experiments.
About the job
Google Cloud's software engineers develop the next-generation technologies that change how billions of users connect, explore, and interact with information and one another. We're looking for engineers who bring fresh ideas from all areas, including information retrieval, distributed computing, large-scale system design, networking and data storage, security, artificial intelligence, natural language processing, UI design and mobile; the list goes on and is growing every day. As a software engineer, you will work on a specific project critical to Google Cloud's needs with opportunities to switch teams and projects as you and our fast-paced business grow and evolve. You will anticipate our customer needs and be empowered to act like an owner, take action and innovate. We need our engineers to be versatile, display leadership qualities and be enthusiastic to take on new problems across the full-stack as we continue to push technology forward.
In this role you will be responsible for quality of next-generation AI agents. Your work will bridge the gap between raw model power and deployed agentic systems through data engineering and systems optimization. You will run quality iterations for entity management (CRUD), information retrieval (IR), and conversation naturalness/robustness. Beyond model evaluation, you will drive the evolution of context engineering solutions. This involves optimizing Retrieval-Augmented Generation (RAG) and agentic constructs to enhance system robustness and help customers deploy agents with confidence. You will architect the data pipelines necessary to ingest, process, and analyze vast evaluation datasets.
The Google Cloud AI Research team addresses AI challenges motivated by Google Cloud's mission of bringing AI to tech, healthcare, finance, retail and many other industries. We work on a range of unique problems focused on research topics that maximize scientific and real-world impact, aiming to push the state-of-the-art in AI and share findings with the broader research community. We also collaborate with product teams to bring innovations to real-world impact that benefits our customers.
Responsibilities
* Create high-quality datasets for evaluation and training to improve model performance for cloud customers.
* Design new agentic systems and context engineering algorithms, utilizing existing models and identifying need for new training/evaluation objectives where necessary.
* Define and implement metrics that correspond to business problems.
* Prototype and iterate on the solution working closely with customers, product management and business development.
* Design and implement evaluation frameworks and tooling for dataset creation and curation.