AI
Google proposes Agile Deliberation to help define subjective image classification
Google Research introduced the Agile Deliberation framework, which uses a large-model-driven interactive process to help users gradually define subjective categories such as “healthy food” and handle borderline cases. The team observed concept evolution through 18 real-person tests and said it reduced fatigue and anxiety, outperforming conventional baselines.
Google Research research scientist Otilia Stretcu and student researcher Leijie Wang introduced Agile Deliberation. The method addresses the challenge of building AI image classifiers for subjective categories. Through an interactive deliberation process, it guides users from subdividing categories to resolving edge cases, and uses a large-model-driven optimization loop to automatically build accurate, aligned classifiers.
The article notes that traditional machine learning assumes annotators already have clear, stable category definitions before starting, which works for objective categories such as stop signs and golden retrievers. But scenarios such as healthy food, inspiring artwork, or content moderation are affected by cultural backgrounds, personal beliefs, and changes in operational policies. People usually clarify conceptual boundaries, exceptions, and nuances only after seeing real examples.
Agile Deliberation treats the creation of subjective classifiers as an iterative dialogue. Instead of having people act as static annotators, AI helps users clarify definitions. The team said that resolving human ambiguity first can more systematically build effective, robust, and aligned vision systems, and it has open-sourced the related code and algorithms on GitHub.
Rather than relying solely on static benchmarks, the study conducted 18 real-person sessions, each 1.5 hours long, to observe how user intentions changed while interacting with the intelligent system. The team said users only needed to give direct feedback on borderline images without manually adjusting prompts, significantly reducing fatigue and anxiety, while the final model was more consistent and performed better. The process consists of two stages—concept scoping and concept iteration—turning vague ideas into precise blueprints for model instructions.