LaMDA: Language Models for Dialogue Applications

The LaMDA proposed in the paper is a set of Transformer language models for dialogue applications, with up to 137B parameters, pretrained on 1.56T words of public dialogue data and web text. The authors state that simply scaling up model size improves quality, but yields limited improvement on safety and factual grounding; fine-tuning with annotated data and having the model call external knowledge sources such as information retrieval systems, translators, and calculators can achieve significant improvement on these two aspects. For safety, it is quantified with a set of metrics based on exemplary human values; the authors find that a LaMDA classifier fine-tuned with a small amount of crowdsourced annotated data to filter candidate responses is a promising approach to improving model safety. For factual grounding, it is quantified with the groundedness metric, making responses grounded in known sources rather than merely sounding plausible. The paper also explores LaMDA's usefulness and role consistency in education and content recommendation.