Opportunities and risks of foundation models
On the Opportunities and Risks of Foundation Models
Quick read
Based on the paper's full text (AI-generated); refer to the original
Stanford University's Center for Research on Foundation Models (CRFM) released a report systematically reviewing the opportunities and risks of foundation models, summarizing this paradigm with the two words emergence and homogenization. The report defines foundation models as models trained on broad data that can be adapted to many downstream tasks, citing BERT, GPT-3 and CLIP as examples, and notes that GPT-3 has 175B parameters. The authors argue that homogenization, while bringing high leverage, creates single points of failure, as defects of foundation models are inherited by all downstream adapted models, while emergent capabilities make it difficult to understand how they work, the conditions under which they fail, and the boundaries of their capabilities. The report is divided into four parts—capabilities, applications, technology, and society—covering language, vision, robotics and reasoning; healthcare, law and education; and safety and alignment, security and privacy, interpretability, fairness, misuse, environment and law. The authors also point out that research in this field is almost entirely concentrated in industry, and that academic participation is limited by rising barriers to compute and data, and call for social and ethical considerations to be brought forward to the model-building stage.
- AI is undergoing a paradigm shift driven by models such as BERT, GPT-3, and DALL-E that are trained self-supervised on large-scale broad data and can be adapted to many downstream tasks, which the authors call foundation models. The scale of such models brings emergent capabilities, and their cross-task effectiveness drives homogenization, but it is not yet clear how they work, when they fail, or what they can do, and their flaws are inherited by all downstream adapted models.
- The report takes the form of a review plus interdisciplinary discussion, organized around the two keywords emergence and homogenization, divided into four parts—capabilities, applications, technology, and society—with 26 sections in total, surveying capabilities such as language, vision, robotics, reasoning, and interaction; applications such as medicine, law, and education; technical topics such as modeling, training, adaptation, evaluation, systems, data, safety, robustness, theory, and interpretability; and societal impacts such as fairness, misuse, the environment, legitimacy, economics, and ethics of scale.
- The report acknowledges itself to be incomplete, not covering applications in natural sciences, music, finance, agriculture, etc., nor discussing connections with neuroscience, cognitive science, and psychology; the authors emphasize that understanding of foundation models remains very limited, and their emergent properties bring uncertainty about capabilities and flaws, requiring deep interdisciplinary collaboration commensurate with the sociotechnical nature of such models.
The report proposes the term foundation model and links capability, application and safety issues along two threads, emergence and homogenization, and can serve as a conceptual groundwork for understanding subsequent discussions of foundation model risk.