OpenAI evaluates Codex, a large language model trained on code
Evaluating Large Language Models Trained on Code
Quick read
Based on the paper's full text (AI-generated); refer to the original
OpenAI releases the Codex paper, introducing a GPT language model fine-tuned on Python code collected from 54 million public GitHub repositories, and evaluates its ability to write Python code; one production version of it powers GitHub Copilot. The paper also releases the HumanEval benchmark containing 164 hand-written programming problems, measuring functional correctness with unit tests; Codex-12B solves 28.8% of problems with a single sample, GPT-3 0%, and GPT-J 11.4%, and when sampling 100 per problem, Codex-S can solve 77.5%. On docstrings describing long chains of operations, the model's performance decreases exponentially with chain length, and it struggles to bind operations to variables. The paper also provides a harm analysis covering over-reliance, alignment failure (when prompts contain subtle errors, larger models are more inclined to follow the errors), bias, safety and economic impacts, and recommends measures such as documentation, code review and output filtering to reduce risks.
- The research examines whether large-scale language models can generate functionally correct Python function code from natural language docstrings, and evaluates their capabilities, limitations and broader impacts.
- GPT models were fine-tuned on publicly available Python code on GitHub to obtain Codex, evaluated by functional correctness (unit tests) rather than text matching; a 164-problem HumanEval dataset and the pass@k metric were built, and supervised fine-tuning was performed using competition problems and CI trace data to obtain Codex-S, while Codex-D, which generates docstrings from code, was also trained.
- Codex-12B solves 28.8% of HumanEval problems with a single sample, while GPT-3 is close to 0%; Codex-S achieves 37.7% with a single sample, and sampling 100 per problem and filtering with unit tests can reach 77.5%. Selecting samples by average log probability outperforms random selection; BLEU scores cannot reliably distinguish correct from incorrect solutions. The model shows a marked decline on long-chain operation docstrings, variable and operation binding, and system-level specifications, and will generate unsafe or biased code.
- Training sample efficiency is low, and the model is prone to errors on long docstring chains and variable bindings, and may generate unsafe, biased or user-intent-inconsistent code; the authors propose mitigation measures such as human oversight, output filtering, data screening and RLHF, and call for further research on alignment measurement, safety and economic impacts.
Beyond capability evaluation, the paper provides a harm analysis, documenting an alignment failure observation that when prompts contain subtle errors, Codex tends to follow the errors, and this becomes more pronounced as the model gets larger.