GPT-4 Technical Report

Core question: how to characterize the capabilities, limitations and safety properties of the large multimodal model GPT-4, and predict part of its performance before training. The report covers three main lines: capability evaluation, predictable scaling and safety mitigation.

• Problem setting: GPT-4 accepts image and text inputs and outputs text; the authors believe such models can be used in a large number of scenarios such as dialogue, summarization and translation, so its understanding and generation capabilities in complex scenarios need systematic evaluation.
• Shortcomings of existing approaches: the authors state that for a training run of GPT-4's scale, extensive model-specific tuning is not possible, so infrastructure and optimization methods whose behavior is predictable across multiple scales are needed.
• Core observation: the authors believe that for a well-trained large language model, the final loss can be approximated by a power law with respect to training compute; on this basis they extrapolate GPT-4's loss and HumanEval subset pass rate from models with 1,000×–10,000× less compute.
• Paper output: a technical report containing exam and benchmark evaluations, scaling prediction methods, limitation analysis (hallucination, calibration, bias), and safety mitigation (domain expert red teaming, RLHF, RBRM) and an accompanying System Card.
• Safety positioning: the authors believe GPT-4's capabilities and limitations bring new safety challenges; the accompanying System Card describes risks such as bias, disinformation, over-reliance, privacy, cybersecurity and weapons proliferation, and mitigation measures. This is not an attack or defense paper and does not define an attacker threat model.

GPT series and few-shot evaluation

• Brown et al. (2020)'s GPT-3 proposed few-shot prompting; all benchmark evaluations in this paper use few-shot prompting; Radford et al. (2018, 2019)'s GPT/GPT-2 are predecessor models.
• The authors state that GPT-4 shares limitations with earlier GPT models: not fully reliable (hallucination), limited context window, and does not learn from experience.

Scaling law and predictability

• Kaplan et al. (2020), Henighan et al. (2020), Hestness et al. (2017) and others proposed that loss follows a power law with compute; this paper fits using the form with an irreducible loss term from Henighan et al.
• The Inverse Scaling Prize (McKenzie et al., 2022) proposed tasks where performance decreases with scale; Wei et al. (2022) found that inverse scaling can become U-shaped, and the authors state that GPT-4 reverses this trend on Hindsight Neglect.

RLHF and alignment

• Christiano et al. (2017) proposed deep reinforcement learning from human preferences; Ouyang et al. (2022)'s InstructGPT used RLHF to train models to follow instructions, and the RLHF process in this paper follows such methods.
• Glaese et al. (2022) and Perez et al. (2022) are related to this paper's RBRM technique; Bai et al. (2022)'s Anthropic-LM is a comparison target on TruthfulQA.

Red teaming, risk classification and reporting format

• Perez et al. (2022) used language models to red team language models; Ganguli et al. (2022) red teamed language models to reduce harmful outputs; Weidinger et al. (2021) gave a risk classification for language models.
• Mitchell et al. (2019)'s Model Cards and Green et al. (2022)'s System Cards are the format sources for this paper's System Card.

Baselines and benchmarks: the baselines for capability comparison are GPT-3.5 and the external SOTA of each benchmark (U-PaLM, Flan-PaLM, PaLM, LLaMA, Minerva, ST-MOE, ALUM, QDGAT, CodeT, etc., see Table 2); benchmarks include MMLU, HellaSwag, ARC, WinoGrande, HumanEval, DROP, GSM-8K, TruthfulQA, RealToxicityPrompts. The authors position this paper as a characterization of GPT-4's capabilities, limitations and safety properties, rather than proposing a new model architecture.

The overall approach treats GPT-4 as a pretrained Transformer plus an RLHF post-training system, characterized along three aspects: predictable scaling, capability evaluation and safety mitigation; the method names can be summarized as predictable scaling and model-assisted safety pipeline.

Model and report scope

• GPT-4 is a Transformer-style model whose pretraining objective is to predict the next token in a document, with data from public data plus third-party licensed data, followed by RLHF fine-tuning.
• The authors explicitly do not disclose architecture (including model size), hardware, training compute, data construction or training method details, citing the competitive landscape and safety implications.

Predictable scaling

• Loss prediction: models trained with the same method with up to 10,000× less compute are used to fit a scaling law with an irreducible term L(C) = aC^b + c, where C is training compute; predictions are made shortly after training begins and use no partial results.
• Capability prediction: for a HumanEval subset, −E_P[log(pass_rate(C))] = α C^−k is fitted, where P is the subset of problems each solved at least once by every model under a given sampling budget; predictions are registered before training completes.
• Except for the 15 hardest problems, HumanEval problems are divided into 6 difficulty buckets by small-model performance and predicted separately.

Capability evaluation protocol

• Exam evaluation: simulates real exam conditions and grading; multiple-choice questions use few-shot prompting (temperature 0.3 sampling explanations then extracting the answer letter), free-response questions use temperature 0.6; methods are tuned on a validation set and final results run on held-out exams; scores and corresponding examinee percentiles are reported.
• Contamination handling: 50-character substring matching is used to detect training set contamination; for each exam a variant with contaminated questions removed is also run and the lower score reported; part of GSM-8K's training set was mixed into pretraining, and the authors suggest understanding that result as between few-shot transfer and benchmark-specific tuning.
• Benchmark evaluation: all use few-shot prompting (GSM-8K uses chain-of-thought), with the base GPT-4 model; multilingual evaluation uses Azure Translate to translate MMLU into multiple languages for 3-shot evaluation.
• User preference evaluation: 5,214 ChatGPT and API user prompts are collected, and human annotators judge under blind testing which model's response better matches user intent.

Safety mitigation process

• Expert red teaming: starting in August 2022, more than 50 external experts were recruited, covering areas such as alignment, cybersecurity, biological risk and international security, to conduct adversarial testing on GPT-4-early and mitigated versions in development.
• RBRM: a set of zero-shot GPT-4 classifiers that take a prompt (optional), the policy model output and a human-written rubric, and classify the output (e.g., desired refusal, undesired refusal, contains violating content, safe non-refusal), providing an additional reward signal in RLHF's PPO fine-tuning.
• Supporting measures: safety-related RLHF training prompts, compute-optimal RBRM weights, targeted SFT data, rewriting violating requests into borderline prompts to train against false refusals, and collecting ranking data on attempted bypass behaviors to improve robustness.
• Hallucination mitigation: in the open domain, reward models are trained on human-annotated comparison data labeled as untrue; in the closed domain, GPT-4 itself generates comparison data in multiple steps (find hallucinations, rewrite, review, repeated up to 5 times).

Experimental setup

• Models tested: GPT-4 (with vision and no vision conditions), GPT-4 base, GPT-3.5; safety metrics additionally compare text-davinci-003 and gpt-3.5-turbo; TruthfulQA additionally compares Anthropic-LM.
• Benchmarks: academic and professional exam suite (Bar Exam, LSAT, SAT, GRE, USABO, USNCO, AP, AMC, Codeforces, sommelier, Leetcode, etc.); traditional benchmarks MMLU, HellaSwag, ARC, WinoGrande, HumanEval, DROP, GSM-8K; TruthfulQA, RealToxicityPrompts; internal adversarial factuality evaluation (9 categories); internal safety evaluation (sensitive and disallowed prompts).
• Metrics: exam scores and test-taker percentiles; benchmark accuracy (F1 for DROP); incorrect behavior rate; proportion of toxic generations; calibration ECE.
• Evaluation method: free-response exam questions are scored by 1–2 third-party graders according to the official rubric; preference evaluations are blind-rated by human annotators; safety evaluations use internal classifiers plus human analysis.
• Samples and repetitions: preference evaluation uses 5,214 prompts; Codeforces uses 10 contests, each simulated 100 times; the exam held-out set is run only once; the paper does not report the number of repetitions for benchmark evaluations.
• Model snapshots: GPT-4 multiple-choice questions use the 2023-03-01 snapshot, free-response questions use the 2023-02-23 non-final snapshot, USABO uses the 2022-12-16 snapshot.

Main results

Swipe horizontally to read the table

Benchmark (condition)	GPT-4	GPT-3.5	Best external LM / SOTA
Uniform Bar Exam (Table 1)	298/400 (~90th)	213/400 (~10th)	—
MMLU 5-shot (Table 2)	86.4%	70.0%	70.7% U-PaLM / 75.2% Flan-PaLM
HellaSwag 10-shot (Table 2)	95.3%	85.5%	84.2% LLaMA / 85.6% ALUM
ARC 25-shot (Table 2)	96.3%	85.2%	85.2% PaLM / 86.5% ST-MOE
HumanEval 0-shot (Table 2)	67.0%	48.1%	26.2% PaLM / 65.8% CodeT+GPT-3.5
DROP 3-shot F1 (Table 2)	80.9	64.1	70.8 PaLM / 88.4 QDGAT
GSM-8K 5-shot CoT (Table 2)	92.0%	57.1%	58.8% Minerva / 87.3% Chinchilla etc.

• The authors state that GPT-4 surpasses existing LMs on all 7 benchmarks, and surpasses SOTA with benchmark-specific training on all benchmarks except DROP; on DROP, GPT-4's 80.9 is lower than QDGAT's 88.4.
• Counterexamples exist in the exam suite: on AMC 10, GPT-4 scores 30/150, lower than GPT-4 (no vision) and GPT-3.5's 36/150; on AP English Language and English Literature the three score the same; on Codeforces GPT-4 scores 392 (below 5th).
• The authors state that GPT-4's exam ability comes mainly from pretraining: the base and RLHF models average 73.7% and 74.0% on multiple-choice questions (Table 8).

Scaling predictions

• What was measured: Fit scaling laws with models up to 10,000× less compute to predict GPT-4's final loss on an internal codebase; in Figure 1 the power-law fit line (dashed) matches the GPT-4 observed point, and the authors call the prediction accurate.
• HumanEval: Use models up to 1,000× less compute to predict the mean log pass rate on a subset of 23 coding problems; Figure 2 shows the prediction is accurate for the 3rd easiest bucket, and the authors state the other five buckets are almost equally accurate, with the main exception being that GPT-4 is below prediction on the easiest bucket.
• Inverse scaling: In Figure 3, Hindsight Neglect accuracy decreases from ada, babbage, curie to gpt-3.5 and rises again for gpt-4; the authors state that GPT-4 reverses the inverse scaling trend.

Safety and factuality evaluations

• Safety metrics: The authors report that the tendency to respond to disallowed content requests is 82% lower than GPT-3.5, and the proportion of policy-compliant responses to sensitive requests (such as medical advice, self-harm) is 29% higher (Figure 9); on RealToxicityPrompts the toxic generation rate is 0.73% for GPT-4 and 6.48% for GPT-3.5.
• Factuality: On the internal adversarial factuality evaluation GPT-4 is 19 percentage points higher than the latest GPT-3.5 (Figure 6); on TruthfulQA the base model is only slightly better than GPT-3.5, with a clear improvement after RLHF (Figure 7, which the authors call significantly better than GPT-3.5 and Anthropic-LM).
• Negative results: The pretrained model is well calibrated (MMLU subset ECE 0.007), and calibration worsens after post-training (ECE 0.074, Figure 8); the authors state that the model still hallucinates, can make mistakes confidently, and that jailbreaks that bypass mitigations still exist (such as adversarial system messages, System Card Figure 10).

Other ablations and analyses

• Multilingual MMLU (Figure 5): GPT-4 3-shot surpasses the English scores of GPT-3.5, Chinchilla, and PaLM in 24 of 26 languages; English is 85.5%, Telugu is lowest at 62.0%.
• User preference: On 5,214 prompts, the proportion of cases where GPT-4's answer is preferred over GPT-3.5's is 70.2%; the System Card additionally reports 61.1% against GPT-3.5 Turbo RLHF.
• Contamination analysis (Table 9–11): most exams have low contamination rates and small score changes after removal; LSAT contamination is 39%, and GRE Writing and AP English FRQ contamination is 100% with no uncontaminated scores.
• ARC evaluation (System Card 2.9): GPT-4 is ineffective at autonomous replication, acquiring resources, and avoiding shutdown tasks; the authors note this is a preliminary evaluation without task-specific fine-tuning, and ARC has not yet retested with the final deployed version and its own fine-tuning.

Limitations and future directions noted by the authors

• Hallucination and reliability (Limitations section): GPT-4 still hallucinates and makes reasoning errors; the authors recommend cautious use in high-risk scenarios along with measures such as human review.
• Calibration (Limitations section): post-training reduced calibration; the authors state the model may make mistakes confidently.
• Bias (Limitations section): outputs contain various biases; the authors state that fully characterizing and managing them will take time.
• Knowledge cutoff (Limitations section): pretraining data mostly cut off in September 2021, and the model does not learn from experience.
• Residual jailbreaks (Risks & mitigations section): model-level interventions made it harder to elicit undesirable behavior, but jailbreaks that bypass mitigations still exist; the authors believe supplementary measures such as deployment-time monitoring are needed.
• Future directions (System Card Conclusion): the authors encourage research on evaluating dangerous emergent capabilities (situational awareness, persuasion, long-horizon planning), interpretability and calibration, economic impact, and public participation in governance; ARC needs the final deployed version and its own fine-tuning before it can judge the emergent-capability risk of GPT-4-launch (System Card 2.9).

Scope of experiments

• The models tested are the GPT-4 series (including base, no vision, early/launch versions) and the comparison GPT-3.5 series; external models are cited only as benchmark SOTA references.
• Vision capabilities are given only as qualitative examples (Table 3, 14–19) and preliminary results in the blog; the authors plan to release more information on vision capabilities in future work.
• The report does not contain details on architecture, model size, hardware, training compute, data construction, or training methods (Scope section).
• Safety mitigation and measurement were designed mainly from an English and US perspective; the authors note that safety mitigation in multilingual settings was not robustly tested (System Card 3.1 footnote).
• The paper does not report the number of repetitions or confidence intervals for benchmark evaluations; RLHF post-training data was not checked for contamination with TruthfulQA (footnote 9).

OpenAI's GPT-4 technical report characterizes a large multimodal model that accepts image and text input and outputs text, organized around three main threads: capabilities, predictable scaling, and safety.

• Capabilities: GPT-4 scores 298/400 on a simulated Uniform Bar Exam (about top 10%, GPT-3.5 about bottom 10%); 86.4% on MMLU 5-shot (GPT-3.5: 70.0%), surpassing existing LMs on 7 traditional benchmarks, with only DROP below the benchmark-specific SOTA (80.9 vs. 88.4); on multilingual MMLU, it exceeds the English scores of GPT-3.5 and others in 24 of 26 languages.
• Predictable scaling: using models with 1,000×–10,000× less compute to fit a power law, the authors state they accurately predicted GPT-4's final loss and pass rate on a HumanEval subset, and consider this important for safety, planning to promote it as a convention of registering predictions before training.
• Safety: red teaming by 50+ domain experts plus RLHF and RBRM processes; the authors report an 82% reduction in the tendency to respond to disallowed requests compared with GPT-3.5, a toxic generation rate of 0.73% on RealToxicityPrompts (GPT-3.5: 6.48%), and 19 percentage points higher on internal factuality evaluations.
• Limitations: the authors state the model still hallucinates, has biases, has a knowledge cutoff of September 2021, has worse calibration after post-training (ECE 0.007→0.074), and jailbreaks remain possible; ARC's preliminary assessment found that GPT-4 cannot yet autonomously replicate and acquire resources.
• Authors' conclusion: GPT-4 achieves human-level performance on some difficult professional and academic benchmarks while introducing new risks; the authors position it as a step toward broadly usable and safely deployable AI systems and emphasize that substantial work remains.