GPT-4 is coming!

Paper

Blog

[Official] GPT-4 is OpenAI’s most advanced system, producing safer and more useful responses

What is new in GPT-4

• Support image as input: “multimodal model which can accept image and text inputs and produce text outputs.

Image from the paper

What is not changed?

• Use text as outputs
• Transformer based architecture

Model architecture is based on Transformer, however, details are not available

“Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar.” From the paper

Performance

Pretty good on standard testing

From the paper

Not so bad on Leetcode

From the paper

Luckily, seems our programmer with enough practise can do better than GPT-4.

Which part gain most when using vision?

Seems that AMC 12 and GRE Quantitative can achieve the most significant gain when using vision. For GRE Quantitative, does this cause by there are some illustration about the problem in the test?

GPT-4 understands the key of ML

From the paper