Karpathy's tweet on ChatGPT 4.5

The tweet illustrates the vibe well.

After ChatGPT 4.5 release, the first benchmark I check is Paul Gauthier's Aider Polyglot Leaderboard. I had a feeling with such a big fuss by OpenAI but underwhelming Aider score, there should be something good in 4.5's writing creativity. There are several tweets testing its story telling and I tried it too. It seems good.

I finally found Andrej Karpathy's tweet about GPT 4.5, and I want to note several points from it:

10X more pretraining compute than GPT4

Everything is a little bit better and it's awesome, but also not exactly in ways that are trivial to point to.

That's a bit of training context and overall vibes. Here's my favorite part [emphasis's mine]:

Keep in mind that that GPT4.5 was only trained with pretraining, supervised finetuning, and RLHF, so this is not yet a reasoning model. Therefore, this model release does not push forward model capability in cases where reasoning is critical (math, code, etc.). In these cases, training with RL and gaining thinking is incredibly important and works better, even if it is on top of an older base model

HOWEVER. We do actually expect to see an improvement [by GPT 4.5] in tasks that are not reasoning heavy, and I would say those are tasks that are more EQ (as opposed to IQ) related and bottlenecked by e.g. world knowledge, creativity, analogy making, general understanding, humor, etc.

So, the models will diverge...? The creative and the reasoning. On top of that, we'll have a harmonizer. Left brain, right brain... soon, the LLM's cerebellum.