Andrej Karpathy

Dusting off this tweet from April because this gap in shared understanding of LLM capability is *widening*. It's now less "two groups of peo...

Cool eval. Simply ask an LLM “Land or Water?” and give it a latitude and longitude coordinate as text. Ask 16,200 times, plot as image. The ...

We'll be spending a lot more time trying to understand the outputs of language models. A few thoughts, tips & tricks: Writing. Something I'...

I love this and really hope we can come together as an industry and make it happen. https://x.com/karpathy/status/2098811935114551617/photo/...

We're starting to leave the territory where you'd test an LLM by e.g. "create an svg of pelican on a bicycle". As one idea to generalize it,...

One pattern I find useful for working with LLMs is a nice long ramble session. Sometimes the LLM needs more bits to understand what you're t...

In awe of SpaceX and its story - past, present and the future. You can think about it in 10+ different ways and continue re-blowing your min...

Personal update: I've joined Anthropic. I think the next few years at the frontier of LLMs will be especially formative. I am very excited t...

LLM Knowledge Bases Something I'm finding very useful recently: using LLMs to build personal knowledge bases for various topics of research...

- Drafted a blog post - Used an LLM to meticulously improve the argument over 4 hours. - Wow, feeling great, it’s so convincing! - Fun idea ...

When I built menugen ~1 year ago, I observed that the hardest part by far was not the code itself, it was the plethora of services, tools, d...

One common issue with personalization in all LLMs is how distracting memory seems to be for the models. A single question from 2 months ago ...

My autoresearch labs got wiped out in the oauth outage. Have to think through failovers. Intelligence brownouts will be interesting - the pl...

Human orgs are not legible, the CEO can’t see/feel/zoom in on any activity in their company, with real time stats etc. I have no doubt that ...

Three days ago I left autoresearch tuning nanochat for ~2 days on depth=12 model. It found ~20 changes that improved the validation loss. I ...

The next step for autoresearch is that it has to be asynchronously massively collaborative for agents (think: SETI@home style). The goal is ...

I packaged up the "autoresearch" project into a new self-contained minimal repo if people would like to play over the weekend. It's basicall...

nanochat now trains GPT-2 capability model in just 2 hours on a single 8XH100 node (down from ~3 hours 1 month ago). Getting a lot closer to...

It is hard to communicate how much programming has changed due to AI in the last 2 months: not gradually and over time in the "progress as u...

Bought a new Mac mini to properly tinker with claws over the weekend. The apple store person told me they are selling like hotcakes and ever...