LLMs are still partly dark matter

LLM is a "dark matter": researchers can see it exists, measure its effects, map around it, and still can't explain what it actually is. 🔭

I watched a YouTube breakdown of Anthropic's research "On the Biology of a LLM". Their researchers used circuit tracing to look inside Claude 3.5 Haiku and see how the model thinks, not just what it outputs.

The interesting part is that even with attribution graphs — where you can trace why the model said what it said — big chunks of behavior stay unexplained. They call it "dark matter" and say so openly. I genuinely appreciated that level of transparency about what they don't yet understand. 🧠

When we were building our GenAI for German banking — document generation, loan agreements, all that stuff — we didn't fully know it would work either.

We had all the right preconditions.

Karpathy had already shown the direction.

But in the end, we tested the system, checked if it gave us almost the results we needed at key points. Then ran pretty chaotic experiments to close the gap. What actually helped us get there was volume — a lot of experiments and statistical analysis that told us which parameter changes genuinely moved the needle.

We were hoping more than knowing. 😅

And I think that's what AI is, across the board. You can't fully predict how it will behave — because the model's behavior is only partially understood, even by the people who built it. At least now we have the first honest map of where the fog begins. 🗺️

Do you deploy LLMs mostly believing? 🙃

Writing this post brought back memories of our chaotic experiments — and reminded me of this moment from Rosencrantz & Guildenstern Are Dead. Felt appropriate to add as a GIF. 😄