Thoughts on AI Today – Why Persistence Matters

Thoughts on AI Today – Why Persistence Matters

**By Dave Hibbitts June 2025**

Artificial Intelligence has come a long way in a short time. We’ve seen models generate text, code, images, and even simulate conversation. But as impressive as these capabilities are, I believe we’re still missing something fundamental on the path to true intelligence.

Too much of today’s AI thinking is driven by resets.

Most large models are trained, fine-tuned, then deployed — and when the next version comes, we do it all over again. Even during training, vast amounts of edge-case data are removed to keep things “clean.” And once deployed, models forget everything they experience unless it’s explicitly fed back into the system.

This isn’t how humans learn.

We accumulate. We adapt. We remember the rare events, the mistakes, the edge cases. Our knowledge grows over time. Intelligence, in any meaningful form, isn’t episodic — it’s persistent.

What AI Needs Next

  • Memory that persists across sessions
  • Learning architectures that evolve over time
  • Access to diverse, even messy, data
  • A shift away from optimization-only thinking

Scaling laws (Kaplan et al., 2020) have taught us that bigger models and more data work — but there are limits. The next breakthroughs in AI will come not from size alone, but from structural changes in how systems retain and build on knowledge.

Why Founder-Model Reuse Isn’t Enough

While reusing base (founder) models like GPT, LLaMA, and Claude gives us a starting point, it’s also becoming a crutch. Repeating and fine-tuning on the same underlying weights leads to diminishing returns. These models are optimized for general performance — not for chaos, conflict, anomaly, or nuance.

I believe true AGI will not emerge from optimizing past success. It will emerge from learning through difference, failure, contradiction, and randomness.

We need to start thinking of chaos as a feature of intelligence — not a bug.

Learning Brick by Brick

I’m currently studying foundation model architecture from the ground up — brick by brick. I believe we need to understand not just how these systems behave, but how they’re built.

This process is helping me think critically about how we train, what we keep, and what we throw away. And it’s becoming clear: we’re still throwing away too much to achive zero one shot, responses.

Final Thought

AI today is powerful — but still narrow. To move forward, we must treat intelligence not just as computation, but as something that grows. Like a mind.

Let’s stop resetting.
Let’s start remembering.
Let’s embrace chaos.

In my humble opinion, this may sound unconventional — but think about how a toddler learns. They don’t just observe. They touch, taste, feel. They explore the world through mess, sensation, and trial.

Current models don’t do that. They see data patterns and adjust weights — but without experience.

For example: does a foundation model really understand the sea? Or a beach?

It can recognize a photo. It can predict text about “waves crashing” or “sand underfoot.” But these are abstractions, not experiences. The model builds a probabilistic illusion — a delusion model — based on data correlations, not sensory reality.

What’s missing is the chaos:

  • The roar of surf in the ears
  • The salt sting on lips
  • The unpredictability of motion and sound and texture

These are millions of additional parameters that no current model truly engages with.

In cognitive science, this idea aligns with embodied cognition: that real intelligence isn’t learned from labels — it’s learned from the body interacting with the world.

Until AI can grapple with this messy, embodied input — not just vision and language — its intelligence will remain confined.

📚 References & Further Reading

Written on June 15, 2025