May 6, 2025
Schaun Wheeler

JEPA and Semantic Memory: Why Current AI Models Fall Short

May 6, 2025
Schaun Wheeler

JEPA and Semantic Memory: Why Current AI Models Fall Short

May 6, 2025
Schaun Wheeler

JEPA and Semantic Memory: Why Current AI Models Fall Short

May 6, 2025
Schaun Wheeler

JEPA and Semantic Memory: Why Current AI Models Fall Short

I wrote a posts recently on the need for agentic systems to have semantic-associative memory capabilities. Someone asked whether something like LeCun’s Joint Embedding Predictive Architecture (JEPA) gives us semantic memory.

The answer: not really, but it does potentially do many other good things.

JEPA, as I understand it, trains a model to predict latent representations of future or missing input. That pushes the system to learn abstract features, not just reconstruct pixels or tokens. It's a meaningful step away from shallow pattern matching and toward conceptual understanding.

But semantic-associative memory — the kind humans use to form, refine, and reuse concepts — needs more. To mention just a few items:

  • Persistent structure. Human memory stores and stabilizes concepts over time; JEPA’s representations are ephemeral unless paired with external memory.

  • Explicit associations. Human concepts are richly linked (by function, causality, hierarchy). JEPA relies on proximity in embedding space, which is statistically shallow by comparison.

  • Learning dynamics. Humans consolidate and reinforce categories through use and salience. JEPA lacks this selective strengthening over time.

JEPA gives us abstraction without accumulation. It could be foundational for building systems with real semantic-associative memory, but it's not really sufficient to do so in its current formulation.

0

Related

Shaping the future of marketing with Aampe through innovation, data.

Jun 10, 2025

Schaun Wheeler

Explore how agentic systems define and execute decisions. This article delves into the five key decision types—Go/No-Go, Context, Creative Policy, Item Recommendation, and Freshness—that guide autonomous agents in delivering personalized user experiences. Learn how these systems prioritize meaningful choices to enhance engagement and effectiveness.

Jun 10, 2025

Schaun Wheeler

Explore how agentic systems define and execute decisions. This article delves into the five key decision types—Go/No-Go, Context, Creative Policy, Item Recommendation, and Freshness—that guide autonomous agents in delivering personalized user experiences. Learn how these systems prioritize meaningful choices to enhance engagement and effectiveness.

Jun 10, 2025

Schaun Wheeler

Explore how agentic systems define and execute decisions. This article delves into the five key decision types—Go/No-Go, Context, Creative Policy, Item Recommendation, and Freshness—that guide autonomous agents in delivering personalized user experiences. Learn how these systems prioritize meaningful choices to enhance engagement and effectiveness.

Jun 10, 2025

Schaun Wheeler

Explore how agentic systems define and execute decisions. This article delves into the five key decision types—Go/No-Go, Context, Creative Policy, Item Recommendation, and Freshness—that guide autonomous agents in delivering personalized user experiences. Learn how these systems prioritize meaningful choices to enhance engagement and effectiveness.

Jun 5, 2025

Schaun Wheeler

Explore Aampe's innovative approach to personalization, combining classical recommender systems for item selection with real-time reinforcement learning agents for dynamic message composition. Learn how this dual-path strategy enhances user engagement and content relevance.

Jun 5, 2025

Schaun Wheeler

Explore Aampe's innovative approach to personalization, combining classical recommender systems for item selection with real-time reinforcement learning agents for dynamic message composition. Learn how this dual-path strategy enhances user engagement and content relevance.

Jun 5, 2025

Schaun Wheeler

Explore Aampe's innovative approach to personalization, combining classical recommender systems for item selection with real-time reinforcement learning agents for dynamic message composition. Learn how this dual-path strategy enhances user engagement and content relevance.

Jun 5, 2025

Schaun Wheeler

Explore Aampe's innovative approach to personalization, combining classical recommender systems for item selection with real-time reinforcement learning agents for dynamic message composition. Learn how this dual-path strategy enhances user engagement and content relevance.

Jun 3, 2025

Schaun Wheeler

Explore why focusing solely on short-term metrics like lift can be misleading when assessing adaptive systems, and discover alternative approaches for meaningful evaluation.

Jun 3, 2025

Schaun Wheeler

Explore why focusing solely on short-term metrics like lift can be misleading when assessing adaptive systems, and discover alternative approaches for meaningful evaluation.

Jun 3, 2025

Schaun Wheeler

Explore why focusing solely on short-term metrics like lift can be misleading when assessing adaptive systems, and discover alternative approaches for meaningful evaluation.

Jun 3, 2025

Schaun Wheeler

Explore why focusing solely on short-term metrics like lift can be misleading when assessing adaptive systems, and discover alternative approaches for meaningful evaluation.

Jun 2, 2025

Schaun Wheeler

A fresh perspective on customer engagement using multi-armed bandits and real-time decision-making, moving beyond static journey planning.

Jun 2, 2025

Schaun Wheeler

A fresh perspective on customer engagement using multi-armed bandits and real-time decision-making, moving beyond static journey planning.

Jun 2, 2025

Schaun Wheeler

A fresh perspective on customer engagement using multi-armed bandits and real-time decision-making, moving beyond static journey planning.

Jun 2, 2025

Schaun Wheeler

A fresh perspective on customer engagement using multi-armed bandits and real-time decision-making, moving beyond static journey planning.

Load More

Load More

Load More

Load More