OpenMythos's Recurrent Loops: Do they make AI smarter at small scale?

This review evaluates an independent experiment using OpenMythos, a PyTorch reconstruction of a recurrent-depth transformer, to investigate the efficacy of recurrent loops in AI architectures on synthetic tasks.

Best for: Researchers and hobbyists exploring the theoretical implications of recurrent-depth architectures and their behavior on controlled, synthetic tasks. Skip if: You expect out-of-the-box frontier AI capabilities, or if your focus is on large-scale natural language processing without architectural experimentation. Bottom line: While recurrent loops show promise in theory, small-scale implementations like OpenMythos on simple tasks may quickly plateau, indicating complexity in scaling their benefits.

METHODOLOGY

This v0 review draws on an independent experiment published by a dev.to author on 2026-05-19, investigating the OpenMythos architecture. The author used their home AI machine, a DGX Spark, to train an OpenMythos-style mini model on synthetic multi-digit addition. The experiment specifically tested whether increasing recurrent loops at inference time (more "thinking time") improved performance. The review covers the author's setup, their comparison to existing literature (Saunshi et al. 2025, Geiping et al. 2025, Micheal Bee 2026-04), and their observed results on the synthetic task. This review does not include independent performance benchmarks, long-term workflow integration assessments, or comprehensive evaluations of edge cases. Update cadence: re-tested when claims diverge from observed behavior.

WHAT IT DOES

OpenMythos: A Hypothesis-in-Code

OpenMythos is presented as a PyTorch reconstruction of the suspected Claude Mythos architecture, developed by Kye Gomez (Swarms). It is explicitly not affiliated with Anthropic or the proprietary Claude Mythos model, which Anthropic announced on 2026-04-07 and keeps behind a limited-access coalition called Project Glasswing. Instead, OpenMythos serves as an independent, community-driven theoretical reconstruction based on publicly available research and speculation.

Recurrent-Depth Transformer (RDT)

The core architectural idea behind OpenMythos is a Recurrent-Depth Transformer (RDT). This design incorporates Mixture-of-Experts (MoE) Feed-Forward Networks (FFNs) and Multi-Query Attention (MLA)/Grouped-Query Attention (GQA) mechanisms. The model is designed to be trained from scratch on standard text data, without relying on leaked weights or distillation from proprietary models.

Investigating Recurrent Loops

The primary purpose of OpenMythos, as explored in the source experiment, is to test the efficacy of recurrent loops at inference time. The hypothesis is that giving an AI more "thinking time" through these loops might enhance its capabilities. The author's experiment specifically used OpenMythos to observe how a looped/recurrent-depth structure behaves on a small, controlled synthetic task: multi-digit addition. This setup aims to provide a data point on the architectural idea itself, separate from the broader "ASI is near" hype surrounding the original Claude Mythos.

WHAT'S INTERESTING / WHAT'S NOT

What's Interesting

The most compelling aspect of OpenMythos, and the experiment built around it, is the direct investigation into the architectural principle of recurrent depth. While the "ASI is near" rhetoric surrounding Claude Mythos is largely marketing, Kye Gomez's OpenMythos provides a concrete, open-source artifact for researchers to test a hypothesis-in-code. The author's methodical approach to adding a fourth data point to existing literature (Saunshi et al. 2025, Geiping et al. 2025, Micheal Bee 2026-04) is valuable. This experiment, conducted on a DGX Spark with a controlled synthetic task, helps to isolate the effect of recurrent loops from other confounding factors present in large-scale, natural language benchmarks. The explicit distinction between OpenMythos and Claude Mythos is also crucial, setting a pragmatic tone for architectural exploration.

What's Not

The primary "not interesting" element is the prevalent hype that conflates OpenMythos with Anthropic's proprietary Claude Mythos. The source explicitly states OpenMythos is a theoretical reconstruction, not a replication of actual weights or capabilities. This distinction is often lost in public discourse, leading to unrealistic expectations. Furthermore, the experiment's finding that recurrent loops plateau at T=2 in a small-scale setup, where the hidden state reaches a fixed-point, suggests that the benefits of "thinking time" are not universally applicable or easily scalable. This indicates that simply adding more loops is not a silver bullet for intelligence, especially without corresponding architectural or training innovations to escape these fixed points. The limited scope of a synthetic multi-digit addition task, while useful for isolation, does not provide insights into real-world performance or generalizability.

PRICING

OpenMythos is an open-source PyTorch reconstruction available on GitHub. There are no direct costs associated with the software itself. The experiment described used a "home AI machine (DGX Spark)," which represents a significant hardware investment, but no specific pricing for the hardware or associated services is provided in the source. (Pricing snapshot: 2026-05-19)

VERDICT

OpenMythos, as an architectural exploration, is best suited for researchers and hobbyists interested in the theoretical underpinnings of recurrent-depth transformers. It offers a tangible way to experiment with the concept of "thinking time" through recurrent loops. However, those expecting a direct, high-performance alternative to frontier AI models like Claude Mythos will be disappointed; OpenMythos is a hypothesis, not a product. The experiment on DGX Spark highlights that while recurrent loops can be beneficial, their effectiveness is highly task- and scale-dependent. For small-scale synthetic tasks, benefits may quickly plateau, suggesting that simply adding more loops does not guarantee increased "intelligence" without addressing the potential for hidden state fixed points.

WHAT WE'D TEST NEXT

Our next steps would involve expanding the experimental scope beyond synthetic multi-digit addition. We would test OpenMythos on a range of small-scale, but more complex, reasoning tasks to see if the T=2 plateau persists or if benefits emerge at higher loop counts. We would also investigate different initialization strategies or architectural modifications within the RDT framework that might prevent the hidden state from reaching a fixed point. Benchmarking against a fixed-depth transformer of equivalent parameter count and compute budget would clarify the true efficiency gains, if any, of the recurrent approach. Finally, we would explore the impact of different MoE and attention configurations on the recurrence behavior.