Reconstructing the paper
The reconstruction below is the complete reading path. The exact-PDF reader is optional progressive enhancement.
What is being reconstructed
In Search of the Ingredients of Open-Endedness: Replicating Picbreeder with Large Vision-Language Models does not aim to reproduce Picbreeder’s particular archive, image representation, or genealogy. It aims to reproduce the conditions that allowed discoveries to become stepping stones for later discoveries (source-bound replication aim).
That makes the object of study a process, not a gallery: an image substrate, an evolutionary session, a shared archive, agents that choose and publish, a separate rating loop, and measurements of the archive that accumulates.
The system at a glance
| Stage | Operational question | Source boundary |
|---|---|---|
| 1. Substrate | What can mutate into what? | CPPN and NEAT substrate |
| 2. Entry | Branch from the archive or start fresh? | Session entry |
| 3. Evolution | Which parents continue through the session? | Session evolution |
| 4. Publication | Which generation-20 image receives a title and enters the archive? | Publication step |
| 5. Critic | How are archived images rated for later sampling? | Rating loop |
| 6. Evaluation | What kinds of quality and diversity did the archive exhibit? | Metric framework |
The image substrate
The recreation uses NEAT to evolve compositional pattern-producing networks. A CPPN maps spatial coordinates and radius to hue, saturation, and brightness; the evolutionary loop renders those outputs at 128 by 128 pixels (source-bound substrate claim).
This matters because the VLM does not draw pixels directly. It chooses among images produced by mutations of a generative representation.
One session, step by step
The agent either branches an archived image or starts with a fresh random CPPN population. Branching mutates the selected archived parent into an offspring population (source-bound session-entry claim).
The agent selects one or more parents from populations of 15 images; mutation and optional crossover produce the next generation (source-bound session-evolution claim).
At generation 20, the agent selects an image, gives it a title, and publishes it into the shared archive (source-bound publication claim).
The archive therefore changes the starting material available to later sessions. That feedback loop is the structural reason a single publication can become a later stepping stone.
The archive and its critic
Ratings are produced by fresh VLM instances that score samples from the collaborative archive. Separately, ten concurrent VLM workers branch from, publish to, and rate that shared archive (source-bound rating-and-workers claim).
Two “agent counts” that must not be conflated
Execution concurrency: the system runs ten concurrent VLM workers against one shared archive (worker-count source).
Experimental Number of Agents: the intervention changes the size of a personality-prompt subset. Each session draws one generated personality prompt from that subset (personality-pool source).
The sweep’s “agent count” is thus behavioral variety available to sessions, not the number of worker processes executing simultaneously.
Four instruments, not one verdict
The paper evaluates archives with Semantic Recall, Visual Coverage, Semantic Coverage, and phylogenetic Tree Balance (source-bound metric framework). The authors present these as attempts to mechanize intuitions about the human archive’s quality through recall and embedding- or tree-based diversity measures (source-bound metric qualification).
| Instrument | What it is used to inspect |
|---|---|
| Semantic Recall | Whether the archive depicts a broad vocabulary recognizably; the paper’s prose specifies a noun-to-image embedding comparison (prose specification). |
| Visual Coverage | Diversity in an image-embedding space (metric qualification). |
| Semantic Coverage | Diversity in an embedding space derived from image descriptions (metric qualification). |
| Tree Balance | Diversity in the archive’s branching structure (metric qualification). |
For this audit, we treat the measurements as complementary sensors. Agreement among them can strengthen an interpretation; disagreement is information rather than an invitation to silently collapse them into a composite score.
Three interventions
Exploration noise
Random selection can encourage exploration and reduce mode collapse, while larger amounts trade away image legibility and Semantic Recall even when coverage rises (source-bound exploration result).
Context length
The benefit of a little interaction history does not scale monotonically: context lengths 2 and 10 reduce recall, and the larger context can reinforce repeated objectives and collapse diversity (source-bound history result).
Context length is not cross-session memory. The context window never reaches before the current session’s branching step. The full-history condition also changes the prompt by adding a novelty-justification directive, so it does not isolate retained context alone (source-bound context qualification).
Personality-prompt variety
At high personality-prompt counts, coverage and Tree Balance improve without a clear recall gain. The smaller 10-prompt setting is an exception, with lower Visual Coverage, while the largest setting produces widespread high-frequency patterns that the authors interpret as likely adversarial artifacts (source-bound multi-agent result).
What the results support
The paper’s prose describes the default as context length 1, random-selection probability 0, and one agent, with 2,000 sessions repeated across six seeds (source-bound protocol claim). The grey table row and released sweep code instead identify the implemented default with zero personality traits; the audit preserves that discrepancy rather than choosing one silently (signed discrepancy).
The overall result is a set of tradeoffs. Noise, retained context, and behavioral variety each change different aspects of archive quality and diversity, and none behaves like a monotonic “more is better” control.
The paper itself calls for broader conditions, longer runs, and carefully designed human studies to calibrate the metrics and test VLM open-ended potential (source-bound future-work qualification).
What the audit keeps separate
Paper statement: what the PDF says, tied to an exact reviewed span.
Released computation: what the pinned implementation actually calculates.
Reconstruction: what the available evidence lets us recompute.
Interpretation: what we infer from those bounded results.
For the recurring implemented default, the separately signed six-run Semantic Recall and Tree Balance reconstructions match the paper’s rounded values (Semantic Recall reconstruction, Tree Balance reconstruction). Visual Coverage remains a three-seed diagnostic, while Semantic Coverage cannot be recomputed without substituting new caption embeddings (Visual Coverage status, Semantic Coverage status).
These finite, intervention-specific runs are evidence about measurable archive behavior, not proof that the system achieves unbounded open-endedness (source-bound interpretation).
Source access and privacy
The canonical source is arXiv version 2, SHA-256 679175218b73178230d04cb16290f406f1788eddf8fea6f993787797991399a7, 43,112,420 bytes, published under CC BY 4.0. You can open the canonical paper at arXiv.
The enhanced reader opens the same-origin immutable copy when that separately gated object is available. A PDF chosen from your device is hashed and opened entirely in the browser; it is never uploaded. Only the exact canonical digest is marked verified, and saving a local copy in browser storage requires separate, explicit consent.
The authored reconstruction remains the primary accessible reading path. The 43,112,420-byte enhanced PDF reader and whole-document search are desktop-first and can be limited by device memory or canvas capacity. The upstream PDF's tagging may be incomplete; the reader provides keyboard-operable controls and extracted text, but it cannot repair missing source-document structure.