From Biophysical to Functional: Two Generations of Neural Digital Twins
The first generation of neural digital twins was biophysical. The Blue Brain Project (EPFL, 2005–2024) reconstructed cortical microcircuits at morphological and biophysical detail — individual neurons with reconstructed dendrites, calibrated ion channels, stochastic synapses. Validation meant checking 40+ experimental constraints: layer-specific firing rates, connection probabilities, orientation selectivity indices. The framework that systematized this validation was DMT (Data, Models, Tests), developed 2017–2024 and published in eLife.
The second generation is functional. Brain-Score (DiCarlo lab, MIT) asks a different question: not “does this circuit reproduce biophysics?” but “does this model represent visual information the way primate cortex does?” The answer is measured by regressing model activations against neural recordings (PLS regression, ceiling-normalized). The models are deep networks, not compartmental simulations. The data are population responses in V4 and IT, not single-cell traces.
Both generations need systematic validation. Both produce structured scientific reports. The methodology is the same; the domain changes.
DMT as the Bridge
DMT-Eval rebuilds the proven BBP validation methodology for the functional era. The architectural insight — decoupling analyses from models through formal interfaces — transfers directly:
At BBP: An interface specified what a circuit model must provide (e.g., layer-specific cell densities). An adapter wrapped a NEURON simulation to expose those quantities. A validation test compared model output to experimental reference data and produced a structured report.
At Brain-Score: An interface specifies what a vision model must provide (activations at a given layer for a set of stimuli). An adapter wraps a PyTorch model to expose those activations. A benchmark compares model representations to neural recordings and produces a score.
The pattern is identical. DMT-Eval provides the scaffolding: @implements
validates compliance at registration time, PluginRegistry manages
per-interface registries, and the LabReport pipeline renders results as
scientific documents.
Platform Mastery: The Brain-Score Tutorial Series
Five tutorials (codev/08–12) reverse-engineer the Brain-Score stack:
- Installation and first score — AlexNet on MajajHong2015.IT-pls: raw r = 0.48, ceiling = 0.817, normalized = 0.588.
- Plugin architecture — how Brain-Score discovers models and benchmarks via entry points and YAML manifests.
- Model commitment — the
BrainModelprotocol:start_recording(),look_at(),commit(). - Benchmark internals — ceiling estimation, cross-validation splits,
the
BenchmarkBaseclass. - Data assemblies — the NeuroidAssembly data structure, stimulus sets, and the packaging pipeline.
Three Architectural Fixes
The original DMT (BBP era) used InterfaceMeta — a custom metaclass that
caused MRO conflicts when interfaces needed to compose. The Brain-Score
domain adapter fixes three problems:
- Metaclass to
__init_subclass__: No MRO conflicts, simpler composition. - Late failure to early enforcement:
@implementsvalidates at registration, not at benchmark runtime. - Shared registry to per-interface: The original had a single mutable
__implementation_registry__across all interfaces. Fixed with per-interfacePluginRegistryinstances.
The EU AI Act Connection
The EU AI Act (2024) requires “appropriate levels of… validation” for high-risk AI systems. Neural digital twins — whether biophysical models used in drug discovery or functional models used in brain-computer interfaces — will need systematic validation pipelines. DMT provides the methodology: structured argumentation, traceable from data through model to verdict, rendered as a scientific document.
What’s Next
DMT applied to the new generation of neural digital twins:
- Ganguli’s group (Stanford): Statistical mechanics of deep networks meeting neuroscience. DMT can validate functional similarity claims.
- Mathis lab (EPFL): Motor control models benchmarked against primate kinematics. The Scenario pattern fits directly.
- Brain-Score expansion: New modalities (audition, language), new brain regions, new species. Each needs the same validation discipline.
The live service at bench.mayalucia.dev demonstrates the framework in action — weather prediction, drug efficacy, and Brain-Score evaluations all rendered through the same LabReport pipeline.