What "AI-ready genomics" actually means

Google DeepMind, Google.org and the Wellcome Sanger Institute announced a five-year, $25M consortium yesterday to build genomic datasets "designed for AI from the start." That phrase is doing a lot of work, and most of the coverage I've read skips the part that matters. Here is what an "AI-ready" genomic dataset actually is, why almost none of the data sitting in genomics archives today qualifies, and what the consortium is really trying to fix.

The short version: the bottleneck in genomics machine learning is not model size. It is that the datasets we have were built for a different job, and the seams show up the moment you try to train on them.

Why existing genomic data is hard to learn from

To see the problem, it helps to know what "a genomic dataset" usually is. A research group recruits, say, 10,000 people. They take blood samples, sequence each person's DNA, record some phenotype information (does this person have type 2 diabetes, what is their cholesterol level, what medications are they on), and store the lot. Multiply this by every hospital, biobank, and research consortium on earth and you get the current landscape: a patchwork of cohorts, each assembled for a specific scientific question, each with its own protocols.

For classical statistical genetics, finding which DNA variants are associated with a disease, this patchwork is workable. You can run the same statistical test cohort by cohort and combine the results. But for training a model that learns directly from raw DNA sequence, the patchwork creates four specific problems.

Batch effects. Different sequencing machines, run in different labs, in different years, produce subtly different data. The same stretch of DNA, sequenced twice, will have slightly different error patterns depending on which Illumina machine ran it and which version of the chemistry was used. A model trained across cohorts will learn these lab signatures as eagerly as it learns biology — and when you deploy it on a new cohort, the lab signature is gone and the model's predictions degrade. This is the genomic version of the "tank classifier" parable.

Inconsistent variant calling. Raw sequencing output is not "the genome." It is a pile of short reads that have to be aligned to a reference genome and processed by a variant-calling pipeline to produce a list of differences between this person's DNA and the reference. Different pipelines (GATK, DeepVariant, others) produce subtly different variant calls. Two cohorts processed with different pipelines look like they disagree about biology when they actually disagree about software.

Missing or inconsistent phenotype labels. This is the big one. If you want a model to learn the link between DNA and disease, you need labels — and labels in genomics are messy. One cohort records "diabetes: yes/no." Another records "HbA1c level, measured on this date." A third has electronic health records with ICD-10 codes that may or may not have been entered correctly. The labels are not wrong, exactly, but they are incommensurable.

Consent that doesn't permit ML reuse. Participants in older studies consented to specific research questions. Using their data to train a foundation model that will then be used for other purposes is, depending on the consent form, somewhere between awkward and forbidden.

What "designed for AI from the start" changes

Now imagine you are building a new cohort, and you know from day one that the data will be used to train ML models. What do you do differently?

You standardise the wet lab. Every sample is sequenced on the same generation of machine, using the same chemistry, processed by the same variant-calling pipeline, with quality-control thresholds chosen once and applied uniformly. Batch effects don't disappear, they never disappear entirely, but they become a known, characterised noise floor rather than a confound.

You design the phenotype schema as a labelling schema. Instead of recording whatever the recruiting clinician happened to write down, you decide in advance what labels you want the model to learn, and you collect them consistently. Continuous measurements are taken with the same instrument under the same conditions. Disease status is adjudicated against a written rubric, not pulled from coding systems built for billing.

You collect metadata the model will need. Age, ancestry, environment, medications, time of day the sample was taken. Anything that might be a confound becomes a covariate the model can either condition on or be evaluated against.

You write the consent form for ML reuse. Participants are told, in plain language, that their data will be used to train AI models that may be applied to questions not yet imagined. The consent is broad and explicit rather than narrow and retrofitted.

The metaphor I'd reach for: existing genomic datasets are like field recordings made by different people, on different equipment, in different rooms, of different songs. You can learn a lot from them, but training a model to generate music from them is hard because the model keeps learning the rooms. An AI-ready dataset is a studio session: one room, one engineer, deliberate choices about what to capture.

The metaphor breaks down at consent and biology — studio sessions don't have ethics committees, and even a perfectly engineered genomic dataset will not dissolve the underlying messiness of how DNA maps to traits. But for the engineering question of "why is this hard to train on," it holds.

What this enables, and what it doesn't

Sanger already co-released the Nucleotide Transformer with InstaDeep in 2023 — a 2.5-billion-parameter DNA language model trained on 3,202 human genomes plus genomes from other species. It works, in the sense that it learns useful representations of DNA sequence. But its performance on downstream tasks like predicting which variants cause disease is limited by exactly the data problems above. A bigger Nucleotide Transformer trained on the same kind of data would mostly learn the same artefacts more confidently.

What an AI-ready dataset enables is the next generation of these models — ones whose limits are set by biology rather than by lab noise. That is genuinely valuable. It is also a more modest claim than "AI will solve genomics," and I'd be careful with anyone making the bigger one. The combinatorial problem of complex traits — most diseases are influenced by thousands of variants interacting with environment — is a biology problem, not a data problem. Better data helps. It does not dissolve.

What to watch

Two things. First, the governance terms. "Shared widely" is not the same as open access, and human genomic data carries re-identification risks that protein structures don't. The specific licence and access framework, who gets the data, under what conditions, with what oversight, will tell you what kind of consortium this actually is. Second, whether other institutions join. A single cohort, however well-designed, is still a single cohort. The consortium's stated openness to expansion is the part that would, if it materialises, make this infrastructure rather than a project.

The $25M is small money by frontier-AI standards. What is being bought with it is not compute or scale — it is protocol design, and the patient work of building a dataset that a model can actually learn from without learning the room.

Footnotes and links

Further reading

• Nucleotide Transformer paper (Dalla-Torre et al., 2023): https://www.biorxiv.org/content/10.1101/2023.01.11.523679v3
• DeepMind's Enformer (genomic sequence model, 2021): https://www.deepmind.com/blog/predicting-gene-expression-with-ai
• UK Biobank, for comparison on cohort scale and design: https://www.ukbiobank.ac.uk/

¶