A Taxonomy-Informed Sparse DNA Foundation Model for Microbial Genomics
Journal:
bioRxiv
Published Date:
Sep 24, 2026
Abstract
Microorganisms are crucial to Earth's ecosystems, with genomes encoding functions important to agriculture, biotechnology and human health. Although genomic language models have advanced DNA representation learning, the extensive diversity of microorganisms and highly imbalanced taxonomic composition of existing pretraining corpora pose challenges for effective microbial genomic sequence modeling. Here we present MicroGlot, a taxonomy-informed microbial DNA foundation model pretrained on 3.70 million sequences comprising 378.3 billion nucleotides from 99,700 species. MicroGlot represents hierarchical relationships among taxa using hyperbolic embeddings and integrates this knowledge into a sparse mixture-of-experts architecture. Probing frozen embeddings across layers showed that MicroGlot captures both microbial phenotypic traits and taxonomic identity. Compared with a taxonomy-ablated variant pretrained under the same scheme, incorporating taxonomic knowledge consistently improved representation quality across model depth. Combining sparse computation with efficient architectural and training techniques, MicroGlot achieved competitive probing and fine-tuning performance with low computational requirements. Species-level expert-routing fingerprints aligned more closely with taxonomic groups than tetranucleotide-frequency profiles, indicating that expert usage recapitulates taxonomic structure. These results show that combining sparse architecture with taxonomic knowledge enables efficient and biologically informative genomic language modeling across diverse microbial taxa. MicroGlot is publicly available at https://huggingface.co/athanzli/MicroGlot.
