RNA DNA Structure Sequence Analysis
Geometric and causal methods applied to sequence, structure, and variation in nucleic acids.
Graded Evaluation of RNA Structure Awareness
A Graded Evaluation of RNA Structure Awareness in Foundation Models: From Stem-Loop Discrimination to Partner Specificity
A three-rung composition-controlled evaluation of RNA structure awareness across ten foundation models (five RNA-pretrained, five DNA-pretrained) and 52 Rfam families. Rung 1 (stem/loop discrimination) applies a nucleotide-stratified permutation null that absorbs 3x of the naive signal, exposing GC enrichment as the dominant confound. Rung 2 (dinucleotide-stratified null) eliminates all but one family. Rung 3 (partner specificity) tests whether models resolve which specific position pairs with which, using within-stem derangement nulls (1,000 derangements per stem).
Two models clear the third rung. RiNALMo (650M) achieves the highest mean perturbation specificity (PS = 0.227, 28/29 qualifying families, partner-is-max precision 87.4%). ERNIE-RNA (86M) follows (PS = 0.117, 28/30 families, 88.3%). An untrained control (randomized weights, preserved architecture) produces PS = 2.5 × 10-8, confirming seven orders of magnitude separating learned from architectural signal. Evo (7B) shows the weakest signal despite 500x the parameters of the next-largest SSM.