If you can teach a model genomics, it can teach genomics back to you. You can follow its eyes to understand how it thinks!
That information lies within the attention map. In nucleotide-based models, it holds a truth about which area of the genome is linked to which other.
Depending on the task the model has been trained on performing, the information gathered through this process can represent different kinds of relationships.