Hybrid Sterility

One-sentence definition. Hybrid sterility is the failure of hybrids between two species or populations to reproduce, caused by incompatible gene combinations (Dobzhansky-Muller incompatibilities) that do not disrupt either parent but are deleterious when combined.

One-sentence analogy. Hybrid sterility is like two machines from different manufacturers that each work fine on their own but jam or break when you connect them together — the parts are incompatible even though each part is internally coherent.

Why it matters. Sex chromosomes disproportionately harbor hybrid sterility loci (the “large-X effect” and Haldane’s rule). In threespine sticklebacks, hybrid male sterility maps to the ancestral X chromosome (LG19) but not to the neo-X, while behavioral isolation traits map to both. This dissociation suggests that chromosome age matters: a young sex chromosome has not yet accumulated the kinds of allelic differences that cause sterility in hybrids. This adds a temporal dimension to the large-X effect — the neo-X contributes to behavioral reproductive isolation but not yet to hybrid male sterility.

Where you meet it in the wiki.

Hybrid sterility — detailed empirical and theoretical treatment.
Postzygotic isolation — hybrid sterility as one of several postzygotic barriers.
Sex chromosome evolution — chromosome-age dependence of sterility loci accumulation.

Primary citation.

“The neo-X chromosome harbours loci for male courtship display traits that contribute to behavioural isolation, while the ancestral X chromosome contains loci for both behavioural isolation and hybrid male sterility.” — Kitano et al. 2009, Finding 1

Prerequisites: heterogamety Next, learn about: Haldane’s rule, neo-sex chromosome

Background

Theodosius Dobzhansky and Hermann Muller proposed independently in the 1930s and 1940s that hybrid sterility arises not from single mutations that harm either parent, but from combinations of alleles that diverge in separate lineages and clash when brought together in a hybrid genome. Two derived alleles, each tolerated in its own genetic background, may be incompatible in combination. Jay Coyne and Allen Orr systematized this framework in Speciation (2004), distinguishing postzygotic isolation from prezygotic barriers such as mate choice. Hybrid sterility frequently obeys Haldane’s rule: the heterogametic sex is disproportionately affected.

Sterility mechanisms divide broadly into two classes. Genic incompatibilities involve protein or regulatory mismatches; chromosomal incompatibilities arise when structural differences between parental karyotypes disrupt meiosis. Prdm9, a histone methyltransferase that directs double-strand break positioning during meiosis, is one of the clearest speciation genes identified in vertebrates: divergent Prdm9 alleles in Mus musculus musculus and M. m. domesticus hybrids cause genome-wide failure of meiotic recombination and spermatogenesis arrest. Drosophila sterility factors such as OdsH show that sex-linked genic loci are major contributors to hybrid male sterility.

How It Works

Meiotic failure is the most common route to hybrid sterility. When parental genomes carry divergent alleles at loci that regulate chromosome pairing, double-strand break repair, or sex-chromosome silencing (see MSCI), meiosis arrests at the pachytene stage and sperm never mature. Prdm9 failure in mice works this way: each parental allele recognizes different genomic sites, leaving DSBs unrepaired genome-wide.

Chromosomal incompatibility provides a second route. A hybrid heterozygous for a large inversion or a reciprocal translocation cannot pair chromosomes correctly along their full length, producing unbalanced gametes. A heterozygous Robertsonian translocation generates a trivalent at meiosis; uneven segregation delivers the wrong chromosome doses to gametes. Not all heterozygotes are sterile: short inversions may allow enough pairing that some balanced gametes form, and holocentric chromosomes tolerate fission heterozygosity better than monocentric ones do.

A third route is meiotic drive suppression failure: when a hybrid carries a driver allele from one parent and the suppressor from the other fails to silence it, one haplotype destroys competing gametes and overall fertility collapses.

The timing of sterility matters for interpreting crosses. F1 sterility means first-generation hybrids are themselves sterile. F2 breakdown means F1s are fertile but their offspring show reduced fertility as unfavorable allele combinations reassort. Backcross sterility is particularly useful for mapping: investigators localize sterility loci by repeatedly backcrossing to one parent while tracking chromosomal heterozygosity.

A Worked Example

The Mus musculus musculus x M. m. domesticus system shows genic hybrid sterility clearly. Male hybrids show spermatogenesis arrest because the domesticus Prdm9 allele directs DSBs to sites that are poorly positioned in the musculus genomic background, producing asynapsis and meiotic arrest. Females from the same cross are fertile. XY males are affected first because X-hemizygosity exposes any incompatibility involving the X, and because MSCI makes the X broadly inaccessible during meiosis, amplifying incompatibilities.

Sorex araneus illustrates chromosomal hybrid sterility. Shrew races differ by Robertsonian fusions; at contact zones, heterozygotes carry complex multivalents at meiosis. Testis-weight data from these zones suggest partial fertility reduction, though the effect varies with which chromosomes are involved and the size of the multivalent.

Common Misconceptions

Sterility is not the same as inviability. A hybrid can be healthy and long-lived yet completely sterile; the two phenotypes arise through different mechanisms.
Haldane’s rule describes a pattern, not a mechanism. The dominance hypothesis and the faster-male hypothesis are both proposed to explain why the heterogametic sex is sterile first.
Chromosomal rearrangements do not automatically cause sterility. Many inversions and Robertsonian fusions segregate at moderate frequency within species because heterozygotes produce enough balanced gametes to reproduce.
F1 sterility does not require many loci. Two incompatible loci are theoretically sufficient, and empirically hybrid sterility between close relatives often maps to a small number of large-effect loci.

How to Spot It in Papers