Three-dimensional (3D) genome organization is closely linked to gene regulation and cell fate during mammalian development. This review examines how 3D chromatin architecture is established and dynamically remodeled from gametogenesis and early embryogenesis to postnatal development and disease. Rather than providing a conventional summary of hierarchical genome organization, we focus on the interplay among loop extrusion, biomolecular condensation, transcriptional and RNA-mediated regulation, and developmental context. Particular attention is given to species-associated differences in early embryonic genome reconstruction and to the unresolved relationship between chromatin architecture, transcriptional regulation, and developmental outcomes. We also discuss emerging technologies and their limitations in distinguishing structural associations from causal regulatory relationships.