A variety of visual cues can trigger defensive reactions in mice and other species. In mice, looming stimuli that mimic an approaching aerial predator elicit flight or freezing reactions, while sweeping stimuli that mimic an aerial predator flying parallel to the ground typically elicit freezing. The retinal ganglion cell (RGC) types involved in these circuits are largely unknown. We previously discovered that loss of RGC subpopulations in Brn3b knockout mice results in distinct visual response deficits. Here, we report that retinal or global loss of Brn3b selectively ablates the fleeing response to looming stimuli while leaving the freeze response intact. In contrast, freezing responses to sweeping stimuli are significantly affected. Genetic manipulations removing three RGC subpopulations (Brn3a⁺ betta RGCs, Opn4⁺Brn3b⁺, and Brn3c⁺Brn3b⁺ RGCs) result in milder phenocopies of Brn3b knockout response deficits. These findings show that flight and freezing responses to distinct visual cues are mediated by circuits that can already be separated at the level of the retina, potentially by enlisting dedicated RGC types.