Quantifying multilabeled brain cells in the whole prefrontal cortex reveals reduced inhibitory and a subtype of excitatory neuronal marker expression in serotonin transporter knockout rats

The prefrontal cortex regulates emotions and is influenced by serotonin. Rodents lacking the serotonin transporter (5-HTT) show increased anxiety and changes in excitatory and inhibitory cell markers in the prefrontal cortex. However, these observations are constrained by limitations in brain representation and cell segmentation, as standard immunohistochemistry is inadequate to consider volume variations in regions of interest. We utilized the deep learning network of the StarDist method in combination with novel open-source methods for automated cell counts in a wide range of prefrontal cortex subregions. We found that 5-HTT knockout rats displayed increased anxiety and diminished relative numbers of subclass excitatory VGluT2+ and activated ΔFosB+ cells in the infralimbic and prelimbic cortices and of inhibitory GAD67+ cells in the prelimbic cortex. Anxiety levels and ΔFosB cell counts were positively correlated in wild-type, but not in knockout, rats. In conclusion, we present a novel method to quantify whole brain subregions of multilabeled cells in animal models and demonstrate reduced excitatory and inhibitory neuronal marker expression in prefrontal cortex subregions of 5-HTT knockout rats.