The Sadness of the Males

do i need to put the male plant into flowering for it to produce pollen? or can it happen in veg? i see the sacks starting do i have to worrie when its in veg of droping pollen on my beautiful ladies?