I ran into a similar problem when I first started growing. What’s happening is you are moving your pots around causing soil compaction. If, like me, you used the perfect aerated soil mix, ie: say 30% perlite 
, you are breaking and crushing the roots in addition to compacting the soil when you move them around. Good luck and don’t move the pots around plus it also stresses the plant as it tries to regrow toward the new light angle…slows down growth.
