Not sure what the problem is, but if it was bugs, it would’t dry out so quickly. Flyin’ fuckers eat so much root material, that the plant can’t absorb water.
Have you checked for salt content in your soil or water. I never used peat pots because I read somewhere salt is an occasional problem in them.
It seems to be all or partially root related. Try this product I found. It is an excellent additive for roots that you put in the water and use it every time you feed. It is referred to as a “root inoculant”
https://www.botanicare.com/products/hydroguard/
It is available many places including Amazon and eBay. What it actually is is a beneficial bacteria. It eats dead root material and promotes vigorous root growth. The healthier the roots are, the prettier the plant will be.