I’d say the shed is your best option there. Like you said insulate the hell out of it. If you keep a veg area going 24/7 it will provide some heat but you’ll probably need a supplemental space heater or something as well. Check into local codes, usually there’s a maximum size you can build without getting any permits.
Also you’ll probably need an air-conditioner in the summer, a window unit should do a small space like that so keep that in mind when you’re designing it.