Yes, you'd leave it in a storage solution when not in use.
Sealed glass bulb PH pens need to be stored in a solution that contains potassium chloride (KCL) that is buffered at either a PH of 4 or 7. Pens usually come with a starter quantity of storage solution but it can be easily found on-line or at most hydro shops. If you look closely, these pens will have a glass bulb similar to:
This storage solution has two purposes, it keeps the solution within the glass bulb from drying out and it ensures the solution that is in the glass bulb of the probe has a proper concentration of ions that are used when making a PH measurement (keeps it conditioned). You do not want to store a sealed glass bulb probe in RO or deionized water as it will leach out the ions from the glass bulb solution over time. You also do not want to leave the glass bulb to dry out. In a pinch, regular tap water can be used until you are able to buy or make some storage solution. It is not always easy to resurrect a PH probe that has been left to dry or has the ions leached out of solution. Your best bet is to get in the habit of placing it into a storage solution once you are done using it. This is true of cheap or expensive probes. Some probes can be refilled but those are rare outside of a laboratory.
The Bluelabs pens have a cap with a sponge as argo945 mentioned. You'd make certain that the sponge remains saturated with the storage solution occasionally and you'll be good to go.
There are a variety of discussions around regarding the accuracy of cheap vs more expensive probes and I do not have much to say other that there are probably some inexpensive models that will maintain acceptable accuracy over time. In particular, if you calibrate them regularly.