In Asia, solvents are generally sold in glass beer bottles with plastic stoppers. It works well, though there is some diffusion through the stopper. Not enough to smell, unless you're storing something like HCl. If you can get stoppers and glass bottles, it's the cheapest solution.
Next, I can give some rules of thumb for plastic, since I find grfrazee's answer to be a bit too conservative. The plastic bottles you'll have available are are acrylic, HDPE, LDPE, PET, or PP. Don't put solvents in acrylic. PET should generally only hold alcohol or white spirits (naphtha/lighter fluid/paint thinner). Alcohol, along with the other oxygenated solvents like acetone and MEK, seem to keep okay when they're stored in LDPE. However, the more nonpolar hydrocarbons (xylene, naphtha, paint thinner) slowly diffuse through the sides of LDPE bottles and escape. Lighter fluid is sold in HDPE, so the rate of diffusion of white spirits through HDPE should be acceptable. On paper, HDPE and PP are generally better than LDPE. So in short, polyethylene/polypropylene are your cheap choice. Oxygenated solvents will stay in anything without diffusing through the plastic too quickly in summer. Chemical compatibility charts claim the plastic will weaken, but it won't dissolve.
I haven't used turpentine, but it's normally similar to white spirits. I would not store xylene/toluene, methylene chloride, or lacquer thinner in any plastic bottle that is not specifically made for solvents.
