1/4in dimensioned pieces are going to be a challenge no matter what finish you use, or at all.
As suggested in the comments, you are going to have to choose your wood carefully at that dimension to reduce warping over time. I'd go further and say this is an opportunity not only to look for dimensionally better cuts, but also varieties known for stability under these circumstances.
Consider what wood is used in saunas, or outdoor patio furniture. Wood like cedar and acacia hold up reasonably well with no finish, and are somewhat traditional for these applications. (Though I'm not sure how cedar would hold up at such small dimensions.) Check your local builders to see what they would use for saunas.
To address your main question on finishes: almost any reasonable poly or varnish type finish will do. Even so-called "indoor" finish. In most cases, the water resistance is similar, with outdoor finishes being made with more UV inhibitors. You'll want to follow the instructions for multiple light coats. And, whatever you finish it with, if it gets wet a lot you will probably have to refinish it semi-regularly as the finish wears. (This is the big win for not finishing stuff like this: less maintenance.)
So-called spar finishes are usually a urethane type with various oils added, and these also tend to be more pliable than polyurethane finishes. So they might work better with something expected to get dings and knocks as it ages.
You might look at "sealers" as well, which aren't as waterproof, but allow for a nice even finish and allow the wood to breathe somewhat. These also need to be reapplied occasionally, though reapplication is usually easier than with polyurethane.
I'd also advise you to finish the pieces completely before assembly, because moisture likes voids and cracks, and water ingress at a joint on otherwise finished wood will have a tough time drying out, weakening the joint and discolouring the finish around the joint. It is likely you will be using a modern glue for assembly, so care will have to be taken to remove the finish from the gluing surfaces.
At 1/4in your choice of fasteners and joinery will be important, as well. For fun, you could consider not using metal fasteners at all. But if you do want metal fasteners I'd advise stainless-steel types. Even those coated types for roofing and siding will rust and stain eventually.
I'd advise you to look and see what those "claw-foot" bathtub wood caddies are built from, and how they are built, as this is a similar application to yours.