As a general thing it's not a good ideal to sand this finely before applying certain finishes, including varnish, but in this case you solved a practical problem with what you had available which is often what we're forced to do in real-world situations that occur outside of the finishing textbooks. In case you weren't planning on doing so anyway I would recommend you thin your first coat of poly rather than apply it full-strength (note: not absolutely required, just a security measure).
Another option is not to worry about the rough surface now but instead, apply the first coat or two of varnish and then sand. In this way only the tops of the roughness poke through the finish layer and the sanding shaves off just those. After which you can proceed to varnish as normal.
A third option is actually to apply a thick enough coating of finish (in multiple coats) that you don't sand the wood surface at all. Instead, the roughness becomes fully covered in the topcoat and as amazing as it might seem you can't see it. Obviously many of us don't apply varnish this thickly so it's likely this is the least practical of the solutions for the home woodworker.
For next time, the best solution of all is to pre-raise the grain prior to staining so that no post-stain sanding is required. Sanding after staining should always be avoided if possible.
however I'm wondering if this is bad for the bonding of the polyurethane (oil base), or if that only applies to the in-between coats.
There are no bonding issues with sanding finely between coats of varnish (poly or otherwise) despite what you might have read.
In fact no sanding of any kind is required between coats of varnish to ensure bonding of the next layer. Ideally the one, and only, reason you should sand between coats of varnish is to 'de-nib' — to sand off minor surface blemishes, e.g. from dust particles landing in the finish before it has dried.
Runs or drips, streaks and brushmarks, none of these are best dealt with by sanding between coats. These are broadly speaking all avoidable issues and one's application process needs to looked at if these occur regularly, but all of them are best deal with in a single step at the end of finishing rather than dealing with them at each stage of the process. In this way you save both time and finish, and the resulting surface is frequently far better too (after fine wet-sanding and buffing/polishing).
