Both because you're new to carving and due to the results you're getting I'm going to presume you're relatively inexperienced in sharpening too, so the sharpness of your edges will most definitely have some bearing on what you're experiencing. Now, some variation between the surfaces created when you cut with and against the grain is perfectly normal and should be expected — especially with a coarser wood like red oak versus carving-friendly woods like fruitwoods and lime/basswood — but having your tools super-duper sharp will go a long way towards evening out the playing field so to speak.
Because it's such a cornerstone of hand-tool woodworking there are numerous previous Q&As here on various aspects of sharpening. But the two that are perhaps most directly relevant are:
How often should I sharpen?
How do I sharpen curved tools like gouges?
The rest is largely down to knowing various tricks (reading the grain, using shear cuts, riding the bevel) that help when you're carving specific shapes that either cross grain lines or go against the prevailing grain direction, which you'll find lots of guidance on in carving and whittling demos and books.
I am using Pfiel butt chisels.
If you bought a small/beginner set of these, or individually chose the typical starter choices people tend to pick, bear in mind you won't necessarily have every shape that would be most useful.
Although you can generally do a lot with very few tools if you're willing to persevere and really learn the ins and outs of each one it does require some compromise. This is why, as a rule, you'll tend to find large to verrrrry large tool collections on the benches of experienced carvers. Relative to this, another previous Answer worth reading:
What tools can I use for carving
Note: this is not to suggest you need to buy more chisels! But instead to encourage you to be willing to stick it out and do your utmost with the ones you have (along with a very sharp knife if you have one available).
Other than just using sandpaper with is blunting some of the corners and edges but I want to keep sharp
This is a very very common issue with sanding, not at all confined to carving and other complex 3D shapes like gun stocks.
While gun stock are almost always sanded, it's actually very common not to sand carvings because of the extreme difficulty or outright impossibility of sanding into certain recesses by traditional means. So as much as possible carvers try to leave the surface "from the tool".
