I am thinking of doing the legs straight up and down for a more minimal, easier construction. What does it change structurally if I wanted to have the legs straight?
The stool is slightly less stable front-to-back if the legs don't splay. Or to put it another way, it'll tend to want to tip forwards or backwards unless the seat of the stool is deep enough front to back that the legs are far enough apart. For a narrow stool the legs splaying is an important feature.
I'd encourage you to draw a sketch or better yet, make a quick mockup of the design with vertical legs. They're less common not only because of lower stability, they also tend to look a bit odd..... although some of that might be from us being conditioned to expect splay in chair legs :-)
If I wanted to copy the angled assembly, I planned on drilling through the top with a drill press set at a fixed angle
Remember legs like in the pictured stool are at a compound angle, that is, they angle outwards in two directions.
Once you get your head around the angles it isn't too difficult to learn to do this manually (using a brace or with a power drill) along with a pair of bevel gauges or other angle guides to help check and maintain alignment. There are numerous guides to this process in books, magazine articles and online if you want to explore this option.
But there's no shame in doing it on a drill press, if you own one you might as well use it! There's a good old guide to doing this on the drill press in the Nov. 1941 issue of Popular Mechanics which you can see here on Google Books.
I'm not sure how to go about removing the material for the leg at an angle?
Make the tenon portion deliberately overlong and then saw flush if you have a suitable saw1, or nearly flush with another type of saw (protecting the surface from scratching if needed), after which you use some combination of planing, paring with a chisel or sanding to remove the excess. Regardless of which you do expect that some final planing, scraping or sanding will be necessary for a perfectly smooth and flush finished result.
As these things are fundamental to the process of making a stool such as this I think you could do with reading a good guide to making any similar stool, and/or looking at a video or two. These will cover the above, as well as other important aspects2 which will help ensure your project is a success.
1 Flush-cutting saws, are now widely and inexpesively available e.g. from Amazon.
2 Marking the legs for cutting them to final length is one, how to safely wedge the through-tenons is another.