As clarified in the Comments, some of the abrasive used was silicon carbide paper (wet 'n' dry). The abrasive grains on this kind of paper fracture during use1, releasing a very fine grey/black dust. On the hard materials wet 'n' dry paper is suited to2 this dark residue isn't a problem, but obviously on wood it can become lodged in the surface. On some softer or more open-textured woods the dark staining can be much worse than this.
You can try to scrub it from scratches or divots in the wood surface using a very stiff brush like a toothbrush, or even a fine wire brush, but my experience has been this is almost never 100% effective. And on end grain it simply doesn't work. So unfortunately the only effective way to deal with this is to remove the affected wood; file, scrape or sand until you've gone deep enough that the stains are gone.
When you're faced with such a task on a well-sanded piece like this almost everyone doing it by sanding is tempted to stick with a finer grit, to avoid creating deeper scratches again. But this is a false time economy — you spend more time sanding at higher grits than you would starting with a coarser grit (which removes material so much faster) and then working through your progression of grits to successively remove each previous stage's sanding scratches.
1 Which is as intended, fracturing reveals fresh sharp edges on the grit particles which continue to cut well as the paper wears, in contrast to aluminium oxide abrasives which tend to round off during use and progressively slow
2 Which includes hard finishes used in woodworking (including shellac, varnish and lacquers) when aiming for a very flat surface and/or working towards a super high gloss or 'piano finish'.