What should I have done to get the chip to match the piece?
The best fix hands-down for something like this is to glue the chip back into place. Assuming the chip can be located of course.
It is important however to strive not to get glue onto the surface of the chip as you're glueing it back in, as it leads to the classic problem associated with glue-contaminated wood when it comes time to finish.
I was thinking of rubbing the chip's top surface briefly with mineral oil before the gluing, but I was worried that might weaken the bond.
You were right to be worried, any oil contamination of glueing surfaces severely compromises the bond.
Where using the original chip is not possible (the chip can't be found or it has been damaged so it won't neatly slot into place any longer) glueing in a piece of the same species would be next best, striving to get a good grain match to the wood in that location. Where the divot in the wood is an irregular shape it can be difficult to get a good edge on a patch so often it's advisable to create a patch with a regular shape (a diamond with the long dimension pointing along the grain is often recommended) and then chiselling out wood to match that. In this case you'd just use a triangle.
If you must use a filler probably the best filler is one made from sanding dust and glue. Now this tip is very common and often in the next breath the person recommending it will say it gives a good colour match*, but generally it does not make a good colour match to the surrounding wood (almost invariably it's darker) and it doesn't stain or finish well (almost invariably ending up lighter). However this can work much better than it normally does: if hide glue is used.
*Some people recommending the wood dust + glue tick may be using hide glue and are just unaware that it makes such a big difference in the outcome, compared to using regular PVA-type wood glues, epoxy or superglue, so they don't know to specify that that is the adhesive they're using.