In other areas of woodworking tools I can see why one brand would cost more than another because of quality, manufacturing, and material differences
You forgot one other important factor, which is paying for the name.
Unfortunately there's a lot of this in woodworking and it extends through all levels of tools (both power and hand). You tend to pay a premium for established name brands and the better the name's reputation the greater the additional payment. And this additional money does not necessarily net you a greatly superior tool.
This is not to say you can't get what you pay for when you buy an established name, but unfortunately paying a premium price is not a guarantee of getting a premium product.
The 'sweet spot' for steel in chisels is arguably now CrV, an alloy of steel containing chromium and vanadium which give some additional benefits over basic carbon steel without adding greatly to its cost. As a result inexpensive, even cheap, chisels can now be of more than merely acceptable quality, as shown by the very inexpensive chisels sold by the Aldi chain that were plugged for many years by Paul Sellers as being the best buy in all of woodworking.
The above being said the heat treat is everything, and no matter the quality of the steel it can be wasted if it's not heat treated well — the edge won't be worth a damn, if too hard it'll be prone to crumbling or cracking, if too soft it won't hold an edge for long as well as being prone to what's called edge folding which is exactly what it sounds like.
The good news is that if you do get a chisel that is too hard or too soft you can rectify this at home with simple equipment that won't cost much (if you have a gas cooker or range you may even be able to do it using one of your burners). This is extremely good news for any budget-minded carver as in addition to the obvious of giving you the ability to improve the quality of the edge of any cheaper carving tools you buy (I should stress here, IF this is needed) it opens up the possibility of making your own tools from scratch, either starting with cheap bench chisels or from raw steel stock1.
What determines the quality of one set over another?
One additional issue here is your focus on sets. Although this varies a little with some tool types it is generally acknowledged that sets are often not the great buy they appear to be — they tend to give you a few tools you'll use a lot, one or two you'll use infrequently and a couple you will never use. The larger the set the more this is true, for most people.
On the other hand sets can be very affordable and give an introductory selection of tool types that you want to try if you haven't carved before so the appeal is understandable. But do be aware that you will very likely pay for one or two you won't use much or at all.
The usual recommendation when new users are being steered away from sets is to identify the tools you're sure you'll need2 and buy only those initially. Once you start to price individual carving tools this sounds like a very expensive route and it can be if you go crazy and buy dozens of tools initially, but no beginner should do this.
While many long-time pros have very large chisel collections (some are nearly unbelievably large3) you can do a lot with just a handful of chisels and a knife or two if you apply yourself and figure out how to shape. I have a sculptor friend, not a carver, and looking at his work and the minimal toolkit he uses to do it (the majority of which he made himself) has convinced me yet again of the fact that it's possible to do great work with minimal tools.
In carving, at the other end of the spectrum from Grinling Gibbons we can look today at some of the virtuoso carvers in the Middle East and Asia working in dirt-floored workshops who sometimes only have about a dozen tools at their disposal!
So my advice would be to get a few tools (in a set if you work it out and this is the cheapest way to get every shape you've identified you need) and then learn to use them to their fullest potential — this isn't carving for 10 or 20 or even 50 hours and struggling to do certain tasks, this is weeks of dedicated work, or months if you can only carve occasionally. Then go shopping for more if/when you find it's necessary.
Don't overlook sharpening
Even some very good tools won't be supplied fully sharpened. Cheaper carving tools (which may be of acceptable quality for you at this stage) will almost invariably be sold blunt, just ground to shape and with the expectation that the user will sharpen them before they're first used.
If you do buy a set that's sharp just as with bench chisels you will need to sharpen them fairly soon after you buy them because they will blunt slightly through use (the harder or more abrasive the wood your work with the faster this will occur). More sharp is always preferable to less charp obviously but carving chisels in particular are very often used cross-grain, this requires the best edge you can manage for the cleanest cuts because you're consistently working show surfaces that won't be cleaned up or refined afterwards.
There are numerous previous Q&As related to sharpening here if you need some guidance in this area. If you don't have anything currently at the least I would recommend getting some quality abrasive paper (wet/dry type) up to 600 grit and making yourself a strop.
A final word about handles
A lot is said about the need for good handles on chisels generally and carving tools as well. So broad agreement there, what there isn't broad agreement on is what constitutes a good handle. It's very much a matter of personal preference and hand size what makes a good handle.
What one guy likes another won't mind, and you can bet you'll find a third who loves them.
So don't be completely put off by someone else's hatred for a handle type due to material (e.g. they hate plastic) or size (two common complaints being "They're too big" and "They're tool small" so you can see that it's hard for manufacturers to win that one LOL) or shape.
You can remake existing handles to better suit your tastes, as shown by Paul Sellers on the Aldi chisels. The handles on the Aldi chisels are not widely liked so there are also many re-handling jobs out there online to find (including by Mr. Sellers) and they range from the utilitarian to the extravagant.
Remember this above all: you're a woodworker, if you need to you can make a set of handles for yourself!
Although if you pay a lot for a tool you won't want to sometimes this is the best route to getting a handle that you like best. Even some pretty high-end chisels (e.g. Veritas, L-N, Sorby) have had the DIY handle treatment to better suit the tastes or requirements of their owmers, so complete replacement is by no means confined to cheaper chisels with mediocre handles.
1If you're interested in this there are many resources online you can tap, but a great starting point would be Make Your Own Woodworking Tools: Metalwork Techniques to Create, Customize, and Sharpen in the Home Workshop by Mike Burton.
2Use a number of books and online guides to determine this, not what looks like it will be useful to your untrained eye. You can use your own judgement in the future to buy selectively once you have some experience and have identified gaps in your tooling that would be useful or essential to fill, but initially it's of little or no use.
3The famous Grinling Gibbons is said to have left a collection of carving tools numbering in the thousands when he died, although it's not clear if he used all of them his 'working set' was in the hundreds apparently.