Note that I'm referring to metal-bodied planes throughout here, wooden-bodied planes are different enough that some of the generalisations don't apply to them at all. There are many lower-cost wooden planes that can match or exceed the performance of planes that cost ten times as much, and some that can match planes that cost upwards of fifty times more.
What is the difference between a high-quality (e.g. $150+) and a cheapo (e.g. $20) bench plane?
I'm glad you included this in the body of the Question as just the title question is more likely to generate opinion-based replies rather than something more objective.
The two obvious things that come to mind are blade quality (specifically, how well it holds an edge) and the amount of out-of-the box setup required (e.g. lapping the base, cleaning off grease, sharpening the blade).
Yes those are both definitely factors. Of these two the amount of setup required to get the most from the plane — or to use the older term, fettling — is far and away the more significant factor.
Steel quality
This is actually a little overplayed, starting with the manufacturer's own claims (which we should automatically be a little wary of just on general principles). It's actually not quite as clear-cut as they make out that one of the modern alloy steels (e.g. A2) makes a cutting iron superior to the traditional high-carbon tool steel; while edge retention can be better there are certain trade-offs, including in peak sharpness, and for some users they're essentially overkill.
In addition to the steel itself, the greater thickness of most modern cutting irons is often cited as a key advantage, preventing chatter. But hundreds of thousands of users of traditional thinner-bladed planes — both historically and today — show that chatter is not an inherent problem with planes that have thinner cutting irons, as long as other aspects of the plane's setup are all working together properly. So don't believe the "Thicker irons are better." hype!
Fettling
High end: with a quality expensive plane you should expect to have to do virtually no fettling for the plane to work, or none at all. In fact some makers specifically highlight that their planes are ready to use straight from the box (note that this doesn't stop some users from doing one or two minor tweaks to suit their preferences).
Cheaper end: quality at the lower end of the market is about what you would expect, with wider tolerances and an acceptance of far more defects (both in number and magnitude).
In general it is the quality of the iron castings that are most prone to variations, and it is on these that most fettling is done so this is a key factor. Almost all of the other metal components, in steel and brass, are machined to their final shape so far less subject to variation. They tend to be fairly uniform in being rougher, but still serviceable.
The body components (the main body and the frog in most cases) on virtually all planes of this type are cast iron cast in sand moulds, with machining work in only a few key places and on the cheaper planes all of the as-cast surfaces are prone to significant variation: surface blobs, overall rougher texture and indifferent painting should all be expected.
Note: because there is so much variation in cheaper planes it means that some are much better while some are poor or very poor (these ones tend to get the most press from dissatisfied buyers, colouring the view on planes of this level as being not fit for purpose). So there is an element of luck in how good or bad the one you get actually is. This doesn't mean the less-good ones are all not worth getting, but unquestionably the worst ones are not worth the effort to try to get working well.
Mechanism
Planes which are clones of the Stanley/Bailey design tend to inherently have some 'slop' or 'backlash' in their adjustment mechanism, simply a by-product of the 19th-century mechanics. This is definitely more pronounced in lower-end planes than in high-end ones however.
It should be emphasised that this doesn't in any way make cheaper planes unusable, or in fact significantly harder to use. It just takes a little longer to make adjustments — a few more seconds in some cases — and those adjustments can be as precise and can be held just as well.
But what if we replaced the blade in the cheap plane with the same high quality blade and a skilled worker tuned both planes? Is it just ergonomics & looks after that, or are there other qualities which set apart high quality (or even mid quality?) planes?
I think it would mostly come down to these, yes. In some cases it is aesthetics that are the key difference, in others there are major ergonomic problems*, with the shape of the tote (rear handle) in particular coming in for criticism, but the front knob as well. In addition to their shapes, the material these are made from: not a quality hardwood with a nice finish but an inexpensive plastic casting.
My personal observation on the plastic handles is that despite their obviously cheap appearance they can be comfortable to use, so if you can get past their looks they may actually be surprisingly user-friendly. And it is worth noting that a few of the cheaper planes do still feature wooden handles.
*I think I should highlight here that there's a significant amount of subjectivity when it comes to handles amongst tool users. Some people are pathologically opposed to plastic so regardless of their shape a plane with plastic handles would not meet with their approval, others are very particular about the shape of a tote... often it is this: if not almost exactly like the Stanleys they trained on they're bad, by definition! This is one of those areas where your own preferences have to trump everything else.
