I read that having a proper saw set reduces the chance of the saw blade buckling.
One purpose of set is to reduce the chance of a saw binding in the kerf. This can lead to buckling, but with some saws it would be nearly impossible for them to buckle in use (a quality backsaw for example, due to the added stiffness provided by the sawback).
Saws with no set can also be described as having '0' set.
Set is not the only way to help prevent binding. Not used as much today, a saw can be made with a tapered blade: thicker near the cutting edge and slightly thinning towards its back.
Either method is particularly needed for rip saws, used for cutting along the grain. During ripping cuts pinching of the saw by the wood is common and starts to become more and more likely as the depth of cut increases (especially once the cut depth exceeds the blade height).
Note: even with set binding can occur so on long rip cuts or when re-sawing by hand it can be necessary to keep the kerf open with a small wedge near the start of the cut:
The other purpose of saw setting is to improve cutting performance (especially for crosscut saws). In addition to an improvement in the actual cutting done by the teeth, any saw with a flat blade and no set will naturally experience increased friction within the kerf so more effort will be required during sawing.
Would there be a reason to remove it permanently?
Yes, it is not advisable for general-use saws but can work well for certain specialist applications.
Some specialist saws are manufactured with no set, for example flush-cutting saws. When sawing flat against a piece of wood any set on the teeth will tend to scratch the surface, so no set is very desirable on such a saw. Often the teeth have no set at all, but occasionally will have '0' set on one side only; for this type of saw care must be taken to have the side with set facing upwards.
You can also modify a saw for a particular purpose — very fine, shallow, finished cuts for example. As noted previously binding is increasingly likely as the depth of cut increases, which is why no set can work for shallow cuts that don't exceed the depth of the saw blade (also especially on crosscuts where there is less of a tendency to bind anyway). Regardless of the type of cutting being done, because of the increased friction regular lubrication of the blade with wax is advisable.