I recently bought a belt sander and noticed that while using it stationary it can get clogged (on a certain region) real fast.
The sharpness of the area is not lost but it's just clogged with wood dust.
What is a proper way to clean it?
There are special products out there called sanding belt cleaners (one such is harbor freight sku 30766), they're essentially large gum eraser sticks that grab particles out of the sanding belt.
Another avenue to look at is consider why the belt is loading up. Try applying less pressure while sanding, or run a shop vac in one hand and the sander in the other.
In addition to cleaning your belt with a cleaning stick as rockerpult suggested, you should also take a look at the material you're sanding and the belt you're sanding with.
Finishes that form a film on top of the wood (paint, shellac, polyurethane, etc.) often clog sandpaper, so if you're using the sander on a large finished surface, you might want to consider other options (e.g. chemical stripper or scraping) to remove the finish before sanding. Woods that contain a lot of resin or oils can also build up easily. In can help to keep a block of wood that's not prone to clogging on hand, and just run it against the sander when buildup starts.
Switching belts may also help. I find that coarser grits tend to clog less, so take the time to switch to a coarse belt if you're removing a lot of material. Also, some belts are coated with stearate, which is a sort of release agent that prevents clogging; using one of these could help. (Some kinds of stearate don't get along with water-based finishes, though, so read the labels and think ahead.)
I've been using an old sneaker because I saw it recommended repeatedly. Seems to work fine, and doesn't effect the remaining grit so far as I can tell.
I have one of the rubber sanding belt cleaners, but I've found compressed air to be just as effective, and most people would have at least a small air compressor in their shop.