Fine Woodworking is typically considered as using only solid hardwood for construction. Portable saws are made for the construction industry. What is considered as "accurate" on a job site, say 1/16 of an inch, is not very accurate by fine woodworking standards, where 1/64 or higher tolerances are sometimes required. In keeping with that definition, here are some reasons why portable table saws are not considered as good for the task.
Power
Cabinet saws are run by large induction motors and have a very smooth power curve. They can handle rip cuts in 8/4 hardwood without bogging down. Most portable saws will use a much smaller motor that will not handle the sustained load. Smaller saws compensate somewhat for their lower power by using thin kerf blades. It can also be tough to run a dado stack with a portable saw - they often lack the power to make deep, wide cuts.
Rigidity
The top of most cabinet saws is cast iron. It is quite heavy, but very stable. The more solidly built cabinet, along with the more powerful motors allow full kerf saw blades to be used. These blades are less likely to deflect, keeping the cuts at a consistent angle. The aluminum tables on a portable saw are more likely to deflect when cutting larger lumber.
Repeatability
The fences on cabinet saws lock very tightly, and multiple cuts can be made without deflection or slippage. Multiple cuts made with the same settings will typically all be identical. With lighter saws, you typically see some deflection in the fence.
Many of the deficiencies that the portable saws have can be overcome, and many may not be an issue for whatever you are wanting to do. If you never plan on having to rip a 10' long 8/4 hard maple board, or trim large panels (as in building a dresser), or need very repeatable cuts (making a chessboard), then a portable saw may serve you well. You may start with one, and later find that it isn't adequate for your needs. In that case, sell it, and buy something more suitable.