Hardwood is usually from a deciduous tree and softwood is usually from a coniferous one. Hardwoods typically have a higher density (hence hardwood).
Seriously?
For the most part that is the general accepted (although broad) definition and yes there are several exceptions.
Little more than that please
Much like identifying wood species; determining if a particular wood is soft or hard depends on the kind of tree it came from. More specifically:
Hardwood
Comes from dicot angiosperm which mean the tree reproduces with flowers and most have broad leaves that are shed in response to natural climate change or drought. There are several species of evergreen that fit into this category as well. These evergreens are usually located in more tropical/subtropical zones.
Hardwood trees have large vessels for transporting water. These pores are responsible for the grain appearance in hardwood and are best seen under microscope.
Softwood
Almost all softwood comes from gymnosperm plants such as conifers or also known as coniferous trees. Where hardwood tree use flowers for reproduction softwood trees use seeds, such as cones. Conversely to hardwood, water and sap are transported via medullary rays and tracheids which can appear corrugated (like cardboard).
Comparison
As mentioned before, the best way to be sure is to examine the wood under microscope. In the hardwood you can see the "pores" shown as large holes (In the diagram below it is the picture on top.) Softwood does not have visible pores.

Picture comes from Wikipedia
Notable exceptions
Just because a wood is classified as soft does not mean it is necessarily softer than a hardwood. There is a wide range of hardness when it comes to the many species of trees.
I mentioned earlier that there are a couple of exceptions as far as the generic hardwood/softwood definition are concerned.
Balsa: Is actually a soft hardwood.
Yew: Is actually a hard softwood.
Bamboo: Considered a hardwood but classified as a grass.
More reading
In trying to keep scientific analysis out of this answer I refer keen readers to other more in depth sources of information. Always remember that the accuracy of Wikipedia should be taken with a grain of salt and cross verified where possible.