Applications are tools that allow you interact with your friends and networks. Examples of applications that Facebook has built are Photos, Notes, and Groups. Now, Facebook allows third parties to develop applications that live within the site. Adding applications allows you to expand the usefulness of the site.
To add an application, go to http://www.facebook.com/apps/
You can read more articles about applications at Category:Applications.