Categories allow you to group your posts and organize them along your website. Depending on the theme and widgets you have enabled on your website, the name of the category can be displayed before or after a post or publish a post in a specific area on a page.
They give some sort of ID to your posts, and you can call them and make them show up whenever and wherever you want, which is a more advanced way of using categories beyond grouping.
Tags on the other hand, are more specific than categories, they are generally used to describe your post in more detail. The use of tags is entirely optional, but it can help your readers find related content to Drama, instead of going through all the posts under the category Books.
So a category can be Book, and tags can be drama, science fiction, nature, painting, small editions, spanish, dutch.
ARTICLES FOR YOUR INSPIRATION
Tag Optimization for WordPress
Simple Tags: Tag Management Plugin
How to Merge and Bulk Edit Categories and Tags in WordPress
How to Use WordPress Categories to Attract Search Engines