Identifying the right genre for the book you’ve just written is important for all kinds of reasons. Most literary agents want to know what sort of book you’re trying to entice them into representing, usually in the first sentence or two of your query letter. Publishers and publishing imprints are often associated with interest in specific genres. And in the happy event that your book is published, the genre your book falls into will ultimately determine where it ends up in the bookstore and to a certain extent what readers expect from the experience of reading what you've written. Genre is a handy little term that conveys the category a novel falls in, the overall nature of its plot, the time period, and in some cases the setting of the story.
There are a few more terms used commonly in the publishing industry that are not included in the genre glossary below but are equally important to understand considering their pervasiveness:

Mainstream fiction is a catch-all category for (usually) character-driven stories that don’t have a concretely identifiable readership.

Commercial fiction is often thought of as short and easy to read, but the term is most often used to describe a category that’s proven to have a loyal readership and can thus be expected to sell and sell well.

Literary fiction generally appeals to a smaller audience and features excellent writing and originality of thought and style. These are the titles that may be nominated for Pulitzers or the National Book Award and often stand the test of time, but aren't necessarily beach reads.

Genre fiction is what's elaborated on in the glossary below—works that fit, at least reasonably well, into a particular genre that has certain conventions. Raymond Chandler’s The Big Sleep is a prototypical hardboiled detective novel, and Louis L’Amour’s novels are classic westerns.

While all genres have a perfect example from some point in time, most titles overlap more than one genre. For example, Foer’s Everything Is Illuminated is part historical fiction, part romance, part mystery, part detective, and part frontier travel. But if I were to throw all these terms into a query letter, the agent might laugh all the way to the recycle bin. To try to explain a novel entirely by genre is a ridiculous situation a surprising number of writers put themselves in—while genre is important to include, most agents want to know the story, not just the label. Everything Is Illuminated is a humorous literary novel that follows Jonathan Safran Foer, a character conveniently of the same name as the author, to Ukraine, where mum farmers, muddled history, and a language barrier aren’t the only hurdles to discovering who rescued his relatives from the Nazis. See? Hooks are easy. Just equate a genre definition with salt: too much and you’ll make your pitch inedible.

Action-Adventure
Naturally, this genre features physical action and violence. Often located in exotic or forbidding spots such as jungles, deserts, or mountains, action-adventure stories usually center on a quest, escape attempt, a botched or troubled journey, or a military-style mission. Lead characters might include terrorists, pirates, commandos, or mercenaries who serve as a principle source of conflict. Themes might include courage/pride/honor, male bonding, survival, and/or betrayal, and the stories often feature technology, weapons, and other hardware.
Classics in this genre include Robinson Crusoe (Daniel Defoe), The Beach (Alex Garland), and most anything by Jack London, Michael Crichton, Ian Fleming, or Clive Cussler.

Crime
“Crime,” “detective,” and “mystery” tend to overlap, but novels solidly in the crime genre generally focus on the criminal enterprise. These stories are usually told at least in large part from the point of view of the perpetrators, since focus on any other point of view would put the book more closely in the detective or mystery genres. Crime novels range in tone from lighthearted "caper" stories to darker plots involving organized crime or incarcerated convicts.
American Psycho (Brett Ellis), James M. Cain, Edgar Wallace, and Mario Puzo are classic writers of crime fiction; Carl Hiaasen and Elmore Leonard are enjoyable modern-day examples.

Detective
Detective fiction is usually from the point of view of a sleuth, whether professional or amateur. These stories concentrate on the solving of a crime, usually one or more murders. This large, popular genre has many subgenres, reflecting differences in tone, character, and setting.
Hardboiled detective fiction is distinguished by an unsentimental portrayal of crime, violence, and sex in which the protagonist always seems to have a snappy comeback on the tip of his tongue. This genre was most popular in the 1920’s, 30's, and 40's, though it has never gone out of style. Raymond Chandler’s The Big Sleep and Dashiell Hammett’s The Maltese Falcon are classic examples. Walter Mosely and James Sallis are more modern examples.
Soft-boiled, or cozy, mysteries generally shy away from violence or suspense based on physical confrontation. Modern cozy mysteries are frequently, though not necessarily, humorous and thematic (culinary mystery, animal mystery, quilting mystery, etc.). Examples include Sarah Graves, Jill Churchill, Laurien Berenson, Joanne Fluke, and Valerie Wolzien.

Mystery
Many people consider mystery synonymous with detective fiction, though "mystery" encompasses a much broader range of whodunits, not always involving a murder or other large-scale crime. This genre technically involves characters who try to discover a vital piece of information that's kept hidden until the climax. The writing is often atmospheric and stylized, and the characters of both the perpetrator and investigator are often explored in depth. Good mystery fiction may be so well written and characterized that it crosses over onto mainstream bestseller lists, as do the novels of P.D. James and Elizabeth George.
Note: TED suggests keeping away from the term “suspense novel.” If you’ve written something worth reading, naturally there’s going to be suspense.

Fantasy
This genre features stories set in fanciful, invented worlds or in a legendary, mythic past. The stories themselves are often epics or quests, frequently involving magic. The enormous popularity of J.R.R. Tolkien’s The Lord of the Rings novels and J.K. Rowling’s Harry Potter books demonstrates the wide appeal of this genre. There's a great deal of overlap with science fiction and horror, but fantasy can be typified by the overall look, feel, and theme. In “high fantasy,” language can be stilted or archaic in order to separate characters from the real world. “Low fantasy” is usually more gritty and realistic and less involved in earth-shattering quests.
C.S. Lewis, Piers Anthony, Robert Jordan, and Terry Goodkind are some popular fantasy writers.

Horror
Horror aims to evoke some combination of fear, fascination, and revulsion in its readers. Like all genres, horror continues to evolve, recently moving away from stories with a religious or supernatural basis to ones making use of medical or psychological ideas. There's a modern competition over whether psychological horror or violent horror (also called “splatterpunk”) is more terrifying, but both tend to scare, unsettle, and horrify.
Dracula (Bram Stoker), Frankenstein (Mary Shelley), and The Legend of Sleepy Hollow (Washington Irving) are classic examples of the horror genre. Franz Kafka is a high-brow example. Modern horror writers include Dean Koontz, Clive Barker, and Anne Rice, and the ever-popular (and prolific) Stephen King.

Science-Fiction
This genre is defined more by setting than by other story elements. With a few exceptions, stories set out of earth or in the future qualify as science fiction. Within these settings, the conventions of almost any other genre may be invoked. Speculations based on current science or technology are used, either to change the course of the novel’s history (alternative history), comment on the social progress of science (social sci-fi), or detail military advances or strategies (military sci-fi).
Hard science fiction is characterized by rigorous attention to accurate detail in quantitative sciences, especially physics, astrophysics, and chemistry. Soft science fiction is the antithesis of hard science fiction. It may describe works based on social sciences such as psychology, economics, political science, sociology, or anthropology.
Social science fiction, considered a type of speculative fiction, includes the utopian or dystopian story. Satirical novels with fantastic settings may be considered speculative fiction, such as Gulliver’s Travels (Jonathan Swift). 1984 (George Orwell) and Brave New World (Aldous Huxley) are classic examples of dystopian social science fiction. “Nightfall” by Isaac Asimov is another prototypical social sci-fi.
Apocalyptic and post-apocalyptic fiction is pretty self-explanatory: it is concerned with the end of civilization through nuclear war, plague, or some other general disaster. Post-apocalyptic fiction is set in a world or civilization after such a disaster. The time frame may be immediately after the catastrophe, focusing on the travails or psychology of survivors, or considerably later, often including the theme that the existence of pre-catastrophe civilization has been forgotten (or mythologized). Post-apocalyptic stories often take place in an agrarian, non-technological future world, or a world where only scattered elements of technology remain. A Canticle for Leibowitz (Walter Miller) and The Stand (Stephen King) are seminal works in this subgenre.
Cyberpunk is a subgenre noted for its focus on "high tech and low life.” Its name was originally developed as a marketing term and coined by Bruce Bethke in his short story “Cyberpunk” written in 1980, so that his novel would have more appeal to readers. The term cyberpunk was originally intended to be a character description for Bethke’s characters, but since then it has evolved and developed into a full genre of its own.

Thriller
A thriller is typified by its plot: the main character or several characters are placed in some kind of dramatic or dangerous situation(s) from which they must escape. Time is almost always of the essence in a good thriller, or there's something important to find on which someone’s life probably depends. The story tends to evoke strong feelings of suspense and danger, often involving espionage, crime, medicine, or technology. Often overlapped with mystery or action-adventure novels, thrillers are the modern day quick-read all over the best- seller lists. Take a look at this week's New York Times list for plenty of examples.
Spy thrillers (also a subgenre of spy fiction) sometimes but not always include a government agent hero who must take violent action against agents of a rival government or (in recent years) terrorists. Ian Fleming's James Bond novels are now classic. Robert Ludlam's The Bourne Identity et al and John LeCarre's novels are more recent examples.
Political thrillers have a hero who generally must ensure the stability of the government that employs him. Seven Days in May (Fletcher Knebel) and The Day of the Jackal (Frederick Forsyth) established this subgenre in the early 1960’s.
Military thrillers include a hero, typically a uniformed military officer, operating behind enemy lines alone or as part of a small team of specialists. Rainbow Six (Tom Clancy) and W.E.B. Griffin’s The Corps series are prime examples.
Conspiracy thrillers have a hero who confronts a large, powerful group of enemies whose true extent only he recognizes. Villainous organizations might include the government or secret societies. Examples include Ministry of Fear (Graham Greene) and The Manchurian Candidate (Richard Condon). Techno-thrillers are a subgenre where technology is prominently described and is important to the reader’s understanding of the plot. Michael Crichton and Tom Clancy are both considered to be “fathers of the techno-thriller.”
Eco-thriller is an emerging sub-genre in which the protagonist must avert or rectify an environmental or biological calamity -- often in addition to dealing with the usual types of enemies or obstacles present in other thriller genres. This environmental component is often a central message or theme of the story. Zodiac (Neal Stephenson) and Echoes in the Blue (C. George Muller) are a couple of contemporary eco-thrillers.
In legal thrillers, lawyer-heroes confront enemies inside and out of the courtroom and often are in danger of losing not only their cases but their lives. Most books by John Grisham, Stuart Woods, and Robert K. Tannenbaum are legal thrillers.
Forensic, medical, or scientific thrillers center around the hero’s involvement with an unsolved crime either as a doctor, forensic officer, scientist, or a specialist or expert in any of these fields. His involvement usually endangers his life. The Silence of the Lambs (Thomas Harris) is a classic example, as well as most novels by Jeffrey Deaver, Patricia Cornwell, and Iris Johansen.
Psychological thrillers keep the conflict between the main characters mental and emotional rather than physical, although there is often a violent resolution. The suspense created by books in this popular genre often comes from two or more characters preying upon one another's minds, either by playing deceptive games with the other or by simply trying to demolish the other's mental state. Rebecca by Daphne du Maurier is a wonderfully chilling classic example, as are The Ice Harvest (Phillips), The Killer Inside Me (Jim Thompson), Zombie (Joyce Carol Oates), and Shutter Island (Dennis Lehane).

Romance
Romance is currently the largest, best-selling fiction genre in North America, with a wide array of subgenres. Nearly all romances feature the mutual attraction and love between two main characters as the main plot and have a happy ending. Popular writers include Johanna Lindsey, LaVyrle Spencer, and Ruth Wind.
Historical romances, also called “bodice-rippers,” are love stories set in an earlier century. While the genre has turned away from the trope of forced seduction, contemporary bodice-rippers still feature unrestrained romantic passion and a heroine who initially dislikes and actively resists the hero’s seduction, only to be overcome in the long run by desire. Historical romance writers include Edith Layton, Loretta Chase, and Judith Ivory.
Additional romance subgenres include contemporary romance, romantic mysteries, romantic suspense (a highly popular subgenre), fantasy/sci-fi romance, romance sagas, young adult romance, gay and lesbian romance, Christian and inspirational romance, and ethnic/multi-cultural romance.

Erotica
Erotica is distinct from romance in that it is overtly sexual. There is plot out of necessity, but an erotic novel is unabashedly written to turn on the reader. More attention is paid to detail in clothes, undergarments, and perfume, as well as feelings. Writers include Nancy Flowers, Donna Hill, and Leslie Esdaile.
Some of the same subgenres apply to erotica as above.

Western
Westerns are defined primarily by being set in the American West in the second half of the 19th century, and secondarily by featuring heroes who are rugged, individualistic horsemen (cowboys). Other genres, such as romance, have subgenres that make use of the western setting. Unfortunately, the popularity of the western novel rose and fell with that of the western film. Any novel by Louis L’Amour is a prime example. The fact that they were written decades ago hasn't kept them from selling to readers who still enjoy westerns.

Christian/Inspirational
Any novel with a religious tone, world view, or code of ethics falls into this category. Increasingly, inspirational novels are confronting contemporary issues in which characters are testing their faith in the real world. Since the conflict between good and evil isn’t a new concept and can surface in any genre, the inspirational book confronts these challenges with a character’s faith. June Masters Bacher is quite prolific in this genre, as are Judy Baer and Carrie Bender. Christian romance is a really fast-growing subgenre.

Historical
Historical fiction portrays alternate accounts or dramatization of historical figures or events. Stories in this genre, while fictional, make an honest attempt at capturing the spirit, manners, and social conditions of the person and period they represent with attention paid to detail and fidelity. Memoirs of a Geisha (Arthur Golden), The Other Boleyn Girl (Philippa Gregory), and The Red Tent (Anita Diamant) are contemporary examples of an extensive genre.

Biographical fiction
The written accounts of individual lives, biographies are usually but not always nonfiction. In biographical fiction, most writers aim for historical accuracy and try to depict their subject accurately even though they're inventing dialogue and many details. The truth and the facts are not always identical -- a biographical novel about a historical figure may reveal something important about him or her that "straight" biography couldn't'. Girl With a Pearl Earring (Tracy Chevallier), Frida (Barbara Mujica), and Schindler's List (Thomas Keneally) are great examples.
Memoir is a common subset of autobiography. Traditionally, an autobiography focuses on the "life and times" of the writer, while a memoir has a narrower, more intimate focus on his or her memories, emotions, and insights. Also, memoirs are often devoted to a limited period in the writer's life, whereas an autobiography is more likely to cover many decades.

Children’s/Young Adult
Children’s fiction isn't actually a genre -- children start reading as early as four, are read to when they're even younger, and some kids in their late teens may read fiction targeted to their age group. Picture books are actually a surprisingly hard market to break into, considering everyone thinks they can write one. They're mostly for children young enough to be read to.  Goodnight, Moon (Margaret Wise Brown) is a perfect example. Early readers are books for emergent readers, featuring a repetition of words and a simple, gripping plot to hold a young reader with a short attention span.
Chapter books are transitional between early readers and novels, usually with climax and closure in each chapter. The book often ends with a moral or social lesson.
Middle-grade novels are the next step up, usually for children around third grade. Word counts average between 14,000 and 20,000 words, but the Harry Potter series is a notable exception. Middle-grade novels begin to pare down towards a more select readership by concentrating on sports, escapist fantasies, or boy/girl issues. Series books are popular in this subgenre, such as the Goosebumps series by R.L. Stein.
Young adult novels, also called YA or teen novels, are generally read by children anywhere from fifth grade on up. They tend to be longer (50,000+ words) and are often contemporary stories with modern issues such as drugs, sex, and peer pressure.  The Hardy Boys, Nancy Drew, and The Boxcar Children are classics. Judy Blume, Gary Paulsen, Madeleine L'Engle are more modern examples of YA novelists.

Humor
Comedy, absurdity, incompatibility, or any other way of amusing readers characterize humor books. Dave Barry and Roy Blount, Jr. come immediately to mind.

Satire
Satire is defined as “the use of irony, sarcasm, ridicule, or the like, in exposing, denouncing, or deriding vice, folly, etc.” Satirical fiction features human vice or folly as held up to scorn or derision. Jonathan Swift, Mark Twain, O. Henry, and Voltaire were classic satirists (thanks, BA in English), with John Kennedy Toole’s A Confederacy of Dunces, Kurt Vonnegut’s Slaughterhouse Five, Douglas Adams' Hitchhiker’s Guide to the Galaxy, and Joseph Heller’s Catch-22 as more recent classics. Dead Souls by Nikolai Gogol and Invisible Monsters by Chuck Palahniuk are a couple more.

Multi-Cultural
Multi-cultural can be a tricky genre to pin down because it can mean different things to different literary agents and publishers. Most insiders will agree that multi-cultural fiction is a code word for books that offer racial and ethnic diversity in the depiction of characters, cultures, and conflicts. This includes African-American and Latino fiction. In addition, books characterized as multi-cultural often fall under the broader genre umbrella of commercial fiction, romance, chick lit, or literary fiction. Waiting to Exhale (Terry Millan), House on Mango Street (Sandra Cisneros), and Joy Luck Club (Amy Tan) are all examples of multi-cultural fiction as well as commercial or literary fiction.

Travel
Travel literature typically records the people, events, sights, and insights of a writer touring a foreign place for the pleasure of travel. To qualify as literature the work must have a coherent narrative or perceptions beyond a mere logging of sights and events, such as in a diary or ship’s log. Novels that recount adventure, exploration. and conquest are sometimes grouped under travel literature, and the genre also has a subset, outdoor literature--these genres often overlap with no definite boundaries. On the Road (Jack Kerouac) and Travels with Charley: In Search of America (John Steinbeck) are classics, and notable modern travel writing includes titles by Bill Bryson, Rory Maclean, and Dave Eggers.

Women’s Fiction
Predictably, women’s fiction is fiction about women’s issues for a female readership. While the prose style is more literary than chick lit or romance, women’s fiction is highly commercial. Its characters are typically women attempting to overcome both personal and external adversity. Joyce Carol Oates,  Margaret Atwood,  Amy Tan, and  Barbara Kingsolver are rock stars of this genre.
Chick lit describes its intended readership as much as its story’s content. This genre typically offers light-hearted, amusing tales of dating woes, career foibles, and personal antics as they relate to female twenty- to thirty-somethings: finding the right career, the right man, the right attitude. The stories are usually fun, down-to-earth, quirky, and entertaining —a good beach read, though some of them cut deep. A classic chick lit is Valley of the Dolls (Jacqueline Susann). Marian Keyes and Sophie Kinsella are just two of the more recent writers in this busting-at-the-seams genre.
Other women’s genres include Christian chick lit, yummy mummy/hen lit/mom lit, young adult chick lit, and fiction for women of color.

Gay/Lesbian Fiction
This genre is pretty self-explanatory: it explores homosexual themes, featuring gay and/or lesbian characters and often exploring feelings or opinions about homosexuality. Popular contemporary authors include Felice Picano, Katherine Forrest, Alison Bechdel, Michael Nava, Claire McNab, Bett Williams, and Gordon Merrick.

Recommended Reading

Fiction Genres Considered, Part II: A Guide for Submissions to Agents and Publishers

What The Heck is It? A Guide to Sub-genres in Fantasy and Science Fiction

Many Roads to Him: A Guide to Subgenre in Christian Fiction