WHEN Shana Kelly, a literary agent at the William Morris Agency, submitted Curtis Sittenfeld's first novel, "Cipher," to book publishers in 2003, she had high expectations. She contacted nearly two dozen high-ranking editors at major publishers, expecting every one to make offers for her client's coming-of-age story set at a boarding school.
Within weeks, though, most editors had passed. "They loved it but weren't sure they could sell a lot of copies, because they couldn't figure out how to market it," Ms. Kelly said. In the end, Random House was the only publisher to make an offer, giving Ms. Sittenfeld a $40,000 advance.
Random House published "Cipher" in January 2005, renaming it "Prep" and backing it with a clever marketing and publicity campaign. But the initial print run was just 13,000 copies -- not enough to generate added royalties on the $40,000 advance.
"Prep" proceeded to confound all expectations by making the New York Times best-seller list a month after publication. The hardcover, with a cover price of $21.95, eventually sold more than 133,000 copies, according to Nielsen BookScan, which captures about 70 percent of sales. The paperback also became a best seller, selling 329,000 copies to date. Foreign rights have been sold for publication in 25 languages, and Paramount has optioned the movie rights.
The book was buoyed by favorable press and word of mouth. But other books receive similar attention and go nowhere, so why was "Prep" so successful? Conversely, what causes a book that was all the rage at auction time to fall flat at bookstores?
Brian DeFiore, a literary agent, asks: "Is it the cover? The title? The buzz wasn't there? Timing? It wasn't that good?"
The answer is that no one really knows. "It's an accidental profession, most of the time," said William Strachan, editor in chief at Carroll & Graf Publishers. "If you had the key, you'd be very wealthy. Nobody has the key."
The hunt for the key has been much more extensive in other industries, which have made a point of using new technology to gain a better understanding of their customers. Television stations have created online forums for viewers and may use the information there to make programming decisions. Game developers solicit input from users through virtual communities over the Internet. Airlines and hotels have developed increasingly sophisticated databases of customers.
Publishers, by contrast, put up Web sites where, in some cases, readers can sign up for announcements of new titles. But information rarely flows the other way -- from readers back to the editors.
"We need much more of a direct relationship with our readers," said Susan Rabiner, an agent and a former editorial director. Bloggers have a much more interactive relationship with their readers than publishers do, she said. "Before Amazon, we didn't even know what people thought of the books," she said.
Most in the industry seem to see consumer taste as a mystery that is inevitable and even appealing, akin to the uncontrollable highs and lows of falling in love or gambling. Publishing employees tend to be liberal arts graduates who enter the field with a starting salary around $30,000. Compensation is not tied to sales performance. "The people who go into it don't do it for the money, which might explain why it's such a bad business," Mr. Strachan said.
Eric Simonoff, a literary agent at Janklow & Nesbit Associates, said that whenever he discusses the book industry with people in other industries, "they're stunned because it's so unpredictable, because the profit margins are so small, the cycles are so incredibly long, and because of the almost total lack of market research."
Publishers do engage in limited numbers crunching. In estimating value, editors rely heavily on an author's previous sales or on sales of similar titles. Based on those figures and some analysis -- about the popularity of the genre, the likely audience, the possible newsworthiness of the topic of the economy -- they work up profit and loss projections.
The advance payment to the author is often an estimate of the first year's royalties, usually 10 percent to 15 percent of expected sales. The advance is a liability for the publisher because it is a fixed cost. It doesn't have to be repaid by the author if it turns out to be an overestimate, which it usually is. But when earned royalties exceed the advance amount, the author is paid more.
Calculating the advance accurately would be a prized skill, but no editors claim to have a scientific handle on how a book will sell. Instead, they emphasize the role of intuition and say that while big unexpected losses and gains do happen, somehow it all works out.
But results are not spectacular, for an industry that had $34.6 billion in net revenue in 2005. Net profit margins hover in the mid-single digits for the $14 billion trade segment, which covers adult, juvenile and mass market titles, with an estimated 70 percent of titles in the red.
Sales in the trade segment (which includes both fiction and nonfiction) grew 5 percent in 2005 from the previous year, but year-over-year sales growth is expected to decline to less than 2 percent by 2010, according to book industry trade group data. The industry does follow trends to pursue growth, but when it comes to acquisitions, methods have not changed much in hundreds of years, says Al Greco, a professor of marketing at Fordham University.
IT'S the way this business has run since 1640," he says. That is when 1,700 copies of the Bay Psalm Book were published in the colonies. "It was a gamble, and they guessed right because it sold out of the print run. And ever since then, it has been a crap shoot," Professor Greco said.
There is a "business model" that supports this risk-taking. As Mr. Strachan puts it, "Lightning does strike."
And so it must. To make money, the industry depends on perennial sellers and on best sellers. It's not so much the almost sure-fire best sellers by the well-known authors, because those cost so much to acquire and market, but the surprise best sellers. Those include books like "Prep," "The Nanny Diaries" (bought for $25,000, it sold more than four million copies), "Marley and Me" (bought for $200,000, sold 2.5 million copies) and "The Secret" (bought for less than $250,000, sold 5.25 million copies in less than six months).
"They're the ones we all hope and pray happen to us one day," says Judith Curr, the publisher of Atria Books, which printed "The Secret," a self-help book that explains "the law of attraction."
After seeing the movie version of "The Secret," which existed before the book, Ms. Curr estimated that the planned book could sell a million copies. Based on what? "Just a feeling," she said. She described it as a tingling that went up her spine.
All of the major houses have also lost on big bets. One of the highest advances ever paid was more than $8 million for a proposal that became "Thirteen Moons," the second novel by Charles Frazier. He is the author of "Cold Mountain," which sold 1.6 million copies in hardcover.
Random House printed 750,000 copies of "Thirteen Moons" for the hardcover release in October. The book became a best seller, but it has sold only 240,000 copies so far, according to Nielsen BookScan. That would account for less than $1 million of earned royalties, under standard contract terms. The paperback will be out next month, further diminishing hopes of selling out even the initial hardcover print run.
"It's guesswork," says Bill Thomas, editor in chief of Doubleday Broadway. "The whole thing is educated guesswork, but guesswork nonetheless. You just try to make sure your upside mistakes make up for your downside mistakes."
Downside mistakes are more risky when they come with higher advances. These result from bidding wars or when a house decides that a project is so special it is worth acquiring at nearly any cost.
"On rare occasions, you love it, bypass the numbers crunching, do back of the envelope in your head, call the agent and get in early and offer half a million," Mr. Thomas says. That can lead to auction fever.
"That's often when the business model of this industry falls to pieces," Mr. DeFiore, the agent, says. "Because publishing houses are paying high six and even seven figures in hopes those books will turn into the megahits they need, but some will and some won't, so those big bets are dangerous."
In the case of hardcovers, a few books that the publishers think have best-seller potential are promoted with generous marketing and publicity campaigns. Others are considered long shots, with anticipated sales of maybe a few thousand copies. Most are considered midlist, with respectable sales of 15,000 to 20,000 copies, Mr. Greco says, but not breakout sales.
Titles can shift categories unexpectedly. "Nobody in publishing is smart enough to know which of the big books will be fiascos, which of the little books will be successes and which in the middle might go up or down," Mr. Thomas says.
There are two ways for a book to become a best seller. One is to make it on to a best-seller list by selling many copies in a week. Other books sell steadily over months and years, eventually outselling many official best sellers. "Unanswered Cries," a true-crime book by Tom French, was acquired in 1989 by St. Martin's for $30,000. It now has 400,000 copies in print in paperback and sold at least 31,000 copies last year alone.
These older titles are crucial to profits because marketing and acquisition costs have usually already been recouped. Yet publishers focus much more attention on the hardcover release. As they prepare to introduce a book, any combination of timing, packaging, marketing and other factors could help ignite the mysterious spark that leads to best-sellerdom.
Contributing to the success of "The Secret," for example, could be any or all of the following: the subject, the title, the initial DVD release, the marketing campaign, the power of the Internet, "Oprah" and the latest trend.
But "The Secret" is only the most recent book of its type. "Many people have spent lots of money trying to publish the book ėThe Secret' has become," says Susan Petersen Kennedy, president of the Penguin Group. "A lot of books are just like that that haven't worked."
The same uncertainty surrounded "Prep." When Ms. Sittenfeld was writing the novel, she recalled, colleagues said, "The boarding school book has already been written. Why are you doing it again?"
But after it became a best seller, Ms. Sittenfeld said, she heard the opposite: "Of course it did well! It's a boarding school book!"
The publisher of "Prep" attributes the success, in addition to the story, to a catchy title and book cover and creative marketing and publicity. A team of four publicists made belts that matched the cover for giveaways, and sent splashy gift bags (holding pink and green flip-flops, the belt, notebooks, lip gloss) with the galleys to magazines. The pitch letter included photocopies of the publicists' own high school yearbook photos.
"It got attention, and we started getting calls wanting more and more galleys," says Jynne Martin, one of the publicists.
An impressive lineup of press coverage followed. The book ended up with coveted crossover market appeal: in addition to young women, the book was read by adolescents, more mature readers and males, according to anecdotal evidence. Information about readers is often anecdotal because publishers argue that market research would be too expensive, or too difficult to pull off because one book is so different from the next.
"Some of the best and most interesting books have something contrarian that does surprise and delight. says Susan Weinberg, publisher of PublicAffairs.
TAKE the surprise of "Skinny Bitch," a diet book by Rory Freedman, a former modeling agent, and Kim Barnouin, a former model. "The voice is funny, tough love, no nonsense," the authors' agent, Talia Rosenblatt Cohen, said. But the authors were largely unknown and advocate a vegan lifestyle. "Everyone kept saying, ėNo platform!' and ėVegan!' " Ms. Rosenblatt Cohen said.
The book sold to Running Press in Philadelphia for barely over five figures and has not received much mainstream press since its publication 16 months ago. Word traveled among vegans and college students, however, and it became a Los Angeles Times best seller. More than 100,000 copies are now in print, and the authors have signed a deal for two more books with Running Press for "well into the six figures."
Some experts wonder if book publishers might uncover more books like this if they tried harder to find out more about their buyers and what they want.
"The Newspaper Association of America has a staggering amount of data on people who read newspapers. The book business has, basically, nothing," said Professor Greco. "They're not going into the marketplace and doing mall intercepts and asking people, as they leave the bookstore, ėWhat did you buy? Did you find what you're looking for? What motivated you to choose that book?' "
An exception is the consumer research gathered by the Romance Writers of America, a writers' association that publishes a regular market study of romance readers. It reports survey information on, for example, demographics, what respondents are reading, where they are getting the books and how often, and what kind of covers attract them. Romance authors and publicists use the information to create promotional campaigns.
Most publishers, though, continue to gather data on sales and not much else, though past performance is certainly no guarantee of future results, even from the same author.
After "Prep" became a best seller, Random House signed a two-book deal with Ms. Sittenfeld for a multiple of her $40,000 "Prep" advance that she would not reveal.
Her second coming-of-age novel, "The Man of My Dreams," was published last May, and again the advance payment and sales haven't matched up. According to Nielsen BookScan, "The Man of My Dreams" has sold 36,000 copies in hardcover and 6,000 in paperback.
Ms. Sittenfeld says she is reminded of something she heard from an editor: "People think publishing is a business, but it's a casino."
Editors' note: The editor of the Sunday Business section is under contract to Random House and did not edit this article.