AI and the Book Market Machines that Write Novels?

Artificial intelligence is becoming increasingly important in the book industry too. Some hopes are realistic, while others are more science fiction.
 

In years gone by the advance of artificial intelligence (AI) has led to revolutionary developments in many areas. Above all in the economic sector – and within that particularly in sales and marketing – AI has become a powerful instrument that we encounter everywhere on a daily basis, frequently embedded invisibly in apps or websites. People are inspired by this topic: there are over 200 German-language titles and more than 3,500 in English on this subject published by the Springer Science+Business Media publishing company alone. But what about the usefulness of AI in the publishing industry itself – what opportunities are there, what visions for the future?

Artificial Intelligence as a Truffle Pig for Trends?

Typically for this industry, which tends to have a more traditional approach, there’s a face-off between future sceptics and progress enthusiasts: one side fears manipulation of book buyers and readers, whereas the others firmly believe that consumers’ needs could be identified far better in future with the help of AI, which would ultimately also mean meeting those needs. As in all other fields, they are endeavouring to replicate human thought and behaviour using AI in the book sector as well. The idea is to use AI to solve complex tasks and above all to predict behaviour – specifically the purchasing behaviour of book buyers. At the moment this is about selling books that have already been printed, for which it is possible to identify potential buyers very precisely with the aid of algorithms. Anyone who uses websites like Amazon is familiar with this: once you’ve made a purchase, you are shown suggestions for books with similar content every subsequent time you view the site. Not only that, but newsletter and messenger services can also react accurately to buyer needs. For Amazon, Google and Facebook, all their customers are huge piles of data, transparent and open to all possible forms of influence.

But these methods are not new anymore, and artificial intelligence is not required to sell or circulate books in this way – you just need algorithmic processes that have long been a proven practice. The visions of exploiting artificial intelligence kick in a step before that. Which books, trade publishers all over the world are wondering, should we publish next year and the year after? Of course the literature category is ignored here, because these readers follow their preferred writers. They look forward to new books by their favourite authors whose language they love almost regardless of the subject of the story.

But below the literary peak there is that big grey area of genres – historical novels and crime thrillers, reference books and guides. For some years now they have been trying to be as fast and accurate as possible in their predictions of the trend themes for future years in these categories. Metadata searches can be performed in which key words can provide information about what is frequently searched for on the internet, what time delays there are, how many million people are active on certain sites. The conclusions drawn enable publishers to find and commission suitable authors for particular themes. All this is admittedly still in its infancy – and it’s not discussed very openly in the industry either, because after all it contradicts the ideal that selection is solely dependent on the original book idea.

Books Are Written by Humans

Although in the next few years we certainly aren’t expecting any revolutions, and the culture pessimists need not yet fear that AI will shape or even take over book production, reliable studies are finding that the influence of AI will grow in years to come. Back in 2019 the Frankfurt Book Fair published a white paper in collaboration with Gould Finch based on a survey of 233 participants from the publishing sector in 17 countries. A large majority of those surveyed assumes that the importance of AI in publishing will continue to increase. 25 per cent of the publishing companies questioned have already invested in AI. Nevertheless they don’t expect huge leaps: so far no one wants to rush into major investment because it’s by no means certain whether sales growth can ultimately be achieved as a result. Holger Volland, Vice President of the Frankfurt Book Fair, is convinced that publishers can expect most of all increased efficiency as a result of AI, but also opportunities in new business fields – but he does raise a concern: “Admittedly it will be a challenge to integrate new technologies in such a way that the cultural characteristics of the publishing market are taken into account.”

The unique feature of this market is simply human creativity. At a shareholders’ meeting of a publishing company listed on the stock exchange, one shareholder asked why the publisher didn’t include “something along the lines of Harry Potter” in its range of titles – after all that would bring in the money. But no AI in the world would hit on the idea of sending a boy to a boarding school to learn magic, and having him fight a battle there in which good and evil – and his own life – are at stake. Thus far the machine can only identify what’s already there or on the horizon. It can spot trends or popular colours for book covers. But the books are written by humans for humans.

Complex and Costly

But it might be that development has progressed far enough in a few years that AI, whilst not replacing the work of editors, can support them. In theory it’s possible with the help of AI to scan large quantities of text in a very short time and check whether the manuscripts are categorically suitable for the publishing programme. There is a new software tool: LiSA (Literature Screening and Analysis) was developed by the Hamburg firm QualiFiction and can analyse fiction manuscripts on the basis of theme, sentiment and style – it is even capable of identifying whether the texts are more cheerful or gloomy, more complex or simple in their writing approach. Software such as the tool from QualiFiction, which was chosen as start-up of the year in 2019 by the Börsenverein des deutschen Buchhandels (German Association of Publishers and Booksellers), are likely to become established in the publishing sector sooner or later. They already use computer programs to evaluate manuscripts in the USA, reports freelance editor Walter Greulich from the future!publish 2019 congress. But the basic requirement for implementation of AI in publishing is the availability of data. However it’s an incredibly complex and costly process to filter out the relevant information from the numerous heterogeneous systems – reviews, subscription numbers and cancellations, Amazon reader recommendations et cetera. For smaller publishing houses this is an unaffordable extravagance.

Nevertheless in recent years AI has made it possible to modernise – even revolutionise – a directory called Verzeichnis Lieferbarer Bucher (German Books in Print; VLB), in which around 2.5 million titles are recorded. Admittedly even this began with work by employees within the sector, who sustain and review the learning system. A new classification characteristic is starting to catch on here, known as reading motive. In addition to product categories and theme classifications, they open up new opportunities in book marketing by targeting unconscious reading needs like “thrill”, “desire to laugh”, or “be surprised”. Booksellers should be better able to find reading material to suit their customers in future with this new tool.

So the possible applications of AI are diverse: much of this potential is still at an early stage, some aspects will turn out to be viable but much of it will also prove too expensive for the book market – which features a high proportion of small and medium-sized businesses. The algorithms can support the work performed in the publishing companies and bookshops, but it won’t make humans superfluous. The success of a book depends on so many hard-to-gauge factors that producing “sure-fire bestsellers”, the stuff of dreams for some people, can until further notice be filed under one category: science fiction.