The Oracle and the Librarian

Spreading bitter disappointment that a "Bro Split" doesn't involve Mark Zuckerberg and a samurai sword, Google launched its ad campaign for "AI" search in the UK this week.

Along with everyone who ever read more than three thick books, the advertisement provoked that sinking feeling in me, nausea at the spectacle of humanity circling the drainpipe down to the sea of vacuous banality.

To understand why "AI" is not a technological breakthrough but an audacious political movement to redefine our relationship with computers and with truth, let's think about two characters, the Librarian and the Oracle.

You remember "The Oracle" of course. Sweet yet mysterious cookie-baking Gloria Foster of Matrix fame. If the Wachowskis left any semblance of The Greek in her it was the power of oblique divination. Like all seers and soothsayers, oracles speak in tongues, amplifying ambiguity as much as information. Like the mystic horoscope writer, the economic alarmist who correctly predicts ten out of four crises, or a stopped clock, she is assured of serendipitous correctness.

Nobody remembers the librarian. The librarian is not so much grey as transparent, with sensible shoes and a pencil skirt. As Dita Kraus or Sayuri Komachi, in fiction, their wisdom is highly tailored, to "help people find what they are looking for".

In some ways the librarian is the opposite of the enigmatic oracle. Those who remember the old days of academia will have encountered a gentle, magical "information therapist", who seemingly knew not just every book in the building by Dewey Decimal number (and a dozen other categorical frameworks), but walked you to the precise shelf that changed the course of your research and career. She then turned to help another student in a field so highly specialised that you cannot even pronounce it, yet with the same authoritative poise. How is that possible? How does one invisible woman, paid on par with the cleaning staff and not even on the faculty, know nuclear physics, biochemistry, discrete mathematics and modern art history? The reveal is that epistemological knowledge, the meta-understanding of knowledge itself, is a separate and powerful art.

Unsurprisingly the list of notable librarians is somewhat longer and better organised than the list of great oracles, being written by librarians, or rather by those types who catalogue, curate and care about writing. Librarians like Wikipedians, have both a great love of knowledge and significant power, since by structuring, preserving and shepherding it they also shape knowledge, whether by inclusion, omission or how they reference items. It is philosophically arguable but fair to say that knowledge does not exist outside the structures that hold it.

To see where Google were once the good guys - and hence how far into the fire they have fallen - let's go back to the origins of the Internet… {{ wavy lines}}…

From 1984, JANET, the Joint Academic Network, had as much clout in the origin story of the Internet as DARPA or the French MiniTel, in other words it was an extremely important cultural component that almost nobody has heard of today. Information sciences and academic information services were the birthplace of many concepts we take for granted today. JANET first facilitated the interconnection of university libraries.

The idea that knowledge could be collected, cross-referenced, catalogued, indexed, searched, and that this activity added further "meta-informational" value to the corpus is not a modern "STEM" idea but a development of ancient literary hermeneutics. The analysis and interpretation of comparative authors (authorities), including origins, influences, affiliations, context, intent, and the fate of various writings, is what defines our canon and culture.

Having dumped it's commitment to not being "Evil", Google, again without a jot of hubris, assumed responsibility for "organising the world's information". To see why their "AI" project continues the company's remarkable tradition of abandoning pithy mission statements, we need to see how "AI" in its present form is much more like The Oracle than The Librarian.

It is a real question whether the word "organised" applies at all to weights obtained under gradient descent machine learning. Such an "AI" is a inscrutable heap of floating point numbers, a heterogeneous array of loosely associated stuff.

Behind the gargantuan, perhaps over-reaching project of the library, is the simplest of things. The book. Or rather, any concrete work that has author(s), and can be referenced as an object. Indeed the entire foundation of modernity comes from that peculiar structured mini-writing we call the "Scientific Paper", now locked behind the doors of vast corporate publishers and rarely seen by the eyes of mere mortals except under threat of punishment.

A foundational physical object like a book meets simple but essential verificatioin criteria, such that two people can sit down at the same reading table and assure each other that the same text exists. That's more that can be said for a remote machine, owned and run by someone else, that cannot even be guaranteed to emit the same output on different runs.

As an object, the book or paper is the destination, or leaf node in an activity we called "search", or "research". The digital age brought "documents" and books into new forms, as "web pages", but we kept the essential idea of them as objects. URLs (Uniform Resource Locators) imply a "Resource" that has a "Location".

A traditional search result that returns a book title and author comes with the implied promise that the book and author exist somewhere, even if its accessible representation is digital.

The function of the WWW (World Wide Web) was to knit these resources together into a navigable system with a single interface. Only, in their rush to implement it our formative web-weavers Sir Tim and friends forgot the "navigable" bit. Academics, who were once carey-sharey types, would presumably do the work of impartially linking together all these millions of documents.

Google's historic role in the WWW is essential to understand here. Without search engines the Web was stuck. Many great engines emerged in the late 1990s, but one dominated, with its clean, fast interface and better indexing algorithm. Nonetheless, throughout all of its evolution, the purpose of Google Search has been to connect seekers of information with documents… with books of the modern kind. Google wished to be the global village librarian.

Teachers, detectives, lawyers, analysts and those of us who dabble in epistemology, in ideas like origins, references, evidence, provenance, interpretation, and so on, put a lot of stock in libraries and archives. We are as interested in the structural reality of knowledge, in the processes and storage of records, as in the "bare facts" themselves.

With bourgeois professions dropping out of fashion faster then "Jim'll Fix It" badges, Google - ever ready to hoist the populist flag and bury old, loyal, but no longer needed friends - is moving on. Indeed Google officially disavowed the World Wide Web.

"AI search", is not a "Web Search". You are not searching or asking "The Internet". It's a change to a completely different model of information retrieval and processing, and an abandonment of commitment to verifiable, objective "truth" implied in the librarian model.

Sweet though her cookies are, The Oracle is no librarian. She is a literal portal for the Gods to speak. And Gods do not speak plainly. Importantly, the answers one gets from an oracle depend, in a much more slippery way, on the phrasing and intention of the questioner. They come along with ironic side effects and concealed influence agendas. They depend on who is asking. The oracle plays cold-reader, and she must please or mislead the enquirer in so many dark ways known only to the Gods.

A word that captures the difference between these models of information is Romantic. Much as I loved big hair and eyeliner in my Goth days, and think the world could benefit greatly from a less literal and positivist stance, my scientist mind thinks conflating these models of truth, by conflating neural and structural/procedural knowledge, is dangerous and reckless. Google and friends are epistemically naive. "AI search" may look superficially like indexing actual documents, but it could hardly be more different.

The line between "summary" and "interpretation" is neither trivial nor clear. Disappearance of link provenance, of information breadcrumbs permitting referencing and deep knowledge exploration are extremely worrying. There's also a legal shift going on. Because the Web is ephemeral, search results, even of scientific papers, have never been a solid basis of evidence, (despite the rubbish we still teach graduates in Research Methods classes). "AI" marks a shift similar to that between journalistic reporting of facts versus an opinion column. Whereas before Google might claim no responsibility for information it links to, that is most definitely not true of parochial and biased "AI" summaries which are "authored" by the "AI" search provider.

The librarian has no stories of her own to tell. She is an accountable guide. The Oracle is a story-teller, and is very much a part of the story.

Keep this in mind as you type requests for information into "AI" tools that look like familiar "search engines".