◆ Semantic Web Catalogue

Introduction

In 2001 three researchers described a Web whose pages machines could understand as well as people, so that software could act on them for us. Parts of that idea now run through search engines, encyclopedias, barcodes and digital IDs. Other parts were abandoned, and some have come back with AI. This page tells the story briefly and points you onward.

How the story unfolded

The dates below are rough, and the periods overlap.

Organizing knowledge (1870s–1940s)

Librarians and documentalists built the first shared systems for finding knowledge. The Dewey Decimal Classification of 1876 gave every subject a number; S. R. Ranganathan’s Colon Classification (1933) built subjects from combinable facets; and Paul Otlet’s Traité de documentation (1934) imagined a universal network of documents that people could consult from afar. These ideas of shared terms, faceted description and a network of records come back throughout the story.

Computing arrives, and what it let people imagine (1945–1980s)

Electronic computers appeared at the end of the Second World War, and almost at once people began imagining what they might do with knowledge. Vannevar Bush’s As We May Think (1945) pictured the memex, still a desk of microfilm, that linked a person’s records by association into trails others could follow. Others imagined with the machines themselves. Douglas Engelbart set out to augment human intellect (1962) and showed NLS in 1968, with links, a mouse and shared screens. Ted Nelson began Xanadu in 1960, coined the word hypertext and urged everyone to take up computers in Computer Lib (1974). Paul Baran’s study (1964) showed that a network with no centre survives damage that would cut a centralized one apart.

Others imagined machines that know and reason. John McCarthy’s Programs with Common Sense (1959) proposed a program that deduces what to do from facts written in logic; semantic networks and Marvin Minsky’s frames (1974) drew knowledge as linked concepts; and Cyc (1984) set out to write down common sense by hand. Database researchers gave data a formal shape with Edgar Codd’s relational model (1970) and Peter Chen’s entity–relationship model (1976). Codes that machines could read spread into everyday life: library records in MARC (1965) and barcodes on groceries with the UPC (1973). By 1987 HyperCard let ordinary computer users build their own linked stacks of cards.

The Web and the semantic web vision (1989–2006)

Tim Berners-Lee’s proposal of 1989 became the World Wide Web: documents linked by addresses anyone could create, with no central control. He wanted machines to read the links’ meaning too. His road map (1998) layered logic and trust over the Web, RDF (1999) gave a way to state facts as simple three-part statements, and the 2001 Scientific American article The Semantic Web by Berners-Lee, James Hendler and Ora Lassila promised software agents that would book appointments and compare services by reading data published across the Web.

The W3C standardized the pieces, among them RDF Schema and the ontology language OWL in 2004. Critics answered at once. Cory Doctorow’s Metacrap (2001) argued that people lie, are lazy and disagree, so reliable metadata will never exist; Clay Shirky’s essay (2003) argued that real-world data does not fit tidy logic. In 2006 Berners-Lee and two colleagues, revisiting the 2001 article, wrote that its simple idea remained largely unrealized.

Platforms and linked data, side by side (c. 2003–2014)

While the vision was being argued over, the Web became where ordinary people lived much of their lives. Social networks such as Myspace (2003), Facebook (2004) and Twitter (2006) turned people themselves into data: profiles, friendships, posts, hashtags and likes. Smartphones put the Web in people’s pockets after the iPhone of 2007, and Facebook reached a billion users in 2012. A platform account became the name by which people were known online, and “Log in with Facebook” the login, while open alternatives such as FOAF and OpenID stayed niche. Facebook’s Open Graph protocol (2010) put typed metadata into millions of pages so they could become things in its graph. Behind the platforms, big data grew: Google’s MapReduce (2004) and the open-source Hadoop (2006) let companies process logs and clicks across thousands of machines, and in 2009 three Google researchers argued that simple models fed with enormous data beat elaborate ones.

At the same time the semantic web’s own focus shifted from reasoning to publishing. Berners-Lee’s Linked Data note (2006) set four plain rules for putting data on the Web, and projects such as DBpedia (2007) turned Wikipedia into data, at the centre of the Linked Open Data cloud. The query language SPARQL followed in 2008. Then the search engines took it up: Google’s Rich Snippets (2009) read markup in pages, and Google, Microsoft and Yahoo launched Schema.org in 2011. In 2012 came Wikidata and Google’s Knowledge Graph, “things, not strings”, and JSON-LD (2014) let ordinary web developers publish linked data as JSON.

Learning from data, and the reckoning (c. 2012–2022)

Machine learning turned the data the platforms had gathered into products. AlexNet (2012) turned the field toward deep learning, word2vec (2013) learned word meanings from how words occur together, and TransE (2013) put knowledge graphs into the same kind of vector space. Google’s Knowledge Vault (2014) built a knowledge base by machine, giving each extracted fact a probability. The transformer (2017) and BERT (2018) followed, and Google used BERT in Search from 2019. Knowledge graphs became company infrastructure, and the largest builders compared notes in 2019.

The same concentration of data brought a reckoning. In 2013 documents leaked by Edward Snowden revealed PRISM, through which the US National Security Agency collected communications from Google, Facebook, Apple and others. In 2014 the EU’s highest court gave people a right to be forgotten by search engines, and in 2016 the GDPR gave them rights over any data about them. In 2018 the Cambridge Analytica story showed how data on tens of millions of Facebook users had flowed to a political consultancy, and Facebook cut back what outside apps could read. Others answered by taking up the Web’s original promise of no central control: Solid keeps people’s data in stores they choose, ActivityPub (2018) links social networks across servers, and Verifiable Credentials (2019) and Decentralized Identifiers (2022) let anyone sign and check claims without a central authority.

Language models and agents (2022–)

Language models had been growing in research labs since the transformer, but when OpenAI released ChatGPT on 30 November 2022 it reached a hundred million users in two months, and large language models became the main way software works with text. They changed the questions this catalogue follows. Models write fluently but can be wrong, so knowledge graphs returned as something to ground and check them against, as in GraphRAG (2024). The 2001 vision’s software agents arrived in a new form: AI systems that call tools and data through protocols such as MCP (2024), and the W3C’s 2026 paper argues that structured data, policies and credentials matter more as agents act for people. Whether this fulfils the semantic web, replaces it, or makes it unnecessary is still open, and this part of the story may yet change shape.

Where you already meet it

  • Search results. Star ratings, prices and event dates under a search result come from Schema.org markup in the page.
  • Wikidata. Some Wikipedia infoboxes draw their facts from Wikidata, an open knowledge graph that anyone can edit and reuse.
  • Products. The barcode on a pack carries a GTIN, and the newer 2D codes can carry a web address for the product under GS1 Digital Link.
  • Libraries and museums. Catalogues are moving from MARC records to linked data such as BIBFRAME, and images are shared through IIIF.
  • Credentials and photos. Verifiable Credentials let someone show a signed claim, such as a diploma, that anyone can check, and Google’s Pixel 10 camera signs every photo with Content Credentials.
  • The fediverse. Mastodon and its neighbours talk to each other over ActivityPub, built on the same family of standards.

Five things to read first

  1. The Semantic Web (2001): the vision as it was sold.
  2. Metacrap (2001): the sharpest early objection.
  3. Linked Data (2006): the four rules that made the idea practical.
  4. Introducing the Knowledge Graph (2012): how it reached everyone through search.
  5. Industry-scale Knowledge Graphs (2019): what the largest builders learned.

All 117 works the catalogue counts as essential are in the canon.

A few words you will meet

  • URI: a web address used as a name for anything, not only for pages.
  • Triple: one statement in three parts, subject, property and value, such as “Paris, capital of, France”.
  • RDF: the standard way of writing triples so they can be joined across sources.
  • Ontology: an agreed set of terms and rules for a field, so different sources mean the same thing by the same word.
  • Linked data: data published with URIs and links, so one dataset can point into another.
  • Knowledge graph: a large network of such statements about things and their connections, as companies and Wikidata keep them.

Where to go next

  • Where we are: for each design question, what is settled, what is open, and what pushes it.
  • Problem threads: follow one question from its first proposal to today.
  • The board: what is on now, and where to join in.