Extracting a durable knowledge graph from a full book using a local LLM, Onya, and merge-safe analytics — no cloud model required.
Knowledge graphs are powerful tools for GenAI context. Learn how to work with them using a small, private/local model, building knowledge structure from large documents, and treating the graph as a full-blown system of record rather than a snapshot.
You may not need a frontier cloud model to turn a shelf of documents into a knowledge graph. Responding to Laurent Picard's Gemini-based walkthrough, the author extracts an entity/relationship graph from a full public-domain book using only a local model reachable through the plain OpenAI-style API, and the Onya graph format and library, which he develops at Oori Data. Unlike a networkx-and-matplotlib pipeline that computes, renders, and discards, Onya treats the graph as a durable, mergeable, queryable system of record: extraction results merge idempotently across document chunks, persist to a SQLite or PostgreSQL store, and analytics computed via a networkx projection are written back into the graph as first-class typed assertions.
Given a long document from Project Gutenberg and only a local model: extract a knowledge graph of characters and relationships within an honest local context window (not an 800k-token single request), keep the graph as a queryable, persistent, growable artifact, and run real graph analytics while keeping those results in the graph too.
Point an OpenAI-compatible client at a local inference server such as oMLX. Relationship extraction wants more model heft than entity extraction; the author lands on Qwen 3.6 35B-A3B, a mixture-of-experts model with 35B total parameters but ~3B active per token, needing ~26GB RAM at 6-bit. A smaller Qwen 3.5 4B (8-bit) model still yields a decent entity graph but will not wire relationships together. See also the author's earlier MLX day-one article and mlx-notes repo.
Onya's conceptual model: a node has an IRI identifier, a set of types, and a set of assertions. An assertion is either a property (label IRI to string value) or an edge (label IRI to target node); assertions can themselves carry assertions. Onya Literate, a Markdown dialect, is the serialization that makes this practical for LLM work: a docheader block sets the document IRI and bases, '# NodeID [Type]' blocks declare nodes, '* label: value' declares a property, and '* label -> Target' declares an edge.
Comparing token counts on the Things Fall Apart sample graph with the Qwen 3.6 tokenizer: pretty JSON costs 418 tokens, compact JSON 224, TSV 96, and Onya Literate 187 -- more than halving pretty JSON while remaining self-describing and diff-friendly, unlike TSV's disconnected integer-id tables that must be reassembled by hand.
Onya Literate is not a JSON schema and the sampler is not constrained (3SO); instead the article uses validation by parse, with the parser's own errors fed back as repair feedback. Prompts live in a separate WordLoom TOML file so they can be reviewed, diffed, and swapped without touching Python. See also the author's LLM power steering piece.
A local model sees only ~32k tokens at a time, so documents are chunked. Onya's answer to cross-chunk entity identity is structural: an entity's IRI is minted from its name under a stable nodebase, so the same name in different chunks resolves to the same node, and merge() collapses duplicate occurrences under the spec's identity rules, making extraction an idempotent graph union.
Onya has a pluggable persistence layer whose correctness criterion is that a round trip through a store must be indistinguishable from an in-memory graph union. SQLite ships dependency-free in the box; a PostgreSQL backend is available for scale, with the same merge semantics.
Onya's onya.serial.nx module projects the graph into networkx, unlocking betweenness centrality and Louvain community detection. Results are written back into the graph as typed, merge-safe assertions via an explicit data contract, rather than evaporating in a notebook.
Rendering is deliberately not Onya's job, but from the networkx projection a matplotlib figure is a short walk. Onya's CLI can also export Mermaid or Graphviz DOT directly from a Literate file for a quick structural look.
Honest caveats: the parse-and-repair loop is a statistical comfort, not a guarantee -- two of ten chunks skipped on one run. Extraction quality degrades ungracefully as model size shrinks, and the networkx projection is deliberately lossy in this first version.
The pattern in a trice: prompt a local model to emit Onya Literate, validate by parsing and repair with the parser's own errors, chunk long documents and let IRI identity plus explicit merge() make extraction an idempotent union, checkpoint to a store, project to networkx for analysis, and write the results back as typed assertions so the analysis is part of the record.
Author of the HackerNoon article 'Building Knowledge Graphs with Gemini', which prompted this response article.
Also known as Gustavus Vassa: a kidnapped Igbo child who bought his own freedom and became a founding voice of British abolitionism; subject of the memoir used as the article's full-scale extraction t
Marine artist; painter of 'Racehorse progress', a detail from which illustrates the article's Equiano section.
The company where the author develops the Onya knowledge graph model and format.
Uche Ogbuji's technology and literature blog, where this article was published.
Onya is a knowledge graph model and format developed by Uche Ogbuji at Oori Data, in which nodes have IRI identifiers and assertions (properties or edges), and the graph is treated as a durable, mergeable system of record rather than a disposable computation.
Onya Literate is a Markdown-dialect serialization of an Onya graph: a docheader sets the document IRI and bases, '# NodeID [Type]' blocks declare nodes, and '* label: value' / '* label -> Target' lines declare properties and edges -- self-describing, diff-friendly, and directly parseable.
Running inference locally through an OpenAI-compatible API keeps potentially private information on the author's own machine, works with any local stack (llama.cpp, vLLM, LM Studio, Ollama, oMLX), and demonstrates that a shelf of documents can become a knowledge graph without a frontier cloud model.
Qwen 3.6 35B-A3B at 6-bit: a mixture-of-experts model with 35 billion total parameters but only ~3 billion active per token, giving near-larger-model quality at small-model speed. Entity extraction works with a tiny 4B model, but relationship extraction needed the larger model's heft.
Entity identity is an IRI minted from the entity's name under a stable nodebase, so the same name appearing in different document chunks resolves to the same node by construction; an explicit merge() operation then collapses duplicate occurrences under the spec's identity rules.
It is the identity problem of one real person appearing under different names: Olaudah Equiano is also called Gustavus Vassa in his own memoir's title, so chunks drawn from different life periods mint two distinct nodes that must be reconciled with an alias_of-style edge.
The model's Onya Literate output is parsed with Onya's LiterateParser; on failure, the parser's own error message (naming the offending token and suggesting a fix) is fed back to the model as a repair prompt, for up to two retries, before the chunk is skipped.
After projecting to networkx and computing metrics like betweenness centrality and Louvain communities, the results are written back as typed, merge-safe Onya assertions using an explicit data contract (interp=ONYA_INTERP('number')), so the analysis becomes part of the persisted record rather than a disposable figure.
Onya ships with a dependency-free SQLite backend by default, and a PostgreSQL backend for when you outgrow the laptop, both sharing the same merge semantics via store.put(doc_iri, graph, merge=True).
The parse-and-repair loop is a statistical comfort, not a hard guarantee (two of ten chunks skipped on one run); extraction quality degrades with model size; the small local models can mistype entities (a ship or place tagged as a Person); and the networkx projection is deliberately lossy in its first version.
Laurent Picard's HackerNoon article 'Building Knowledge Graphs with Gemini', which walked through extracting entity/relationship graphs from entire books using Gemini's large context window; the author sets out to walk similar ground with local, private models instead.
'Onya' comes from the Igbo word 'onya' (web, snare, and by extension network); the expanded phrase 'onya uche' means a web of knowledge, reflecting the graph's role as a durable knowledge structure.
A Markdown-dialect serialization of an Onya graph: self-describing, human-legible, and directly parseable, with no integer-id reassembly step.
Constraining an LLM sampler so structured output is guaranteed rather than merely requested, discussed in the author's earlier 'power steering' article.
The identity problem of one real-world entity appearing under multiple names across a document -- e.g. Olaudah Equiano also called Gustavus Vassa -- producing distinct nodes that must be reconciled wi
A knowledge graph model and format, from Igbo 'onya' (web, snare, network), designed so the graph itself is a durable, mergeable system of record.
The Markdown-dialect serialization of an Onya graph, using docheader, node, property, and edge conventions that a language model can readily emit.
A TOML-based prompt file format that keeps LLM prompts separate from application code for review, diffing, and version control.
A native local inference server for Apple Silicon offering continuous batching and tiered KV caching, exposed through an OpenAI-compatible API.
Constraining a model's sampler so structured output is guaranteed by construction, contrasted in the article with the parse-and-repair approach used for Onya Literate.
An explicit, on-demand Onya operation that collapses duplicate node occurrences accumulated from unioning multiple extraction passes, under the format's identity rules.
The stable IRI base declared in an Onya docheader against which node identifiers are resolved, making the same name in different chunks resolve to the same node.
A networkx graph metric measuring how often a node lies on shortest paths between other nodes; used in the article to identify the memoir's central figure.
A networkx community-detection algorithm used to cluster the extracted social graph into groups such as the Montserrat trading circle and the Phipps expedition officers.
A model architecture, such as Qwen 3.6 35B-A3B, with a large total parameter count but only a small fraction active per token, trading memory for near-larger-model quality at small-model inference spe
The recurring identity challenge of one entity appearing under multiple names in a document, illustrated by Olaudah Equiano also being named Gustavus Vassa.
The opening '# @docheader' block of an Onya Literate document, declaring the document IRI, nodebase, and schema against which the rest of the file resolves.
Interactive graph visualization derived from the companion RDF. Click nodes to resolve, drag to explore. Graph data embedded from companion RDF at generation time.
A ready-to-run entity-type summary query against the named graph, once the companion RDF is uploaded to URIBurner.
text/x-html+tr; DESCRIBE/CONSTRUCT queries render as text/x-html-nice-turtle.
This knowledge graph overview was built by extracting the article at Loomiverse into RDF-Turtle using the kg-generator skill, then rendered as this interactive infographic using the rdf-infographic-skill, powered by Claude Sonnet 5 running on Claude Code. The companion RDF file is intended for upload to the URIBurner-hosted Virtuoso quad store as named graph https://linkeddata.uriburner.com/DAV/demos/daas/onya-knowledge-graphs-local-llms-claude_sonnet_5-1.ttl; the SPARQL Explorer below queries that graph once uploaded.