Grey Local Site AI Assistant

Description

Local Assistant adds a small chat panel to the site. The visitor asks a
question, the plugin finds the most relevant passages in the site’s own pages,
and a small language model that runs in the visitor’s browser writes the answer.
The answer cites the page it came from. If the pages do not answer the question,
the assistant says so instead of inventing one.

The site owner builds the search index once from the dashboard: the plugin walks
the sitemap, extracts the readable text of each page, and saves a searchable
index plus its vectors into wp-content/uploads/local-assistant/. Nothing is
sent to a server at any point.

• Answers come from your pages. Retrieval runs against the index built from
  this site; the model is instructed to refuse when the passages do not contain
  the answer.
• The model runs on the visitor’s device. No server-side model, no API key,
  no per-answer cost, and question text never leaves the browser.
• Works offline after the first visit. A service worker caches the widget
  and the index, so a returning visitor can ask again without a network.
• Two engine paths. WebGPU is used when the browser offers it; a small CPU
  model is the fallback, and a plain search result is the last resort. The
  switch is silent — the visitor sees an answer, not engine jargon.

External services

The plugin itself calls no external service, and the site server never sends
visitor data anywhere. On the visitor’s device, the browser downloads model
files the first time the assistant is used, after the visitor accepts the
consent card:

• WebLLM (Apache-2.0) fetches the conversation model, by default
  SmolLM2-360M-Instruct-q4f16_1-MLC, from the MLC model hub on Hugging Face.
• Transformers.js (Apache-2.0) fetches the embedding model
  Xenova/all-MiniLM-L6-v2 and the CPU model HuggingFaceTB/SmolLM2-135M-Instruct
  from Hugging Face.
• ONNX Runtime Web (MIT) is bundled in the plugin; its WASM runtime is
  loaded from the plugin folder, not a CDN.
• WebLLM’s model library is a WebAssembly module the engine compiles on the
  visitor’s GPU. It ships inside the plugin
  (dist/vendor/web-llm/SmolLM2-360M-Instruct-q4f16_1_cs1k-webgpu.wasm) and is
  loaded from the plugin folder, so no executable code is fetched from a third
  party at runtime.

Those downloads are the only network traffic the assistant causes, they are
started only after consent, and they go directly from the visitor’s browser to
Hugging Face under Hugging Face’s own terms. The site owner can point the
downloads at another host with the modelBaseUrl, modelLibUrl, and
modelMirror settings.

See it live

The assistant runs on the author’s own site — ask it something and watch it
cite the page it read:

• Live demo: randybarham.com (the chat panel opens
  on every public page).
• Full readable source, build scripts, and tests:
  github.com/greyok00/local-assistant.

Source Code

The WordPress package is generated from readable sources, and the full
build is public:

• The readable sources and build scripts live at
  github.com/greyok00/local-assistant
  — src/ holds the plugin’s JavaScript modules, scripts/ the build tools.
• This zip also carries a readable copy of every script it was built from
  in its source/ folder.
• To build the package from source: npm ci && npm run vendor && npm run
build (Node.js 20+), then npm run release for versioned zips.
• The third-party libraries in dist/vendor/ are Transformers.js
  (Apache-2.0, sources) and
  WebLLM (Apache-2.0, sources); exact
  versions are listed in the included THIRD-PARTY-NOTICES.txt.

The two .wasm files flagged in review are required, and both are loaded
only from the plugin’s own folder — the plugin never fetches executable
code from a third-party host:

• dist/vendor/transformers/ort-wasm-simd-threaded.jsep.wasm is the ONNX
  Runtime Web engine that runs the embedding model and the CPU fallback
  model inside Transformers.js; the plugin cannot function without it.
• dist/vendor/web-llm/SmolLM2-360M-Instruct-q4f16_1_cs1k-webgpu.wasm is
  WebLLM’s compiled GPU program for the SmolLM2-360M-Instruct-q4f16_1
  chat model; it is the WebGPU conversation engine itself. The fetched copy
  is tracked in the repository under scripts/vendor-extra/.