Vol. 1 · No. 1

An independent lab for AI on WordPress

Chicago · Est. 2026

WPLLMS

WORDPRESS + LLMS, TESTED ON A REAL SITE

Experiment 004: we put a free chatbot next to Chatvox and asked the same three questions

Hand-drawn illustration of two chat windows side by side on a desk with a small potted plant between them

Written by

in

In Experiment 001 we asked Chatvox three questions about this site. It answered two right away and three of three after one small fix. That left a question we never answered. What does a free chatbot do with the same three questions, with no tuning and no AI service behind it?

We installed WPBot, the free chatbot plugin by QuantumCloud, version 8.9.3. It costs nothing and needs no API key. Then we asked it the same three questions, word for word, and asked Chatvox the same three again.

The three questions

  1. “What is WPLLMs?”
  2. “What was the first experiment on this site?”
  3. “How much did the plugins on this site cost?”

WPBot, out of the box: 0 of 3

WPBot installs from the plugin directory in about a minute. Its own help says that by default only site search works, and direct text answers need Simple Text Responses or an AI service. We used no AI service, so first we ran it with defaults only.

It answered none of the three questions. The greeting asks “May I know your name?” and treats the visitor’s first message as a name. We asked “What is WPLLMs?” first and it replied “This is Carrie! Nice to meet you”, then asked what we were looking for. The first question was gone before the test started.

WPBot, after we wrote the answers: 3 of 3

We added three Simple Text Responses in WPBot, using wording from the About page and the Experiment 001 post. One response per question. After that, WPBot answered all three correctly.

But the answers were the exact text we had typed in minutes earlier. That is correct by construction. WPBot did not know anything about the site. It repeated what we wrote. The honest score is 0 of 3 out of the box, and 3 of 3 as a script you write yourself.

Chatvox, asked again: 3 of 3

We asked the same three questions in the live Chatvox widget, with no new tuning since Experiment 001.

“What is WPLLMs?” got a full answer: an independent test lab for AI tools built for WordPress, installing chatbots, agents, search tools and other AI plugins on real sites and reporting what happened, including costs and failures.

“What was the first experiment on this site?” got a correct answer that named Chatvox and linked the Experiment 001 report. It did not say the words “Experiment 001” in the sentence itself.

“How much did the plugins on this site cost?” got the price, $0, with no hedging.

One small change from Experiment 001: the tuned answer back then named all six plugins. This time Chatvox gave the price and stopped there. Nothing was wrong and nothing was invented.

The verdict

Chatvox answered 3 of 3 with no new tuning. WPBot answered 0 of 3 out of the box and 3 of 3 only after we wrote every answer for it. That is the difference between a chatbot that reads your site and a script that repeats your lines.

WPBot is free and it works, but what it does is FAQ repetition, not answering. If you want a bot that can talk about your site without you writing each answer in advance, the free version of WPBot will not do it.

The scorecard

WPBot out of the box: 0 of 3

WPBot after writing the answers: 3 of 3

Chatvox: 3 of 3, no tuning

Setup effort: WPBot install, a few minutes, plus three responses typed by hand

Failure style: WPBot’s name prompt eats the first question; Chatvox drops the plugin list

Extra cost: $0

Site notes

WordPress is on 7.1.4 and the theme is Twenty Twenty-Five 1.5. Seven plugins are installed, six active. One update is available, GA Google Analytics 20260810 to 20261108, and we did not install it during this run. One side effect of the test: both chat widgets now sit at the bottom right of every page and they overlap. WPBot stays installed for now so the test can be repeated.