GPTZero: When a Bloody Guess Dresses Itself Up as the Truth

GPTZero AI detector accuracy is under scrutiny. Greyhat takes a bloody good look at false positives, misleading results and whether we can trust AI detectors.

Apparently, writing like a human isn’t enough any more. Now you’ve got to convince a machine you’re one.

There’s a particular kind of bollocks that comes wrapped in technology, polished up with impressive percentages and presented as though Moses himself had brought it down from the mountain on a bloody tablet.

And then there’s GPTZero.

For anyone who hasn’t encountered this little digital judge and jury, GPTZero is an AI detection service. Feed it some writing and it’ll attempt to tell you whether a human wrote it or whether a computer did the donkey work.

Sounds useful, doesn’t it?

Well, yes. Until you start wondering how the bloody thing knows.

Because here’s the awkward bit. It doesn’t know. It predicts.

And there’s a hell of a difference between those two words.

Churchill, ChatGPT and a Rather Amusing Accusation

We were working on a short piece about Winston Churchill, leadership and the rather depressing collection of political cardboard cut-outs Britain has endured over the years.

The article included a quotation attributed to Churchill:

“It is better to make the news than to take it; to be an actor rather than a critic.”

We used AI to help knock the article into shape. Then we deliberately roughened it up, making it more conversational, more sarcastic and rather less like something written by a committee of accountants frightened of offending the office furniture.

It included such literary masterpieces as:

“But here’s the thing. The bugger led.”

Not exactly Shakespeare, but it made the point.

Then, for a laugh, we fed the whole lot into GPTZero.

And what did our little electronic Sherlock Holmes announce?

AI 100%. Mixed 0%. Human 0%.

It was highly confident.

Not mildly suspicious. Not sitting on the fence. Not scratching its electronic backside wondering whether the evidence was sufficient.

Highly bloody confident.

Now, to be fair, AI really had helped produce the article. So we’re not going to pretend this was some magnificent scientific demonstration that GPTZero got the entire document wrong.

That would be dishonest, and there’s quite enough of that going around without us adding to the pile.

But here’s what bothered us.

The document included an attributed historical quotation, human editorial choices and deliberately altered wording. Yet the headline result was presented with a confidence that would make a second-hand car salesman blush.

The screen even highlighted the Churchill quotation among the supposedly suspicious text.

Churchill died in 1965.

ChatGPT wasn’t exactly available down at the local library in those days.

The detector wasn’t necessarily claiming Churchill had a subscription to OpenAI. Its overall classification and sentence highlighting are different things. But to somebody looking at that screen, the distinction isn’t exactly shouting from the rooftops.

And that’s where the trouble begins.

A Percentage Is Not a Bloody Birth Certificate

There is something almost magical about putting a percentage beside an assertion.

Tell somebody you’re suspicious of their writing and they might ask why.

Tell them a computer has decided it’s 100% AI and suddenly you’re wearing the robes of a High Court judge.

Except you aren’t.

You’ve got a statistical classification. GPTZero itself explains that its percentage represents confidence in a classification, not the proportion of individual words written by a machine.

That’s a rather important detail, isn’t it?

Because ordinary people see 100% AI and naturally think the machine has established something definitive.

It hasn’t established authorship. It has assessed patterns.

And patterns can be misleading.

A human can write predictably. An AI can write unpredictably. A person can be boring, repetitive, formal, grammatically perfect or an absolute disaster with punctuation.

Some of us manage several of those before breakfast.

None of them proves who wrote the bloody thing.

The Question Nobody Should Be Allowed to Dodge

Here’s the question that bothers me most.

Is the system more interested in catching AI writing than in protecting innocent human writers from being falsely accused?

I’m not claiming to have discovered GPTZero’s secret business strategy. I haven’t. And its published documentation actually says it aims to avoid false accusations.

But look at the way this technology is presented to people.

A big confident verdict. Highlighted sentences. Percentages. Coloured indicators. Everything designed to make the result feel authoritative.

Where is the equally enormous warning saying:

This is a prediction. We might be wrong. Don’t use this bloody thing as proof.

That warning matters far more than another impressive-looking number.

Because consider the difference between two mistakes.

A machine fails to detect an AI-written homework assignment. Somebody gets away with using ChatGPT.

Not ideal, perhaps.

Now consider the other mistake.

A student spends three days writing an essay, submits it, and gets accused of cheating because some digital contraption doesn’t like the shape of their sentences.

One mistake lets somebody get away with something.

The other can damage an innocent person’s reputation.

Those are not remotely equivalent consequences.

The Researchers Found Something Rather Uncomfortable

And this isn’t just Greyhat having a moan after playing with a website.

In 2023, researchers associated with Stanford examined seven AI detection tools and found serious problems with how they classified writing by people whose first language wasn’t English.

Across the detectors tested, more than half of the human-written TOEFL essays were incorrectly labelled as AI-generated, on average.

Think about that.

People writing in a second language were more likely to be mistaken for machines.

Why?

Because certain forms of simpler, more predictable language could resemble the patterns those detectors were looking for.

Now there’s a splendid bit of technological progress.

You spend years learning another language, manage to put together a perfectly respectable essay, and a machine potentially decides you must have cheated because your vocabulary isn’t sufficiently adventurous.

Bloody marvellous.

The research didn’t establish that GPTZero’s current model has that same error rate. These were earlier systems, and detection technology has changed.

But it demonstrated something important: AI detection can penalise perfectly legitimate differences in the way human beings write.

And that ought to worry anybody with an ounce of common sense.

Even GPTZero Admits the Bloody Thing Can Be Wrong

Here’s another interesting detail.

GPTZero’s own guidance acknowledges that human writing can be classified as AI and AI writing can be classified as human.

It advises educators not to punish students on the basis of detection scores alone.

Well, quite right too.

The company also publishes accuracy claims and benchmarking results for newer models. It isn’t fair to ignore those simply because we’re giving the service a kicking.

But impressive performance in testing doesn’t turn a probability into proof about one particular writer.

Even a small false-positive rate matters when thousands or millions of documents are being assessed.

And if somebody’s education, employment or reputation is hanging on the result, that little statistical error suddenly becomes a very large personal problem.

That’s what gets lost in all the marketing excitement.

The machine can get another update next Thursday.

The person wrongly accused might spend months trying to clear their name.

Since When Did Being Human Require Certification?

This is the part I find particularly irritating.

We’ve spent decades encouraging people to write better.

Use clear sentences. Don’t waffle. Avoid unnecessary repetition. Structure your argument. Make your meaning understandable.

Then along comes artificial intelligence, which learns to produce clear, structured writing.

And suddenly clear, structured writing becomes suspicious.

What a bloody ridiculous circle we’ve created.

Write badly and somebody complains about your grammar.

Write well and somebody wonders whether ChatGPT did it.

Use a spelling checker? Suspicious.

Use a grammar checker? More suspicious.

Ask a machine to tidy up a paragraph you originally wrote yourself? Apparently we now need a philosophical inquiry into who owns the words.

Meanwhile, the actual quality of the argument risks becoming secondary to a guessing game about its origins.

Is it accurate?

Is it original in its ideas?

Does it make sense?

Can the author defend it?

Those strike me as rather more useful questions than whether an algorithm thinks the writer sounds a bit too tidy.

The Great Digital Witch Hunt

There’s an unpleasant historical familiarity about all this.

Somebody is accused. A mysterious authority claims to possess special knowledge. The accused is then expected to prove their innocence.

Except proving that you personally wrote a particular sentence can be surprisingly difficult.

What are you supposed to do?

Produce a photograph of yourself thinking?

Call your keyboard as a witness?

Ask the dog to confirm you were sitting at the computer all afternoon?

Yes, writing histories, drafts and notes can help establish authorship. But nobody should have to maintain a forensic record of every bloody adjective simply because a commercial detector might one day get suspicious.

The burden of proving wrongdoing shouldn’t be quietly transferred to the person who has been accused.

And no teacher, employer or publisher should treat a detection score as a substitute for proper evidence.

There Is a Place for AI Detection. Just Not on a Bloody Throne.

Before somebody starts shouting that Greyhat wants everyone cheating with AI, no.

There are legitimate reasons to investigate whether work has been generated by a machine.

Schools have assessment rules. Publishers have editorial standards. Employers sometimes need to know how work was produced.

Fair enough.

And AI detectors may be useful as one clue among several.

But a clue isn’t a conviction.

A probability isn’t a witness.

And a brightly coloured dashboard isn’t the bloody truth simply because it looks expensive.

If the technology is uncertain, say so clearly.

If it has limitations, put those limitations somewhere people can actually see them.

And if an accusation could harm somebody, demand more evidence before acting on it.

It really shouldn’t be complicated.

So What Did Our Little Experiment Prove?

Not that GPTZero is always wrong.

Not that AI detectors are useless.

And certainly not that our partly AI-written Churchill article was secretly written by a ghost sitting in a library in 1940.

It demonstrated something rather more interesting.

A modern AI detector can deliver a spectacularly confident classification of a document containing material from different origins, including a historical quotation, without independently establishing the provenance of every sentence.

That’s a limitation worth understanding before anybody starts waving the result around as proof.

And it leaves me with one final question.

If we’re going to use artificial intelligence to judge whether people are being honest, shouldn’t we demand that the judgement itself is held to an extraordinarily high standard?

Because if we don’t, we’re building a world where people can be accused by machines, judged by percentages and expected to apologise for writing in a way some algorithm doesn’t like.

And that, frankly, is bollocks.

Greyhat’s final thought:

I don’t particularly mind a machine telling me it suspects something.

I do mind a machine dressing suspicion up as certainty, especially when somebody else might be daft enough to believe it.

And if GPTZero decides this article was written by AI?

Well, some of it was written with AI assistance. We’re quite happy to tell you that.

Which is more than any detector can establish simply by staring at the finished words.

Perhaps the real test of honesty is whether the person publishing something is prepared to tell you how it was made.

Now there’s a bloody radical idea.

Research supporting the criticism

The strongest evidence is the independent Stanford research, alongside GPTZero’s own explanation of its limitations.

• Stanford-associated research: AI detectors and non-native English writers
• GPTZero’s published limitations
• GPTZero’s explanation of detection scores

The research found substantial false positives in the detectors tested in 2023, while GPTZero says its current systems have improved and cautions against using results alone to punish students.

PMC