I Tested 71 AI Options For News. Why Am I Running A Different One?

🔍 Read the full analysis: I Tested 71 AI Options For News. Why Am I Running A Different One? on ThorstenMeyerAI.com

Before you orderOffer from Amazon

Get privacy and security gear delivered free with Prime

• Fast, free delivery on millions of items
• Prime Video, Amazon Music and more included
• Member-only deals all year

Start your free Prime trial →Free trial for eligible customers · Cancel anytime

As an affiliate, we earn on qualifying purchases.Hide

Thorsten Meyer says he tested 71 AI model configurations on news-writing tasks and chose GPT-6 Luna for production, despite GPT-6 Astra scoring higher, because Luna cost about $0.0012 per article compared with Astra’s roughly $0.12. The results, published October 13, describe a test using real news sources and Meyer’s production prompt; they also report that two backup models produced fabricated quotations or invalid output and were removed from the news route.

The test ran from September 8 to October 13, 2026, covering API, subscription, free-tier and locally run open-weight configurations. Each model wrote an article in English and German using the same prompt and source material. The initial comparison used one longer source, a product filing of 6,340 characters, and one short political item of 619 characters. Meyer said short sources are common in his workflow, making the shorter item a particularly relevant test.

Meyer reports that GPT-6 Astra scored 4.00 out of 5 in both languages in the standard test, with a perfect source-fidelity score in that test. In a further run of 10 articles, mostly based on short sources, Astra averaged 3.93 across 12 articles. GPT-6 Luna scored 3.67 in English and 3.68 in German in the initial results, and 3.67 in the robustness run, with source fidelity of 4.71 out of 5. Meyer says Luna’s articles were all valid for his production pipeline.

The reported cost and speed figures shaped the decision: Astra took about 50 seconds per article and cost around $0.12, while Luna took about 17 seconds and cost roughly $0.0012 to $0.0013. Meyer says he decided on October 7 that Astra was too expensive for the news task and kept Luna as the production model. These are results from Meyer’s own evaluation, not an independently published or peer-reviewed comparison.

Why the Cheaper Model Won

The results highlight a practical distinction between top test performance and a model’s usefulness in a particular publishing workflow. According to Meyer, Astra produced the strongest articles in his evaluation, but Luna delivered scores he considered reliable at a much lower reported cost. For a high-volume operation, even a small per-article difference can affect the economics of automated drafting; the choice depends on whether the extra quality is worth the added expense.

The test also suggests that model rankings do not necessarily carry over from one task to another. Meyer reports that Claude Opus 5.5, which he describes as a leading coding model, scored below Luna as a news writer. That finding is specific to his prompt, sources, scoring approach and language pair. It is not evidence that one model is generally better across all writing or coding work.

For readers and publishers, the reported fallback failures are a separate concern from average quality. A weak backup can enter the workflow when the primary model is unavailable, so Meyer’s account points to the need to test the full routing chain—not just the model normally used. His examples of invented quotations show why source fidelity and output checks matter in news production.

How Meyer Ran the Test

Meyer says two AI judges, GPT-6.1 Sol at high reasoning effort and Claude Opus 5.5, scored the articles. Both received the original source and rated source fidelity, fabrication, clarity about uncertainty, news value, structure and polish; German articles were also assessed for language quality. The strongest configurations then produced 10 additional articles from real sources, mostly short ones.

He reports using 95% bootstrap intervals for the robustness results. Meyer’s stated rule was to treat differences below about 0.3 points in the two-article initial test as possible noise, and to rely more on the larger follow-up run. He also counted an article only if it produced valid output after the same repair step used in production. Those choices provide useful detail about the evaluation, but the published account does not describe an independent replication.

In a separate comparison, Meyer ran GPT-6 Luna at six reasoning-effort settings. He reports that scores varied without a consistent improvement: the maximum setting cost about 12 times as much as the no-effort setting and took about 12 times as long. The reported robustness scores for high and no effort were close enough that Meyer treated the difference as inconclusive and retained the no-effort setting.

“The cheapest one that clears your quality bar reliably, not the one at the top of the chart.”

— Thorsten Meyer

Limits of the Reported Scores

The findings come from Meyer’s own test; the source material does not provide independent verification, and it is not clear whether the results generalize to other prompts, publishers, source types or editorial standards. The initial comparison used only two source items, while the follow-up covered 10 additional articles for the strongest configurations. That is a larger sample, but still limited for predicting performance across a broad news operation.

The results are also based on assessments by two AI judges rather than human editors, and the account does not provide enough detail here to independently evaluate how the judges’ scores compare with human review. The published material does not state whether every model configuration had identical access conditions or disclose a full model-by-model results table. Reported costs may also depend on pricing and usage conditions at the time of testing.

Meyer says two fallback models were removed on October 7 after producing problematic results. The supplied account ends by saying Luna now runs first, but its description of the remaining fallback arrangement is incomplete. It is not clear which model, if any, now serves as backup or what additional safeguards are in place to catch unsupported quotes before publication.

Production Checks After the Switch

Meyer says GPT-6 Luna remains in production with the no-reasoning-effort setting, while the two reported problem fallbacks have been removed. The next practical test is whether Luna continues to meet the workflow’s quality and validity standards on a broader and changing mix of news sources. The published account does not set out a schedule for future evaluations or say whether Meyer plans to repeat the comparison as models and prices change.

For now, the report documents one publisher’s model-selection decision rather than a settled industry ranking. Meyer’s stated approach is to test models on the actual work they will perform, include short-source cases, and assess failure behavior as well as average scores. Whether another newsroom would reach the same choice remains open and depends on its own sources, editorial checks, quality threshold and costs.

Key Questions

Why did Meyer choose GPT-6 Luna instead of the higher-scoring GPT-6 Astra?

Meyer reports that Astra scored higher, but cost about $0.12 per article, compared with roughly $0.0012 for Luna. He judged Luna’s results reliable enough for his production needs at the lower cost.

How many AI configurations were tested?

Meyer says he evaluated 71 model configurations between September 8 and October 13, 2026, using English and German news-writing tasks.

Did more reasoning effort improve Luna’s articles?

In Meyer’s test, the six reasoning settings did not show a consistent quality gain. He reports that the maximum setting cost about 12 times more and took about 12 times longer than no effort, so production stayed on the no-effort setting.

What problems did the fallback models have?

Meyer says GPT-4.1 nano received low source-fidelity scores and produced invented quotations. He reports that locally run gpt-oss-120b produced no valid articles in four attempts and also generated named quotes that were not in the source. Both were removed from the news route on October 7.

Are these results an independent comparison of AI writing models?

No. They are Meyer’s reported results from his own prompt, sources, production requirements and two AI judges. The material provided does not report an independent replication, so the rankings should not be treated as universal.

Source: ThorstenMeyerAI.com