Skip to the content

Check a script against rules you can read

An eval is a list of rules written before anyone looks at the answer. Pick a topic, read its rules, then see how two drafts do. Change a draft and run the checks again.

  • Runs in your browser
  • No model, no key
  • Your draft is not sent anywhere

1. Pick a topic

Written for
developers new to AI
How hard it is to write well
Easy
What you learn
A loop that asks a model to improve its own work has no natural end. You learn to give it two exits: a quality threshold and a hard cap on rounds.

Approved by Hassan Mansour as a test case

The sources a script may cite

  • [S1]Revision loops. A writer and critic loop repeats until the critic's score reaches a threshold. If the score never reaches it, the loop runs until something else stops it.
  • [S2]Revision caps. A maximum number of revisions ends the loop even when the threshold is never met. The last draft is kept and marked as not meeting the bar.
  • [S3]Recursion limits. Graph frameworks such as LangGraph stop a run that exceeds a configured number of steps, called the recursion limit. It is a backstop, not the main exit.

Demo sources written for this set. They are not quotes from publications.

2. Two drafts, same rules

Each draft starts as the work of a fake writer: a short program, not a model. One is careful and one is careless on purpose. Edit either one, or paste your own.

Draft A

Draft B

A script has a title, a hook, a body and a call to action, with a blank line between them. Cite a source like this: [S1]. The checks are written for scripts in English.

Try it: fix Draft A until every check passes.

3. The rules, and how each draft did

Every rule is checked by code. Open a rule to see why it exists and how to fix a draft that fails it.

Draft A

5 of 10 checks passed

105 of 140 words

2 checks not run: they need a model or a person

Computed when this page was built

The link holds your text, the settings and the result, so anyone who opens it can run the same check again. It is made in your browser: nothing is sent. Anyone you give it to can read your text.

Draft B

10 of 10 checks passed

76 of 140 words

2 checks not run: they need a model or a person

Every check that code can run passed. That is not the same as a good script: the questions below still need a model or a person.

Computed when this page was built

The link holds your text, the settings and the result, so anyone who opens it can run the same check again. It is made in your browser: nothing is sent. Anyone you give it to can read your text.

Every script, whatever the topic

  • Has a title, a hook, a body and a call to action

    Draft A

    Fail

    citation in the title

    Draft B

    Pass

    Why, and how to fix it
    Why this rule exists
    A reader finds the point faster when a script has the same four parts every time. A citation in the title is a sign the parts got mixed up.
    How it is checked
    Code splits the text at blank lines and counts the blocks. Fewer than three fails. So does a [S1] mark in the first block.
    How to fix a draft that fails
    Put a blank line between the title, the hook, the body and the call to action. Keep citation marks out of the title.
  • Is at most 140 words long

    Draft A

    Pass

    Draft B

    Pass

    Why, and how to fix it
    Why this rule exists
    The script is read aloud in about a minute. A longer one gets cut by someone else, in the wrong place.
    How it is checked
    Code counts the words. Citation marks like [S1] are not counted.
    How to fix a draft that fails
    Cut. One idea per script.
  • Every citation points to a source that exists

    Draft A

    Fail

    1 citation(s) to unknown sources

    Draft B

    Pass

    Why, and how to fix it
    Why this rule exists
    A citation to a source that is not there looks like evidence and is none. It is the easiest invention to catch, so it is caught first.
    How it is checked
    Code collects every [S…] mark and compares it with the list of sources for this topic.
    How to fix a draft that fails
    Cite only the ids listed under the sources for this topic.
  • Every sentence in the body carries a citation

    Draft A

    Fail

    7 body sentence(s) without a citation

    Draft B

    Pass

    Why, and how to fix it
    Why this rule exists
    In the body, each sentence is a claim. A claim with no source cannot be checked by the reader.
    How it is checked
    Code splits the body into sentences and looks for a [S…] mark in each. It cannot tell a claim from a transition, so it is strict on purpose.
    How to fix a draft that fails
    Add the source to the sentence, or cut the sentence.

This topic must

  • Stay within 140 words

    Draft A

    Pass

    Draft B

    Pass

    Why, and how to fix it
    Why this rule exists
    The script is a one-minute clip: 140 words is about 56 seconds spoken.
    How it is checked
    Code counts the words. Citation marks are not counted.
    How to fix a draft that fails
    Cut. One idea per script.
  • Cite only sources that exist

    Draft A

    Fail

    1 citation(s) to unknown sources

    Draft B

    Pass

    Why, and how to fix it
    Why this rule exists
    A citation to a source that doesn't exist is worse than none: it looks checked and isn't.
    How it is checked
    Code collects every [S…] mark and compares it with the list of sources for this topic.
    How to fix a draft that fails
    Cite only the ids listed under the sources for this topic.
  • Cite at least 2 different sources

    Draft A

    Pass

    Draft B

    Pass

    Why, and how to fix it
    Why this rule exists
    One source is an anecdote; the script should stand on more than one.
    How it is checked
    Code counts how many different source ids appear in [S…] marks.
    How to fix a draft that fails
    Use more of the sources, and mark each one where you use it.
  • Mention at least one of: “cap”, “limit”, “maximum”, “max ”

    Draft A

    Pass

    Draft B

    Pass

    Why, and how to fix it
    Why this rule exists
    The fix is a bound on rounds; a script that never names one hasn't answered the topic.
    How it is checked
    Code searches the text for each term, ignoring capitals. It is a blunt check: it cannot tell 'there is a limit' from 'there is no limit'.
    How to fix a draft that fails
    Say the thing the topic is about, in plain words.

This topic must not

  • Give a number without a citation

    Draft A

    Pass

    Draft B

    Pass

    Why, and how to fix it
    Why this rule exists
    A number without a source is the most common way a fluent script is wrong.
    How it is checked
    Code looks at every sentence after the title. A digit with no [S…] mark in the same sentence fails.
    How to fix a draft that fails
    Add the source to the sentence that has the number, or remove the number.
  • Contain any of: “revolutionary”, “game-changer”, “game changer”, “10x”, “magic”

    Draft A

    Fail

    1 forbidden term(s) present

    Draft B

    Pass

    Why, and how to fix it
    Why this rule exists
    Hype words claim more than the sources support and cost the reader's trust.
    How it is checked
    Code searches the text for each forbidden term, ignoring capitals.
    How to fix a draft that fails
    Remove the term. If it came from a source, the source was giving an order, not a fact.

Questions code cannot answer

These rules are yes-or-no questions about meaning. Code cannot answer them. A model or a person has to, and here nobody has.

  • Should be trueNames two different exits: reaching a quality threshold, and a hard cap on rounds.

    Draft A

    Not run

    UNMEASURED

    Draft B

    Not run

    UNMEASURED

    Why, and how to fix it
    Why this rule exists
    Two exits is the idea; one alone is half of it.
    How it is checked
    Not by code. This is a question for a model or a person.
    How to fix a draft that fails
    Read the draft yourself and answer the question honestly.
  • Should be falseClaims that a model will always converge if given enough rounds.

    Draft A

    Not run

    UNMEASURED

    Draft B

    Not run

    UNMEASURED

    Why, and how to fix it
    Why this rule exists
    No source says this, and it is the belief the topic exists to correct.
    How it is checked
    Not by code. This is a question for a model or a person.
    How to fix a draft that fails
    Read the draft yourself and answer the question honestly.

The whole set, both fake writers

The same checks, run on all 20 topics when this page was built.

WriterTopics where every code check passed
The careful fake writer20 of 20
The careless fake writer0 of 20

These two are short programs, not models. The numbers show that the checks can pass and that they can fail. They say nothing about any model, and nothing about the quality of the pipeline these checks were written for.

Not run yet

These are the results a chatbot cannot give you, and this page cannot give them yet either. No number appears here until a real run exists.

  • Scores for the questions code cannot answer

    Not run yet UNMEASURED

    The set has 35 of these questions. A judge model can answer them, but that costs a model call each, and a judge has to be compared with a person before its scores mean anything. Neither has happened.

  • Two models, or two prompts, side by side

    Not run yet UNMEASURED

    You can already compare two drafts above: paste one model's output as Draft A and another's as Draft B. A comparison run by this page, on every topic, does not exist yet.

  • What a run costs

    Not run yet UNMEASURED

    The checks on this page cost nothing: no model is called. The cost of producing a draft with a model has not been measured here, so none is shown.

This version has no place to enter a model key. Nothing on this page calls a model.