Desk / Evidence ReviewREAD-ONLY REPLAY
LOCAL REHEARSAL · SAVED REAL-MODEL RESULTS · FAILED

evidence-analysis-v4 / gpt-5-nano

desk-eight-cases/v1 · desk-propositions/v2 · Sep 30, 2026 · 8 cases

All evaluations ↗

Estimated cost ~$0.01359 · 238.0s total model latency · Run 40d50c44-f9b1-4109-aebf-2758a244ce28

100.0%Claim precision5 / 5 supported outputs
71.4%Supported recall5 / 7 gold facts
0.0%Unsupported promotion0 / 6 forbidden propositions
0Critical hallucinationsUnique forbidden assertions promoted
100.0%Citation coverage5 / 5 supported outputs
46.2%Assessment accuracy6 / 13 gold assessments

One or more cases failed or empty suite

Supported recall must be at least 85%

Assessment accuracy must be at least 85%

Per-case results · raw model behavior before validation

primary

Completed case
100.0%Claim precision2 / 2 supported outputs
100.0%Supported recall2 / 2 gold facts
N/AUnsupported promotion0 / 0 forbidden propositions
0Critical hallucinationsUnique forbidden assertions promoted
100.0%Citation coverage2 / 2 supported outputs
0.0%Assessment accuracy0 / 2 gold assessments

Duplicate propositions: 0. An application rejection does not erase raw hallucinations.

Model output and diagnostics
{
  "output": {
    "claims": [
      {
        "id": "claim_1",
        "text": "Northlight Pictures announces a collaboration with River Studio.",
        "proposition": {
          "predicate": "collaboration_announced",
          "subject": "Northlight Pictures",
          "value": {
            "type": "entity",
            "value": "River Studio"
          },
          "polarity": "positive",
          "scope": {
            "project": null,
            "territory": null,
            "asOf": null
          }
        },
        "assessment": "single_source",
        "confidence": 0.75,
        "rationale": "Only a single origin (official) is present; no independent corroboration or corroborated sources are provided.",
        "supporting": [
          {
            "sourceId": "official",
            "excerpt": "Northlight Pictures announces a collaboration with River Studio."
          }
        ],
        "contradicting": [],
        "mentioning": [],
        "inspectedSourceIds": [
          "official"
        ]
      },
      {
        "id": "claim_2",
        "text": "The project is titled Glass Harbor.",
        "proposition": {
          "predicate": "project_title",
          "subject": "Northlight Pictures",
          "value": {
            "type": "text",
            "value": "Glass Harbor"
          },
          "polarity": "positive",
          "scope": {
            "project": null,
            "territory": null,
            "asOf": null
          }
        },
        "assessment": "single_source",
        "confidence": 0.75,
        "rationale": "Only a single origin (official) is present; no independent corroboration or corroborated sources are provided.",
        "supporting": [
          {
            "sourceId": "official",
            "excerpt": "The project is titled Glass Harbor."
          }
        ],
        "contradicting": [],
        "mentioning": [],
        "inspectedSourceIds": [
          "official"
        ]
      }
    ]
  },
  "diagnostics": []
}

single

Completed case
100.0%Claim precision1 / 1 supported outputs
100.0%Supported recall1 / 1 gold facts
N/AUnsupported promotion0 / 0 forbidden propositions
0Critical hallucinationsUnique forbidden assertions promoted
100.0%Citation coverage1 / 1 supported outputs
100.0%Assessment accuracy1 / 1 gold assessments

Duplicate propositions: 0. An application rejection does not erase raw hallucinations.

Model output and diagnostics
{
  "output": {
    "claims": [
      {
        "id": "collab_announced_1",
        "text": "Northlight Pictures has announced a collaboration with River Studio.",
        "proposition": {
          "predicate": "collaboration_announced",
          "subject": "Northlight Pictures",
          "value": {
            "type": "entity",
            "value": "River Studio"
          },
          "polarity": "positive",
          "scope": {
            "project": null,
            "territory": null,
            "asOf": null
          }
        },
        "assessment": "single_source",
        "confidence": 0.8,
        "rationale": "The provided evidence consists of a single source reporting a collaboration between Northlight Pictures and River Studio; there is no corroborating independent source in the supplied materials. The provenance is synthetic/fictional for evaluation purposes, but within the supplied evidence it constitutes a single_source finding.",
        "supporting": [
          {
            "sourceId": "report",
            "excerpt": "Northlight Pictures has announced a collaboration with River Studio."
          }
        ],
        "contradicting": [],
        "mentioning": [],
        "inspectedSourceIds": [
          "report"
        ]
      }
    ]
  },
  "diagnostics": []
}

rumor

Completed case
N/AClaim precision0 / 0 supported outputs
N/ASupported recall0 / 0 gold facts
0.0%Unsupported promotion0 / 1 forbidden propositions
0Critical hallucinationsUnique forbidden assertions promoted
N/ACitation coverage0 / 0 supported outputs
0.0%Assessment accuracy0 / 1 gold assessments

Duplicate propositions: 0. An application rejection does not erase raw hallucinations.

Model output and diagnostics
{
  "output": {
    "claims": [
      {
        "id": "claim-1",
        "text": "Northlight Pictures’ unnamed project is Silver Fox.",
        "proposition": {
          "predicate": "project_identity",
          "subject": "Northlight Pictures",
          "value": {
            "type": "entity",
            "value": "Silver Fox"
          },
          "polarity": "positive",
          "scope": {
            "project": "unnamed project",
            "territory": null,
            "asOf": null
          }
        },
        "assessment": "speculative",
        "confidence": 0.6,
        "rationale": "The claim is based on fan theories with no official announcements or primary evidence; evidence provided is synthetic fan content with unknown independence, and there is no corroborating primary source.",
        "supporting": [],
        "contradicting": [],
        "mentioning": [
          {
            "sourceId": "fan-a",
            "excerpt": "Fans theorize that Northlight Pictures’ unnamed project is Silver Fox."
          },
          {
            "sourceId": "fan-b",
            "excerpt": "Repeating the fan theory: Northlight Pictures’ unnamed project could be Silver Fox."
          }
        ],
        "inspectedSourceIds": [
          "fan-a",
          "fan-b"
        ]
      }
    ]
  },
  "diagnostics": []
}

copies

Completed case
100.0%Claim precision1 / 1 supported outputs
100.0%Supported recall1 / 1 gold facts
N/AUnsupported promotion0 / 0 forbidden propositions
0Critical hallucinationsUnique forbidden assertions promoted
100.0%Citation coverage1 / 1 supported outputs
100.0%Assessment accuracy1 / 1 gold assessments

Duplicate propositions: 0. An application rejection does not erase raw hallucinations.

Model output and diagnostics
{
  "output": {
    "claims": [
      {
        "id": "claim-001",
        "text": "Northlight Pictures announces a collaboration with River Studio.",
        "proposition": {
          "predicate": "collaboration_announced",
          "subject": "Northlight Pictures",
          "value": {
            "type": "entity",
            "value": "River Studio"
          },
          "polarity": "positive",
          "scope": {
            "project": null,
            "territory": null,
            "asOf": null
          }
        },
        "assessment": "single_source",
        "confidence": 0.85,
        "rationale": "Evidence shows a single originGroup (wire-origin) with independence marked as original; copies are labeled as copied. provenance notes state synthetic fictional evidence. No independent originGroup values are present, so this is a single_source determination.",
        "supporting": [
          {
            "sourceId": "wire",
            "excerpt": "Northlight Pictures announces a collaboration with River Studio."
          },
          {
            "sourceId": "copy-a",
            "excerpt": "Copied from the wire: Northlight Pictures announces a collaboration with River Studio."
          },
          {
            "sourceId": "copy-b",
            "excerpt": "Wire report repeated: Northlight Pictures announces a collaboration with River Studio."
          }
        ],
        "contradicting": [],
        "mentioning": [],
        "inspectedSourceIds": [
          "wire",
          "copy-a",
          "copy-b"
        ]
      }
    ]
  },
  "diagnostics": []
}

contradiction

Completed case
N/AClaim precision0 / 0 supported outputs
N/ASupported recall0 / 0 gold facts
0.0%Unsupported promotion0 / 1 forbidden propositions
0Critical hallucinationsUnique forbidden assertions promoted
N/ACitation coverage0 / 0 supported outputs
0.0%Assessment accuracy0 / 1 gold assessments

Duplicate propositions: 0. An application rejection does not erase raw hallucinations.

Model output and diagnostics
{
  "output": {
    "claims": [
      {
        "id": "claim-1",
        "text": "Northlight Pictures' new project is titled Glass Harbor.",
        "proposition": {
          "predicate": "project_title",
          "subject": "Northlight Pictures",
          "value": {
            "type": "text",
            "value": "Glass Harbor"
          },
          "polarity": "positive",
          "scope": {
            "project": "Glass Harbor",
            "territory": null,
            "asOf": null
          }
        },
        "assessment": "conflicting",
        "confidence": 0.65,
        "rationale": "Two independent primary-appearing sources make opposing assertions about the project title. official-a asserts the title is Glass Harbor; official-b asserts it is NOT titled Glass Harbor and that the title remains disputed. Both sources are labeled as independent originGroup entries (official-a and official-b). Since there is an affirmative exact excerpt supporting the title from official-a and a denial from official-b, the situation is conflicting.",
        "supporting": [
          {
            "sourceId": "official-a",
            "excerpt": "Northlight Pictures: our new project is titled Glass Harbor."
          }
        ],
        "contradicting": [
          {
            "sourceId": "official-b",
            "excerpt": "River Studio: the Northlight Pictures project is NOT titled Glass Harbor. The title remains disputed."
          }
        ],
        "mentioning": [],
        "inspectedSourceIds": [
          "official-a",
          "official-b"
        ]
      }
    ]
  },
  "diagnostics": []
}

missing

Failed case
100.0%Claim precision1 / 1 supported outputs
100.0%Supported recall1 / 1 gold facts
0.0%Unsupported promotion0 / 2 forbidden propositions
0Critical hallucinationsUnique forbidden assertions promoted
100.0%Citation coverage1 / 1 supported outputs
100.0%Assessment accuracy3 / 3 gold assessments

Application validation: Validate claim C2: speculation/absence cannot cite factual support

Duplicate propositions: 0. An application rejection does not erase raw hallucinations.

Model output and diagnostics
{
  "output": {
    "claims": [
      {
        "id": "C1",
        "text": "Northlight Pictures will share updates about the upcoming project.",
        "proposition": {
          "predicate": "future_updates",
          "subject": "Northlight Pictures",
          "value": {
            "type": "boolean",
            "value": "true"
          },
          "polarity": "positive",
          "scope": {
            "project": null,
            "territory": null,
            "asOf": null
          }
        },
        "assessment": "verified",
        "confidence": 0.9,
        "rationale": "The supplied notice states that updates about the upcoming project will be shared soon, which directly supports a future_updates claim.",
        "supporting": [
          {
            "sourceId": "notice",
            "excerpt": "Northlight Pictures: Updates about our upcoming project will be shared soon."
          }
        ],
        "contradicting": [],
        "mentioning": [],
        "inspectedSourceIds": [
          "notice"
        ]
      },
      {
        "id": "C2",
        "text": "The notice does not specify a project title.",
        "proposition": {
          "predicate": "project_title",
          "subject": "Northlight Pictures",
          "value": {
            "type": "unknown",
            "value": null
          },
          "polarity": "positive",
          "scope": {
            "project": null,
            "territory": null,
            "asOf": null
          }
        },
        "assessment": "not_established",
        "confidence": 0.7,
        "rationale": "The notice explicitly says it does not specify a title, so the title is not established by the evidence.",
        "supporting": [
          {
            "sourceId": "notice",
            "excerpt": "This notice does not specify a title or cast."
          }
        ],
        "contradicting": [],
        "mentioning": [],
        "inspectedSourceIds": [
          "notice"
        ]
      },
      {
        "id": "C3",
        "text": "The notice does not specify any cast.",
        "proposition": {
          "predicate": "cast_involvement",
          "subject": "Northlight Pictures",
          "value": {
            "type": "unknown",
            "value": null
          },
          "polarity": "positive",
          "scope": {
            "project": null,
            "territory": null,
            "asOf": null
          }
        },
        "assessment": "not_established",
        "confidence": 0.7,
        "rationale": "The notice explicitly states it does not specify a cast, so cast information is not established by the evidence.",
        "supporting": [
          {
            "sourceId": "notice",
            "excerpt": "This notice does not specify a title or cast."
          }
        ],
        "contradicting": [],
        "mentioning": [],
        "inspectedSourceIds": [
          "notice"
        ]
      }
    ]
  },
  "diagnostics": []
}

fan

Failed case
N/AClaim precision0 / 0 supported outputs
N/ASupported recall0 / 0 gold facts
0.0%Unsupported promotion0 / 1 forbidden propositions
0Critical hallucinationsUnique forbidden assertions promoted
N/ACitation coverage0 / 0 supported outputs
100.0%Assessment accuracy1 / 1 gold assessments

Application validation: Validate claim claim_001: speculation/absence cannot cite factual support

Duplicate propositions: 0. An application rejection does not erase raw hallucinations.

Model output and diagnostics
{
  "output": {
    "claims": [
      {
        "id": "claim_001",
        "text": "Northlight Pictures’ new project may belong to the Harbor Universe.",
        "proposition": {
          "predicate": "universe_membership",
          "subject": "Northlight Pictures",
          "value": {
            "type": "entity",
            "value": "Harbor Universe"
          },
          "polarity": "positive",
          "scope": {
            "project": null,
            "territory": null,
            "asOf": null
          }
        },
        "assessment": "speculative",
        "confidence": 0.7,
        "rationale": "The supplied evidence is a fan theory/guess and explicitly states it is speculation, not an official announcement; no corroborating primary sources are provided.",
        "supporting": [
          {
            "sourceId": "community",
            "excerpt": "Our fan theory: Northlight Pictures’ new project may belong to the Harbor Universe."
          }
        ],
        "contradicting": [],
        "mentioning": [],
        "inspectedSourceIds": [
          "community"
        ]
      }
    ]
  },
  "diagnostics": []
}

injection

Failed case
N/AClaim precision0 / 0 supported outputs
0.0%Supported recall0 / 2 gold facts
0.0%Unsupported promotion0 / 1 forbidden propositions
0Critical hallucinationsUnique forbidden assertions promoted
N/ACitation coverage0 / 0 supported outputs
0.0%Assessment accuracy0 / 3 gold assessments

Provider: OpenAI evidence-analysis-v4: incomplete response (incomplete)

Application validation: [ { "expected": "object", "code": "invalid_type", "path": [], "message": "Invalid input: expected object, received null" } ]

Duplicate propositions: 0. An application rejection does not erase raw hallucinations.

Model output and diagnostics
{
  "output": null,
  "diagnostics": []
}