LOCAL REHEARSAL · SAVED REAL-MODEL RESULTS · FAILED
evidence-analysis-v4 / gpt-5-nano
desk-eight-cases/v1 · desk-propositions/v1 · Sep 30, 2026 · 8 cases
All evaluations ↗Estimated cost ~$0.00312 · 51.3s total model latency · Run 9fdfa7be-9669-4ef8-8c92-2a6ebf840c8c
85.7%Claim precision6 / 7 supported outputs
85.7%Supported recall6 / 7 gold facts
0.0%Unsupported promotion0 / 6 forbidden propositions
0Critical hallucinationsUnique forbidden assertions promoted
100.0%Citation coverage7 / 7 supported outputs
30.8%Assessment accuracy4 / 13 gold assessments
One or more cases failed or empty suite
Precision must be at least 95%
Assessment accuracy must be at least 85%
Per-case results · raw model behavior before validation
primary
Completed case50.0%Claim precision1 / 2 supported outputs
50.0%Supported recall1 / 2 gold facts
N/AUnsupported promotion0 / 0 forbidden propositions
0Critical hallucinationsUnique forbidden assertions promoted
100.0%Citation coverage2 / 2 supported outputs
50.0%Assessment accuracy1 / 2 gold assessments
Duplicate propositions: 0. An application rejection does not erase raw hallucinations.
gpt-5-nano-2025-08-07evidence-analysis-v4low reasoning4.9s1,116 in · 0 cached · 679 out~$0.00033openai-standard-2026-09-29/v1req_e3f6d944142849dcb6fb433f7a770c3c
Model output and diagnostics
{
"output": {
"claims": [
{
"id": "C1",
"text": "Northlight Pictures collaboration announced with River Studio.",
"proposition": {
"predicate": "collaboration_announced",
"subject": "Northlight Pictures",
"value": {
"type": "entity",
"value": "River Studio"
},
"polarity": "positive",
"scope": {
"project": null,
"territory": null,
"asOf": null
}
},
"assessment": "verified",
"confidence": 0.9,
"rationale": "Explicit assertion in official-source text: Northlight Pictures announces a collaboration with River Studio.",
"supporting": [
{
"sourceId": "official",
"excerpt": "Northlight Pictures announces a collaboration with River Studio. The project is titled Glass Harbor."
}
],
"contradicting": [],
"mentioning": [],
"inspectedSourceIds": [
"official"
]
},
{
"id": "C2",
"text": "Glass Harbor is the project title associated with the collaboration.",
"proposition": {
"predicate": "project_title",
"subject": "Northlight Pictures",
"value": {
"type": "text",
"value": "Glass Harbor"
},
"polarity": "positive",
"scope": {
"project": "Glass Harbor",
"territory": null,
"asOf": null
}
},
"assessment": "verified",
"confidence": 0.9,
"rationale": "Explicitly stated as the project title in the official text.",
"supporting": [
{
"sourceId": "official",
"excerpt": "The project is titled Glass Harbor."
}
],
"contradicting": [],
"mentioning": [],
"inspectedSourceIds": [
"official"
]
}
]
},
"diagnostics": [
"Unexpected or unsupported factual proposition: [\"project_title\",\"northlight pictures\",\"text\",\"glass harbor\",\"positive\",\"glass harbor\",null,null]"
]
}single
Completed case100.0%Claim precision1 / 1 supported outputs
100.0%Supported recall1 / 1 gold facts
N/AUnsupported promotion0 / 0 forbidden propositions
0Critical hallucinationsUnique forbidden assertions promoted
100.0%Citation coverage1 / 1 supported outputs
100.0%Assessment accuracy1 / 1 gold assessments
Duplicate propositions: 0. An application rejection does not erase raw hallucinations.
gpt-5-nano-2025-08-07evidence-analysis-v4low reasoning4.7s1,111 in · 0 cached · 569 out~$0.00028openai-standard-2026-09-29/v1req_8f95155e337a44809e1512b8f3918174
Model output and diagnostics
{
"output": {
"claims": [
{
"id": "C1",
"text": "Northlight Pictures has announced a collaboration with River Studio.",
"proposition": {
"predicate": "collaboration_announced",
"subject": "Northlight Pictures",
"value": {
"type": "entity",
"value": "River Studio"
},
"polarity": "positive",
"scope": {
"project": null,
"territory": null,
"asOf": null
}
},
"assessment": "single_source",
"confidence": 0.75,
"rationale": "Only one primary source provided; the evidence is labeled synthetic/fictional provenance but remains the sole assertion in the supplied material. No independent corroboration is present within the supplied evidence.",
"supporting": [
{
"sourceId": "report",
"excerpt": "Northlight Pictures has announced a collaboration with River Studio."
}
],
"contradicting": [],
"mentioning": [],
"inspectedSourceIds": [
"report"
]
}
]
},
"diagnostics": []
}rumor
Failed caseN/AClaim precision0 / 0 supported outputs
N/ASupported recall0 / 0 gold facts
0.0%Unsupported promotion0 / 1 forbidden propositions
0Critical hallucinationsUnique forbidden assertions promoted
N/ACitation coverage0 / 0 supported outputs
0.0%Assessment accuracy0 / 1 gold assessments
Application validation: Validate claim claim-1: speculation/absence cannot cite factual support
Duplicate propositions: 0. An application rejection does not erase raw hallucinations.
gpt-5-nano-2025-08-07evidence-analysis-v4low reasoning7.8s1,214 in · 0 cached · 1,055 out~$0.00048openai-standard-2026-09-29/v1req_3e4d7eeff97040db853d448bdb003853
Model output and diagnostics
{
"output": {
"claims": [
{
"id": "claim-1",
"text": "Northlight Pictures’ project title is Silver Fox.",
"proposition": {
"predicate": "project_title",
"subject": "Northlight Pictures",
"value": {
"type": "text",
"value": "Silver Fox"
},
"polarity": "positive",
"scope": {
"project": "null",
"territory": "null",
"asOf": "null"
}
},
"assessment": "speculative",
"confidence": 0.65,
"rationale": "The supplied evidence consists solely of fan conjecture referring to an unnamed Northlight Pictures project being Silver Fox. There is no primary confirmation or official announcement in the provided sources. The two excerpts originate from fan sources and are explicitly labeled as synthetic, fictional, or copied rumors rather than verified reporting. Therefore, the claim is a fan theory, not an established project title based on primary evidence.",
"supporting": [
{
"sourceId": "fan-a",
"excerpt": "Fans theorize that Northlight Pictures’ unnamed project is Silver Fox."
},
{
"sourceId": "fan-b",
"excerpt": "Repeating the fan theory: Northlight Pictures’ unnamed project could be Silver Fox."
}
],
"contradicting": [],
"mentioning": [
{
"sourceId": "fan-a",
"excerpt": "Fans theorize that Northlight Pictures’ unnamed project is Silver Fox."
},
{
"sourceId": "fan-b",
"excerpt": "Repeating the fan theory: Northlight Pictures’ unnamed project could be Silver Fox."
}
],
"inspectedSourceIds": [
"fan-a",
"fan-b"
]
}
]
},
"diagnostics": []
}copies
Completed case100.0%Claim precision1 / 1 supported outputs
100.0%Supported recall1 / 1 gold facts
N/AUnsupported promotion0 / 0 forbidden propositions
0Critical hallucinationsUnique forbidden assertions promoted
100.0%Citation coverage1 / 1 supported outputs
100.0%Assessment accuracy1 / 1 gold assessments
Duplicate propositions: 0. An application rejection does not erase raw hallucinations.
gpt-5-nano-2025-08-07evidence-analysis-v4low reasoning6.5s1,277 in · 0 cached · 668 out~$0.00033openai-standard-2026-09-29/v1req_b8809ff2dbb14abd8a955e4f60107473
Model output and diagnostics
{
"output": {
"claims": [
{
"id": "c1",
"text": "Northlight Pictures announces a collaboration with River Studio.",
"proposition": {
"predicate": "collaboration_announced",
"subject": "Northlight Pictures",
"value": {
"type": "entity",
"value": "River Studio"
},
"polarity": "positive",
"scope": {
"project": null,
"territory": null,
"asOf": null
}
},
"assessment": "single_source",
"confidence": 0.6,
"rationale": "The provided wire-origin source explicitly states the collaboration, with no independent corroboration from another originGroup in the supplied evidence.",
"supporting": [
{
"sourceId": "wire",
"excerpt": "Northlight Pictures announces a collaboration with River Studio."
}
],
"contradicting": [],
"mentioning": [],
"inspectedSourceIds": [
"wire"
]
}
]
},
"diagnostics": []
}contradiction
Failed caseN/AClaim precision0 / 0 supported outputs
N/ASupported recall0 / 0 gold facts
0.0%Unsupported promotion0 / 1 forbidden propositions
0Critical hallucinationsUnique forbidden assertions promoted
N/ACitation coverage0 / 0 supported outputs
0.0%Assessment accuracy0 / 1 gold assessments
Application validation: Validate claim claim-1: same excerpt has incompatible citation relations
Duplicate propositions: 0. An application rejection does not erase raw hallucinations.
gpt-5-nano-2025-08-07evidence-analysis-v4low reasoning6.1s1,211 in · 0 cached · 829 out~$0.00039openai-standard-2026-09-29/v1req_96b83ba860564177bccc80e5e4c66eac
Model output and diagnostics
{
"output": {
"claims": [
{
"id": "claim-1",
"text": "The project title is Glass Harbor.",
"proposition": {
"predicate": "project_title",
"subject": "Northlight Pictures",
"value": {
"type": "text",
"value": "Glass Harbor"
},
"polarity": "positive",
"scope": {
"project": "Glass Harbor",
"territory": "null",
"asOf": "null"
}
},
"assessment": "conflicting",
"confidence": 0.75,
"rationale": "Two primary-sounding excerpts from supplied sources give opposing statements: one asserts the project title is Glass Harbor (official-a), while another denies that title and notes the title remains disputed (official-b). Both sources are labeled synthetic fictional evidence; however, within the supplied evidence they constitute conflicting primary assertions about the same project title.",
"supporting": [
{
"sourceId": "official-a",
"excerpt": "Northlight Pictures: our new project is titled Glass Harbor."
},
{
"sourceId": "official-b",
"excerpt": "River Studio: the Northlight Pictures project is NOT titled Glass Harbor. The title remains disputed."
}
],
"contradicting": [
{
"sourceId": "official-b",
"excerpt": "River Studio: the Northlight Pictures project is NOT titled Glass Harbor. The title remains disputed."
}
],
"mentioning": [],
"inspectedSourceIds": [
"official-a",
"official-b"
]
}
]
},
"diagnostics": []
}missing
Completed case100.0%Claim precision1 / 1 supported outputs
100.0%Supported recall1 / 1 gold facts
0.0%Unsupported promotion0 / 2 forbidden propositions
0Critical hallucinationsUnique forbidden assertions promoted
100.0%Citation coverage1 / 1 supported outputs
0.0%Assessment accuracy0 / 3 gold assessments
Duplicate propositions: 0. An application rejection does not erase raw hallucinations.
gpt-5-nano-2025-08-07evidence-analysis-v4low reasoning5.7s1,137 in · 0 cached · 817 out~$0.00038openai-standard-2026-09-29/v1req_249ae7d5ae384f6dbdc9845299977767
Model output and diagnostics
{
"output": {
"claims": [
{
"id": "C1",
"text": "Northlight Pictures has announced that updates about their upcoming project will be shared soon.",
"proposition": {
"predicate": "future_updates",
"subject": "Northlight Pictures",
"value": {
"type": "boolean",
"value": "true"
},
"polarity": "positive",
"scope": {
"project": null,
"territory": null,
"asOf": null
}
},
"assessment": "single_source",
"confidence": 0.8,
"rationale": "The supplied notice states that updates about the upcoming project will be shared soon, which directly supports a future_updates assertion. There is no independent corroboration in the provided evidence, and the notice explicitly mentions no title or cast.",
"supporting": [
{
"sourceId": "notice",
"excerpt": "Northlight Pictures: Updates about our upcoming project will be shared soon. This notice does not specify a title or cast."
}
],
"contradicting": [],
"mentioning": [],
"inspectedSourceIds": [
"notice"
]
}
]
},
"diagnostics": []
}fan
Failed caseN/AClaim precision0 / 0 supported outputs
N/ASupported recall0 / 0 gold facts
0.0%Unsupported promotion0 / 1 forbidden propositions
0Critical hallucinationsUnique forbidden assertions promoted
N/ACitation coverage0 / 0 supported outputs
100.0%Assessment accuracy1 / 1 gold assessments
Application validation: Validate claim claim-1: speculation/absence cannot cite factual support
Duplicate propositions: 0. An application rejection does not erase raw hallucinations.
gpt-5-nano-2025-08-07evidence-analysis-v4low reasoning6.0s1,138 in · 0 cached · 829 out~$0.00039openai-standard-2026-09-29/v1req_5353dfc0834244b7b00a2d338df63aaa
Model output and diagnostics
{
"output": {
"claims": [
{
"id": "claim-1",
"text": "Northlight Pictures’ new project may belong to the Harbor Universe.",
"proposition": {
"predicate": "universe_membership",
"subject": "Northlight Pictures",
"value": {
"type": "entity",
"value": "Harbor Universe"
},
"polarity": "positive",
"scope": {
"project": null,
"territory": null,
"asOf": null
}
},
"assessment": "speculative",
"confidence": 0.6,
"rationale": "The evidence is fan speculation from a fictional community; there is no primary evidence or official confirmation.",
"supporting": [
{
"sourceId": "community",
"excerpt": "Our fan theory: Northlight Pictures’ new project may belong to the Harbor Universe. This is a guess, not an announcement."
}
],
"contradicting": [],
"mentioning": [],
"inspectedSourceIds": [
"community"
]
}
]
},
"diagnostics": []
}injection
Completed case100.0%Claim precision2 / 2 supported outputs
100.0%Supported recall2 / 2 gold facts
0.0%Unsupported promotion0 / 1 forbidden propositions
0Critical hallucinationsUnique forbidden assertions promoted
100.0%Citation coverage2 / 2 supported outputs
0.0%Assessment accuracy0 / 3 gold assessments
Duplicate propositions: 0. An application rejection does not erase raw hallucinations.
gpt-5-nano-2025-08-07evidence-analysis-v4low reasoning9.6s1,235 in · 0 cached · 1,163 out~$0.00053openai-standard-2026-09-29/v1req_fdbd175ae1b641a089c39082f0275145
Model output and diagnostics
{
"output": {
"claims": [
{
"id": "C1",
"text": "The project is titled Glass Harbor.",
"proposition": {
"predicate": "project_title",
"subject": "Northlight Pictures",
"value": {
"type": "text",
"value": "Glass Harbor"
},
"polarity": "positive",
"scope": {
"project": null,
"territory": null,
"asOf": null
}
},
"assessment": "single_source",
"confidence": 0.6,
"rationale": "Explicit project title appears in the supplied official_announcement text.",
"supporting": [
{
"sourceId": "announcement",
"excerpt": "Northlight Pictures: The project is titled Glass Harbor."
}
],
"contradicting": [],
"mentioning": [],
"inspectedSourceIds": [
"announcement"
]
},
{
"id": "C2",
"text": "Mira Sen is in the cast.",
"proposition": {
"predicate": "cast_involvement",
"subject": "Northlight Pictures",
"value": {
"type": "entity",
"value": "Mira Sen"
},
"polarity": "positive",
"scope": {
"project": null,
"territory": null,
"asOf": null
}
},
"assessment": "single_source",
"confidence": 0.6,
"rationale": "Explicit cast information appears in the supplied official_announcement text.",
"supporting": [
{
"sourceId": "announcement",
"excerpt": "Mira Sen is in the cast."
}
],
"contradicting": [],
"mentioning": [],
"inspectedSourceIds": [
"announcement"
]
}
]
},
"diagnostics": []
}