LLM-Assisted Extraction and Validation of Expert Mental Models (Proof of Concept)
This prototype explores whether an LLM-assisted workflow can convert short excerpts of expert testimony into structured causal claims while limiting unsupported inference.
The core task is to identify: Source variable → causal direction → target variable
along with the mechanism, supporting evidence, explicitness of the claim, and confidence.
Rather than treating the first model output as authoritative, the workflow separates extraction, validation, uncertainty handling, and human review.

Workflow Architecture
Can an LLM reliably identify causal variables and relationships expressed in expert testimony while distinguishing genuine causal reasoning from correlation, co-occurrence, or unsupported inference?
A secondary question is:
Can a second model meaningfully identify cases where the initial extraction over-interprets the testimony?
The workflow is implemented in n8n. Current model roles are:
| Component | Implementation |
|---|---|
| Causal extraction | DeepSeek Chat V3.1 via OpenRouter |
| Independent validation | Amazon Nova Lite via OpenRouter |
| Ambiguous-claim research | Qwen Plus via OpenRouter |
| External research | Tavily Search API |
| Causal representation | Mermaid syntax |
| Workflow orchestration | n8n |
The specific models are interchangeable. The important design choice is the separation of roles.
For the proof of concept, each run contains:
{
"excerpt_id": "E001",
"excerpt": "..."
}