image.png

引言，以及一些破碎的想法

该部分旨在推动针对agent的系统层AI Infra研究

最关键的概念之一是：

Semantic Control Plane

语义控制平面。

它表示：控制面不只读 CPU、内存、Pod 状态、HTTP 200、queue depth，而是读取 Agent 的 step、goal、tool call、memory delta、checkpoint freshness、loop score、semantic progress 等信号，再决定是否继续、暂停、恢复、迁移、加密观测、降频心跳或升级人工介入。

Semantic-Pulse 应纳入 AI Infra 的哪个位置：

AI Infra
├── Compute / Cluster Substrate
├── Model Serving / Inference Runtime
├── Distributed Execution Runtime
├── Agent Runtime / Workflow Runtime
├── Observability / Telemetry
├── Scheduling / Autoscaling
├── Reliability / Fault Tolerance
└── Agent Runtime Reliability & Semantic Control Plane  ← Semantic-Pulse

Semantic-Pulse 横跨四个 Infra 层：

Observability  →  采集 Agent 语义状态
Reliability    →  判断 alive / progressing / recoverable
Scheduling     →  调整下一次 heartbeat 与控制动作
Runtime        →  对接 LangGraph / Temporal / Ray / Kubernetes

因此它不是单独属于 Observability，也不是单独属于 Scheduling，而是一个 closed-loop control system：

Observe → Interpret → Decide → Act
观测 → 语义判断 → 调度决策 → 恢复 / 迁移 / 暂停 / 降频 / 升频

Agent Infra的整体面概览

对于整个Agent Infra，自下而上的架构可表示为：

层 主题 你需要掌握的问题
L0 Compute / Network / Storage / GPU Agent 到底运行在哪些物理 / 虚拟资源上？
L1 Probe / Lease / Health Check 传统 Infra 如何判断“活着 / 就绪 / 可接流量”？
L2 Distributed Runtime worker、actor、task、object 丢失时如何恢复？
L3 Model Serving 模型服务拥塞、KV cache、TTFT、queue time 如何影响 Agent？
L4 Tool Runtime 工具调用是否可重试？是否有副作用？是否超时？
L5 Durable Execution checkpoint、replay、resume、idempotency 如何保证？
L6 Agent Runtime Agent 当前处于 plan、act、observe、memory、human wait 中的哪一步？
L7 Semantic Control 是否语义推进？是否循环？是否假活？
L8 Scheduling 下一次 pulse 何时发？是否 pause / retry / migrate / drain？
L9 Evaluation 如何证明比固定 heartbeat、probe-only、checkpoint-only 更好？

具体来说，整体便可以表示为：

flowchart TB
    %% =========================
    %% Semantic-Pulse Architecture
    %% =========================

    subgraph L0["L0 Compute / Cluster Substrate"]
        K8S["Kubernetes<br/>Pod / Node / Lease / Probe"]
        RAY["Ray / KubeRay<br/>Task / Actor / Worker / Object"]
        STORAGE["State Store<br/>Postgres / Redis / S3 / Object Store"]
        GPU["GPU / Accelerator Layer<br/>DCGM / MIG / Device Plugin"]
    end

    subgraph L1["L1 Inference Serving Layer"]
        VLLM["vLLM<br/>KV Cache / Queue / TTFT / TPOT"]
        TRITON["Triton<br/>Inference Metrics / GPU Metrics"]
        KSERVE["KServe V2<br/>Server Live / Ready / Model Ready"]
        LLMCP["LLM Serving Control Plane<br/>llm-d / AIBrix / Router / Autoscaler"]
    end

    subgraph L2["L2 Tool Runtime Layer"]
        MCP["MCP Gateway<br/>Tool / Resource / Prompt / Workflow Access"]
        TOOLBOX["Tool Executors<br/>Browser / DB / Code / Files / APIs"]
        SANDBOX["Sandbox & Policy<br/>Timeout / Quota / Permission / Idempotency"]
    end

    subgraph L3["L3 Agent Runtime Layer"]
        AGENT["Persistent Agent<br/>Planner / Executor / Memory / Reflection"]
        GRAPH["Workflow Runtime<br/>LangGraph / Temporal / Step Graph"]
        CHECKPOINT["Checkpoint Manager<br/>Thread / Step / State / Replay"]
        MEMORY["Memory Store<br/>Short-term / Long-term / Vector / Episodic"]
    end

    subgraph L4["L4 Observability & Pulse Collection"]
        SDK["AgentPulse SDK<br/>Step Hooks / Tool Hooks / Model Hooks"]
        SIDECAR["Pulse Sidecar / Exporter<br/>Normalize / Hash / Redact"]
        OTEL["OpenTelemetry<br/>Trace / Span / Metric / Event"]
        COLLECTOR["Pulse Collector<br/>Ingest / Validate / Deduplicate / Order"]
    end

    subgraph L5["L5 Semantic Analysis Plane"]
        FEATURE["Feature Extractor<br/>Progress / Novelty / Loop / Risk"]
        SEMANTIC["Semantic Analyzer<br/>Semantic Delta / Stall Detection"]
        JUDGE["Policy Judge<br/>Safe Interrupt / Retryability / Human Escalation"]
        SCORE["Semantic Liveness Score<br/>Alive / Ready / Progressing / Recoverable"]
    end

    subgraph L6["L6 Adaptive Scheduling & Control Plane"]
        SCHED["Adaptive Heartbeat Scheduler<br/>Next Interval / Payload Depth / Jitter"]
        POLICY["Policy Engine<br/>SLO / Cost / Risk / Recovery Cost"]
        ACTION["Action Controller<br/>Pause / Retry / Resume / Drain / Migrate / Kill"]
        ALERT["Alert & Human-in-the-loop<br/>Escalation / Approval / Audit"]
    end

    %% Bottom dependencies
    K8S --> RAY
    K8S --> GPU
    K8S --> STORAGE

    GPU --> VLLM
    GPU --> TRITON
    KSERVE --> VLLM
    KSERVE --> TRITON
    VLLM --> LLMCP
    TRITON --> LLMCP

    MCP --> TOOLBOX
    TOOLBOX --> SANDBOX

    AGENT --> GRAPH
    GRAPH --> CHECKPOINT
    GRAPH --> MEMORY
    AGENT --> MCP
    AGENT --> VLLM

    SDK --> SIDECAR
    SIDECAR --> OTEL
    OTEL --> COLLECTOR
    AGENT --> SDK
    GRAPH --> SDK
    CHECKPOINT --> SDK
    VLLM --> OTEL
    TRITON --> OTEL
    KSERVE --> OTEL
    MCP --> OTEL

    COLLECTOR --> FEATURE
    FEATURE --> SEMANTIC
    SEMANTIC --> JUDGE
    JUDGE --> SCORE

    SCORE --> SCHED
    POLICY --> SCHED
    SCHED --> ACTION
    ACTION --> K8S
    ACTION --> RAY
    ACTION --> GRAPH
    ACTION --> LLMCP
    ACTION --> MCP
    ACTION --> ALERT

    STORAGE --> COLLECTOR
    CHECKPOINT --> STORAGE

Agent Infra的各部分解析

上一部分基本讨论了Agent Infra的整体结构，这一部分做一个更细致的探索，目标是要进一步解析清楚：