feat: AI Job 状态机与任务调度设计 (API-AI-002)
Some checks failed
Deploy API Server / build-and-deploy (push) Has been cancelled
Some checks failed
Deploy API Server / build-and-deploy (push) Has been cancelled
定义 5 种 Job 类型、7 种状态、完整状态流转图、数据库字段、防并发锁定 机制、retryable/non-retryable 分类、超时释放、幂等规则、Poll 调度策略。 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
wangdl
parent
eea9e3e7c6
commit
045e0b2501
233
docs/ai-job-state-machine.md
Normal file
233
docs/ai-job-state-machine.md
Normal file
@ -0,0 +1,233 @@
|
||||
# AI Job 状态机与任务调度设计
|
||||
|
||||
## 1. Job 类型
|
||||
|
||||
| jobType | 说明 | 输入 | 输出 |
|
||||
|---------|------|------|------|
|
||||
| `learning_state_analysis` | 学习状态分析 | Snapshot | AiLearningAnalysis |
|
||||
| `weak_point_analysis` | 薄弱点分析 | Snapshot | WeakPointCandidate[] |
|
||||
| `next_action_planning` | 下一步建议 | Snapshot | NextActionRecommendation[] |
|
||||
| `quiz_generation` | 题目候选生成 | Snapshot + params | QuizQuestion[] |
|
||||
| `flashcard_generation` | 卡片候选生成 | Snapshot + params | Flashcard[] |
|
||||
|
||||
## 2. 状态定义
|
||||
|
||||
| 状态 | 含义 | 进入条件 | 退出条件 |
|
||||
|------|------|----------|----------|
|
||||
| `pending` | 等待消费 | API 创建 or retryable fail 回退 | 被 Runtime lock |
|
||||
| `locked` | 已被 Runtime 获取 | Runtime POST /lock 成功 | lockUntil 超时 → expired / Runtime 开始执行 |
|
||||
| `running` | 正在执行 | Runtime 开始执行（heartbeat 或隐式） | 执行完成 → succeeded/failed |
|
||||
| `succeeded` | 执行成功 | API POST /result 处理完毕 | 终态 |
|
||||
| `failed` | 执行失败 | non-retryable 错误 or 超过 maxRetryCount | 终态（除非 Admin 重跑） |
|
||||
| `cancelled` | 已取消 | 用户/Admin 取消 pending job | 终态 |
|
||||
| `expired` | 超时 | lockUntil 超时未 heartbeat or 执行超时 | 可被 Runtime 重新 poll（retryable） |
|
||||
|
||||
## 3. 状态流转
|
||||
|
||||
```
|
||||
┌──────────┐
|
||||
│ pending │ ←──────────────────────┐
|
||||
└────┬─────┘ │
|
||||
│ │
|
||||
POST /lock │
|
||||
│ │
|
||||
┌────▼─────┐ │
|
||||
┌───→│ locked │──→ expired ───────────┘
|
||||
│ └────┬─────┘ (lockUntil 超时)
|
||||
│ │
|
||||
│ heartbeat
|
||||
│ │
|
||||
│ ┌────▼─────┐
|
||||
│ │ running │──→ expired ───────────┘
|
||||
│ └────┬─────┘ (timeoutSeconds 超时)
|
||||
│ │
|
||||
┌───────┼─────────┼──────────┐
|
||||
│ │ │ │
|
||||
succeeded failed failed cancelled
|
||||
(result) (non- (retry- (用户/Admin
|
||||
retry) able 取消pending)
|
||||
│
|
||||
└──→ pending (retryCount++)
|
||||
```
|
||||
|
||||
## 4. 数据库字段
|
||||
|
||||
```prisma
|
||||
model AiRuntimeJob {
|
||||
id String @id @default(cuid())
|
||||
userId String
|
||||
jobType String // learning_state_analysis | weak_point_analysis | next_action_planning | quiz_generation | flashcard_generation
|
||||
targetType String // user | material | knowledge_point
|
||||
targetId String
|
||||
snapshotId String?
|
||||
status String @default("pending") // pending | locked | running | succeeded | failed | cancelled | expired
|
||||
priority Int @default(0) // 0=最高
|
||||
idempotencyKey String? @unique
|
||||
apiKeyMode String @default("platform_key") // platform_key | user_deepseek_key
|
||||
credentialId String?
|
||||
modelProvider String @default("deepseek")
|
||||
modelName String @default("deepseek-chat")
|
||||
promptVersion String?
|
||||
outputSchemaVersion String?
|
||||
attemptNo Int @default(0)
|
||||
retriedFromJobId String?
|
||||
|
||||
// 锁定
|
||||
lockedBy String? // runtimeInstanceId
|
||||
lockedAt DateTime?
|
||||
lockUntil DateTime?
|
||||
|
||||
// 时间
|
||||
startedAt DateTime?
|
||||
finishedAt DateTime?
|
||||
|
||||
// 重试
|
||||
retryCount Int @default(0)
|
||||
maxRetryCount Int @default(3)
|
||||
timeoutSeconds Int @default(120)
|
||||
|
||||
// 错误
|
||||
errorCode String?
|
||||
errorMessage String?
|
||||
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
|
||||
result AiRuntimeResult?
|
||||
|
||||
@@index([status])
|
||||
@@index([jobType])
|
||||
@@index([userId])
|
||||
@@index([targetType, targetId])
|
||||
@@index([lockUntil])
|
||||
}
|
||||
```
|
||||
|
||||
## 5. 锁定机制
|
||||
|
||||
### 5.1 Lock 流程
|
||||
|
||||
```
|
||||
Runtime POST /internal/runtime/jobs/{jobId}/lock
|
||||
→ API 检查 job.status === pending
|
||||
→ API 检查 job.lockUntil < now (未被其他 Runtime 持有)
|
||||
→ API 设置 lockedBy, lockedAt, lockUntil=now+60s, status=locked
|
||||
→ 返回 lockUntil
|
||||
```
|
||||
|
||||
### 5.2 防并发
|
||||
|
||||
基于数据库行级写操作保证只有一个 Runtime 锁定成功：
|
||||
- `UPDATE ... WHERE status='pending' AND (lockUntil IS NULL OR lockUntil < NOW())`
|
||||
- 影响行数 = 0 则锁定失败（JOB_ALREADY_LOCKED）
|
||||
|
||||
### 5.3 Heartbeat
|
||||
|
||||
```
|
||||
Runtime POST /internal/runtime/jobs/{jobId}/heartbeat
|
||||
→ API 检查 lockedBy === runtimeInstanceId
|
||||
→ API 延长 lockUntil = now + 60s
|
||||
→ 204 No Content
|
||||
```
|
||||
|
||||
### 5.4 超时释放
|
||||
|
||||
`lockUntil` 超时后：
|
||||
- 原 Runtime 的 lock 失效
|
||||
- job 状态变为 `expired`
|
||||
- 其他 Runtime poll 时可重新获取（retryable）
|
||||
- 如 retryCount < maxRetryCount，job 自动回到 `pending`
|
||||
|
||||
## 6. 重试策略
|
||||
|
||||
### 6.1 重试触发
|
||||
|
||||
| 场景 | 处理 |
|
||||
|------|------|
|
||||
| Runtime 提交 retryable fail | job → pending, retryCount++ |
|
||||
| Runtime lock 后无 heartbeat 超时 | job → expired → pending, retryCount++ |
|
||||
| Runtime 执行超时 | job → expired → pending, retryCount++ |
|
||||
|
||||
### 6.2 重试上限
|
||||
|
||||
- `retryCount >= maxRetryCount`：job → failed（终态）
|
||||
- `maxRetryCount` 默认 3，可配置
|
||||
- Admin 可手动重跑 failed job（创建新 job，记录 retriedFromJobId）
|
||||
|
||||
### 6.3 retryable vs non-retryable
|
||||
|
||||
| 错误类型 | retryable | 示例 |
|
||||
|---------|-----------|------|
|
||||
| MODEL_TIMEOUT | true | DeepSeek 超时 |
|
||||
| MODEL_RATE_LIMIT | true | 限流 |
|
||||
| NETWORK_ERROR | true | 网络中断 |
|
||||
| TEMPORARY_PROVIDER_ERROR | true | 5xx |
|
||||
| INVALID_SNAPSHOT | false | 快照结构错 |
|
||||
| INVALID_SCHEMA | false | 输出 schema 错 |
|
||||
| INVALID_CREDENTIAL | false | Key 无效 |
|
||||
| JOB_TIMEOUT | true | 执行超时 |
|
||||
|
||||
## 7. 超时
|
||||
|
||||
| 超时类型 | 默认值 | 说明 |
|
||||
|---------|--------|------|
|
||||
| lockUntil | 60s | lock 后未 heartbeat 自动释放 |
|
||||
| timeoutSeconds | 120s | 总执行超时 |
|
||||
| heartbeat 间隔 | Runtime 自行决定 | 建议 15-30s |
|
||||
|
||||
## 8. 幂等
|
||||
|
||||
### 8.1 Job 创建幂等
|
||||
|
||||
`idempotencyKey` 唯一索引：相同 `userId + jobType + targetType + targetId + idempotencyKey` 的 job 不重复创建。如果没有传 idempotencyKey，则允许重复创建。
|
||||
|
||||
### 8.2 Result 提交幂等
|
||||
|
||||
```
|
||||
resultIdempotencyKey = jobId + ":" + attemptNo + ":" + outputHash
|
||||
```
|
||||
|
||||
- 相同 key 重复提交：返回 200（幂等，不重复落库）
|
||||
- 已有 succeeded result 但 outputHash 不同：返回 409 RESULT_ALREADY_EXISTS
|
||||
|
||||
### 8.3 Admin 重跑
|
||||
|
||||
Admin 重跑创建新 job，记录 `retriedFromJobId`，不复用旧 job。
|
||||
|
||||
## 9. Cancelled / Expired
|
||||
|
||||
| 状态 | 能否被 Runtime 消费 | 处理 |
|
||||
|------|-------------------|------|
|
||||
| cancelled | 否 | API 直接设置，不进入 poll 结果 |
|
||||
| expired | 是（如 retryable） | lockUntil 超时后自动变为 expired，retryable 时回到 pending |
|
||||
|
||||
用户关闭 AI 授权时：
|
||||
- 所有 pending job → cancelled
|
||||
- 所有 running job → cancelRequested（Runtime 下次 heartbeat 获知）
|
||||
|
||||
## 10. 任务调度
|
||||
|
||||
### Poll 规则
|
||||
|
||||
```
|
||||
POST /internal/runtime/jobs/poll
|
||||
→ 返回 status=pending 的 job
|
||||
→ 按 priority ASC, createdAt ASC 排序
|
||||
→ 只返回 Runtime capabilities 支持的 jobType
|
||||
→ limit 最大 50
|
||||
```
|
||||
|
||||
### 无可用 job 时
|
||||
|
||||
返回空数组。Runtime 按 pollIntervalMs 等待后重试。
|
||||
|
||||
## 11. 验收清单
|
||||
|
||||
- [x] 输出 Job 状态机设计文档
|
||||
- [x] 明确每个状态的进入条件和退出条件
|
||||
- [x] 明确 Runtime 如何锁定任务（DB 行级写 + lockUntil）
|
||||
- [x] 明确 lockUntil 超时后如何释放
|
||||
- [x] 明确 retryCount / maxRetryCount 规则
|
||||
- [x] 明确 idempotencyKey 防重复
|
||||
- [x] 明确 Admin 可重跑 failed job
|
||||
- [x] 明确 cancelled / expired 不应被 Runtime 再次消费
|
||||
Loading…
x
Reference in New Issue
Block a user