Progressive Enrichment

Resources evolve through enrichment stages over their lifecycle. The platform implements a monotonic, non-regressing enrichment pipeline where each request potentially advances the canonical resource toward full enrichment.

Enrichment Lifecycle

Initial Request                    Background Workers              Subsequent Request
─────────────────                  ──────────────────              ────────────────────
semantic       ✓ completed         preview:thumbnail  → complete  All stages completed
mediaFormat    ✓ completed         preview:blurhash   → complete  completion: 1
metadata:image ✓ completed         preview:palette    → complete  status: "complete"
preview:*      ⏳ pending                                          pendingStages: []

Step-by-Step

1. Initial request — The platform immediately resolves semantic classification, media format detection, and category-specific metadata. For heavy enrichments (preview generation, palette extraction), background workers are dispatched.

2. Background processing — Workers generate thumbnails, compute blurhash placeholders, extract color palettes, and perform other intensive enrichments asynchronously.

3. Subsequent request — The same resource now returns with all enrichments completed. The canonical state has evolved.

Processing State Contract

Every response includes a processing block that reflects the canonical enrichment registry:

{
  "processing": {
    "status": "partial",              // "partial" | "complete"
    "completion": 0.5,                // 0.0 → 1.0, monotonically non-decreasing
    "enrichmentsPending": true,       // boolean
    "enrichmentStatus": "partial",    // mirrors status
    "completedStages": [              // canonical stage names
      "semantic",
      "mediaFormat",
      "metadata:image"
    ],
    "pendingStages": [                // stages awaiting worker completion
      "preview:thumbnail",
      "preview:blurhash",
      "preview:palette"
    ]
  }
}

Processing State Rules

Rule	Behavior
status: "complete"	completion is 1, pendingStages is [], enrichmentsPending is false
status: "partial"	completion is < 1, pendingStages has entries, enrichmentsPending is true
Stage exclusivity	A stage appears in exactly one of completedStages or pendingStages, never both
Profile isolation	Processing state is identical across FAST/STANDARD/FULL for the same resource

Non-Regression Guarantee

Invariant

Enrichment completion is monotonically non-decreasing. Once a stage is marked completed, it never reverts to pending — regardless of profile changes, cache interactions, or time.

Request 1 (FAST)  → completion: 0.5, completedStages: ["semantic", "mediaFormat"]
  ↓ wait
Request 2 (FULL)  → completion: 1.0, completedStages: ["semantic", "mediaFormat", "metadata:image", "preview:*"]

The second request must report completion ≥ the first request. This is validated by the test suite's assertEnrichmentNonRegression invariant.

Profile-Specific Behavior

FAST on Heavy Media

Returns immediately with partial enrichment. Background workers are dispatched but the response doesn't wait.

// FAST on video — partial status
{
  "processing": {
    "status": "partial",
    "completion": 0.33,
    "enrichmentsPending": true,
    "completedStages": ["semantic", "mediaFormat"],
    "pendingStages": ["metadata:video", "preview:thumbnail", "preview:blurhash", "preview:palette"]
  }
}

STANDARD on Heavy Media

Returns partial results immediately with 202 Accepted and a job reference:

// STANDARD on video — 202 Accepted
{
  "success": true,
  "responseSource": "fresh",
  "processing": {
    "status": "partial",
    "enrichmentsPending": true
  },
  "job": {
    "id": "job_abc123",
    "status": "processing"
  },
  "data": {
    "identity": { /* always available */ },
    "semantic": { /* always available */ }
    // partial data — enrichments still running
  }
}

FULL — Synchronous Completion

Blocks until all enrichments complete. Always returns 200 OK with completion: 1.

// FULL on image — deterministic complete
{
  "processing": {
    "status": "complete",
    "completion": 1,
    "enrichmentsPending": false,
    "completedStages": ["semantic", "mediaFormat", "metadata:image", "preview:thumbnail", "preview:blurhash", "preview:palette"],
    "pendingStages": []
  }
}

FULL Upgrades Partial Cache

If a prior FAST request left a partial cache record, a subsequent FULL request will synchronously complete the remaining enrichments and return a fully enriched response:

FAST (creates partial cache) → FULL (upgrades to complete, returns fresh)

The FULL response reports responseSource: "fresh" because it performed synchronous enrichment work, even though it leveraged existing cached partial data.

Enrichment Stages

The platform defines canonical stage names that appear in completedStages and pendingStages:

Stage	Category	Description
semantic	All	Semantic classification and kind detection
mediaFormat	All	MIME type, container, codec, extension detection
oembed	Provider	oEmbed metadata retrieval
metadata:image	Image	Color space, alpha channel, dimensions
metadata:video	Video	Duration, FPS, bitrate, dimensions
metadata:audio	Audio	Duration, bitrate, sample rate
preview:thumbnail	Image, Video	Thumbnail generation
preview:blurhash	Image, Video	Blurhash placeholder computation
preview:palette	Image, Video	Dominant color palette extraction

Stage names follow the pattern namespace or namespace:detail and contain only alphanumeric characters, colons, dots, and hyphens.

Payload-Registry Agreement

Contract Enforcement

If enrichment data exists in the response payload, the registry must report that stage as completed. Payload-registry disagreement is forbidden.

For example, if data.preview.thumbnail.url contains a value, then preview:thumbnail must appear in completedStages. A pending stage must never have its data already present in the payload.