CortexDB 0.9.10 — provenance-first ingestion and durable memory artifacts

CortexDB 0.9.10 adds a provenance-first ingestion path and an opt-in durable artifact plane. Raw experiences can retain exact source structure while CortexDB builds typed, traceable artifacts and derived projections for richer retrieval.

The coordinated client releases are Python SDK 0.11.2, TypeScript SDK 0.9.3, and cortexdb-connectors 0.2.21.

The server, SDKs, and connectors are released under the Apache License 2.0.

Highlights

• Added optional CanonicalSourceV2 metadata for conversation, document, JSON, table, code, log, and media fragments, with algorithm-qualified fingerprints and exact structural locators.
• Added Python and TypeScript validation plus lossless UTF-8 source-fragment helpers. Existing one-event experience APIs and envelopes remain supported.
• Added durable artifact compilation with deterministic identities, provenance, coverage manifests, and replaceable episode, story-arc, entity-dossier, rollup, and working-memory projections.
• Added opt-in lexical and semantic artifact retrieval that projects results back to authorized raw evidence. Adaptive context supports default-off shadow and serve rollout modes.
• Added explicit compiler bounds for source bytes, sampled windows, output tokens, concurrency, retries, and total physical provider attempts. Incomplete work remains visibly Partial or Failed instead of being silently truncated or reported complete.
• Added per-task LLM routing, timeouts, usage accounting, durable retry and dead-letter state, and conservative long-source handling. Unified entity and knowledge extraction remains opt-in.

Durable connectors and source lifecycle

• Connectors now record accepted event IDs, blob IDs, source generations, and opaque idempotency tokens in a local SQLite lifecycle ledger without storing source text or attachment bytes.
• Ledger namespaces include the authenticated CortexDB tenant and may include a stable CORTEXDB_SOURCE_INSTANCE_ID, preventing same-provider accounts from sharing source identity accidentally.
• Replacement content is accepted before older generations are retired. Failed derived or raw cleanup remains durable and blocks cursor advancement for retry.
• Binary attachments use the new keyed-blob-v1 preflight, upload, and retire protocol so retries reuse one durable upload identity. New connector writes never fall back to unkeyed uploads.

ACL and erasure safety

• Verified callers may auto-register only tenant-owned scopes or paths explicitly granted by their signed token. The nearest registered ancestor governs descendant creation and membership.
• Cached answer and compose packs revalidate current traversal and ancestor authorization and are fenced by forget and erasure visibility changes.
• Import-job status and scope-less import lifecycle events are bound to both the authenticated billing tenant and initiating actor. Possession of an import or lifecycle ID is not authorization, and missing legacy bindings fail closed for verified callers.
• Import and synthesis jobs use bounded admission and bounded terminal-result retention. Manual synthesis is genuinely asynchronous and requires scoped Writer, scope.write, understanding.synthesize, and llm.invoke authority; polling revalidates Reader access without exposing foreign IDs.
• Scope export snapshots every descendant ACL/tombstone boundary, authorizes each included event, runs on the bounded storage lane, and revalidates the authorization revision before releasing serialized data.
• Descendant recall plus lifecycle catch-up/SSE filter every exact registered child boundary the caller cannot read. Subtree forget and erasure require owner role and the scoped destructive capability at every affected boundary, then revalidate the registry revision while the operation runs.
• Tenant scope pruning considers only verified durable write activity, is bounded at the registration, scope-key, partition, and event levels, and cannot report a timeout while destructive work continues. Concurrent write admission prevents a registration from being pruned before WAL publication.
• Scope membership updates cannot leave an ownerless roster. Lifecycle cancel holds the exact-scope authorization guard through mutation, and verified callers cannot observe unowned scope-less lifecycle events.
• Forget and erasure close over compiled artifacts, manifests, search projections, dossiers, episodes, arcs, rollups, and working memory. Durable tombstones and erasure-generation fences prevent stale compiler or WAL replay from resurrecting erased content.
• Automatic production LLM response caching is retired because cached customer-derived content sat outside authoritative retention and erasure flows.

Compatibility and rollout

• Artifact dual-write, artifact search, semantic artifact search, adaptive context, unified extraction, long-source fan-out, relation verification, and LLM rollup summaries are all off by default.
• The artifact compiler uses the full profile by default when enabled; explicit settings override profile defaults.
• Artifact and source fusion defaults to compatibility-preserving append mode.
• Auth-disabled and static-operator self-hosted modes retain root-agnostic first-write behavior.

Upgrade and rollback notes

• A single experience whose flattened indexable content exceeds 1 MiB now returns 422 instead of relying only on the 32 MiB HTTP request limit. Migrate larger text writes to the lossless fragmentation helpers experience_source_text (Python) or experienceSourceText (TypeScript), or use the blob ingestion flow for binary content.
• Standalone v0.9.10 tarball binaries require glibc 2.39 or newer and the OpenSSL 3 runtime (libssl.so.3). Use the Docker image when the host does not provide that Ubuntu 24.04-class userspace.
• Upgrade the server before connectors that ingest binary attachments. keyed-blob-v1 is single-node only and returns 503 in cluster mode; older servers fail connector preflight without receiving attachment bytes.
• Back up and retain connector_event_ledger.sqlite3. Keep its path and CORTEXDB_SOURCE_INSTANCE_ID stable; changing either identity creates a new lifecycle namespace. Pre-keyed blob-ID rows remain deletable through the legacy route.
• Use one authoritative connector writer per source namespace unless writers share the same correctly locked ledger. Independent hosts are not coordinated.
• Startup removes the owned legacy extraction-response cache. A custom cache path is deleted only after strict schema validation; otherwise startup fails for manual review. This cleanup is not reversible.
• Verified tokens that previously relied on first-write creation of standalone ws:* or cross-tenant scopes must explicitly register those scopes or receive a signed path-specific write grant.
• DELETE /v1/scopes removes registration only. records=forget now returns 422 UNSUPPORTED_RECORDS_ACTION; complete POST /v1/forget or POST /v1/erasures first when stored data must also be removed.
• Import-status and lifecycle polling must use the same billing tenant and actor principal that created the import; opaque job IDs are not transferable credentials.
• Scope-registry snapshots now persist a server-derived billing-tenant binding and enforce it across registration, read/export, write admission, listing, member edits, and deletion. Legacy unbound org:* records are recovered from their org root; legacy unbound standalone records fail closed for verified tenants because their owner cannot be inferred safely. A trusted local operator can remove and re-register only the standalone registration under the intended tenant without implying deletion of its stored records.
• A binary-only downgrade of an artifact-enabled data directory to v0.9.9 is not supported. Snapshot the data directory before enabling the artifact plane.

Validation

Release dependencies include patched h2 and server-owned lru versions for the current RustSec findings. The older lru copy retained by Tantivy does not call the affected API; its narrow exception is documented in deny.toml until Tantivy 0.27 or later is adopted.

Regression coverage includes canonical-source parity across the server and SDKs, connector restart and supersession flows, keyed-blob failures, compiler budget exhaustion, artifact WAL replay and erasure, scope admission, and cached-pack invalidation. No benchmark uplift or distributed keyed-blob guarantee is claimed by this release.