Registry10x — Full Feature List

What’s New — October 2026

Shipped today

Supply-chain substrate for rebuild, promote, and admission: a tamper-evident registry audit log and cosign-compatible image signing with verify-on-pull. Both ship behind kill-switches defaulting to OFF.

🔐 Cosign image signing + ImageVerificationPolicy New

registry10x-signing signs and verifies manifest digests with keyed cosign simple-signing (sigstore-rs). Signatures store as OCI artifacts under the cosign tag convention sha256-<hex>.sig, with Postgres metadata, OCI referrers, and a typed Sign audit event. Keys resolve from PEM files or HashiCorp Vault (KV + Transit).

An ImageVerificationPolicy CRD (and matching admin API) scopes which projects and repositories require a trusted signature. On pull, get_manifest enforces the policy: block returns 403 DENIED; warn / log audit and allow. Flip with [signing] enabled = true in Helm or config.

Signing details Design 001

📝 Tamper-evident chained audit log New

Registry write ops dual-write into registry10x-audit: an HMAC-SHA256 hash chain over registry_audit_log (separate from the scanner’s audit_log). Push and delete fail closed when the Postgres backend is configured and unavailable. Query, verify, and export over /v2/_audit, with Merkle checkpoints for off-box retention.

Audit details Design 005

✍ registry10x-sign CLI New

Keygen, sign, and verify against a digest with file://, vault://kv/…, or vault://transit/… key refs. Emits a SignatureArtifact ready to POST to /v2/…/signatures/{digest}.

🔍 POST /v2/_verify New

List signatures for a subject digest or cryptographically verify against trusted key refs. Complements verify-on-pull for CI and agent workflows.

📦 Multipart large-layer uploads

Multi-GB blobs write as multipart uploads so a single request deadline never caps object size. Shipped earlier in #35 — see Large Layers.

⚙ Kill-switches default OFF

[signing] enabled = false and [audit] backend = "memory" keep existing clients unchanged until you opt in. Helm values mirror the same defaults.

Change	Slice	Impact
Chained registry audit + `/v2/_audit`	A1–A3	Tamper-evident trail; push fails closed if PG audit is down
Audit export + Merkle checkpoint	A3	Off-box JSONL retention with verifiable roots
`registry10x-signing` keyed cosign (sigstore-rs)	B1–B2	Sign/verify digests with PEM or Vault keys
Signature store + cosign `.sig` tags + referrers	B3	Push signatures via API or OCI tag; audit `Sign`
`ImageVerificationPolicy` + verify-on-pull	B4	Unsigned pulls blocked (or warned) per policy
Multipart large-layer uploads	#35	Multi-GB layers push instead of failing with a 500

Image Signing & Verify-on-Pull

New — Oct 2026

Pure-Rust cosign simple-signing via sigstore-rs. Kill-switch: [signing] enabled = false (Helm signing.enabled). When enabled, signatures attach as OCI artifacts; policies gate pulls.

🔑 Keyed signing

ECDSA P-256 and ed25519 via sigstore-rs
Key refs: file://, vault://kv/…, vault://transit/…
registry10x-sign keygen | sign | verify CLI
Notation stub reserved for a later release

📦 Storage convention

Cosign tag sha256-<hex>.sig in the same repo
POST/GET /v2/{tenant}/{project}/{repo}/signatures/{digest}
OCI 1.1 referrers registration
signatures table for metadata and listing

🛡 Verify-on-pull

ImageVerificationPolicy CRD with project/repo scope
onViolation: block, warn, or log
require_for_pull global fallback when no CRD matches
Admin sync: PUT /v2/_signing/policies/{tenant}/{name}

Config / API	Purpose
`signing.enabled`	Master kill-switch (default `false`)
`signing.require_for_pull`	Refuse unsigned pulls when no scoped policy matches
`POST /v2/_verify`	List or cryptographically verify signatures for a digest
`ImageVerificationPolicy`	Namespaced CRD reconciled into `verification_policies`

# Enable signing (Helm)
helm upgrade registry10x oci://ghcr.io/bwalia/charts/registry10x \
  --set signing.enabled=true \
  --set signing.requireForPull=false

Chained Registry Audit

New — Oct 2026

Durable, verifiable audit for registry mutations. Tables use the registry_audit_* prefix so they never collide with the scanner’s audit_log.

🔗 Hash chain

HMAC-SHA256 row chaining (registry10x-audit)
Categories: auth, push, pull, delete, sign, promote, policy, …
In-memory backend for tests; Postgres for production
Write ops fail closed when the PG sink is unavailable

🔍 Query & verify

GET /v2/_audit — filter by category / subject
GET /v2/_audit/{id} — single chained row
POST /v2/_audit/verify — verify a range
Admin-role gated

📤 Export

POST /v2/_audit/export — pending or ranged dump
Merkle checkpoint roots for retention evidence
Filesystem or in-memory export destinations
Optional background export loop

Registry Core & OCI Distribution API

Stable

A full implementation of the OCI Distribution Specification, compatible with Docker, Podman, Buildah, containerd, and any OCI-compliant client. One Rust binary, no runtime dependencies.

📦 Image operations

Manifest push, pull and delete (v2 schema, manifest lists)
Chunked (PATCH) and monolithic blob uploads
Content-addressable blob storage keyed by digest
Tag listing, pagination and the /v2/_catalog API
Cross-repository blob mounting

🔗 Path models

Standard 2-segment Docker paths (org/image)
Multi-tenant 3-segment paths (tenant/project/image)
Default tenant applied automatically for 2-segment pushes
Works unchanged with docker, podman and containerd

⚡ Runtime

Rust + Axum + Tokio, fully async I/O
Single static binary, small memory footprint
Thousands of concurrent pulls per replica
Registry on :5000, auth on :5001

Large Layers & Blob Uploads

Updated Sept 2026

Machine-learning and data-science images routinely carry layers of a gigabyte or more. Registry10x writes those blobs as multipart uploads so a single request deadline never caps object size.

🔁 Multipart write path

Applied at every blob write site in the registry — monolithic PUT, the PATCH-appended buffer, and upload completion.

Each part carries its own request and retry budget
Parts uploaded concurrently to keep a WAN link busy
Payloads at or below the threshold keep the original single-PUT path
Backend-agnostic: works on S3, MinIO, GCS, Azure and filesystem

🛠 Guard rails

5 MiB minimum part size enforced for non-final parts
Part size grown when an object would need more than 10,000 parts
Aborted upload on failure, so no orphaned parts are billed
Unit tests plus S3 integration tests against MinIO

✅ Verified end to end

180 MiB layer pushed through the real OCI endpoints into MinIO
Pulled back byte-identical on both MinIO and filesystem backends
No dangling multipart uploads left in the bucket
Small blobs and manifests confirmed on the unchanged path

Config key	Environment override	Default	Purpose
`storage.multipart_threshold_mb`	`REGISTRY10X_STORAGE_MULTIPART_THRESHOLD_MB`	64	Blobs above this size use a multipart upload
`storage.multipart_part_size_mb`	`REGISTRY10X_STORAGE_MULTIPART_PART_SIZE_MB`	32	Size of each part (raised to 5 MiB if lower)
`storage.multipart_concurrency`	`REGISTRY10X_STORAGE_MULTIPART_CONCURRENCY`	4	Parts uploaded at once
`storage.retry_max_retries`	`REGISTRY10X_STORAGE_RETRY_MAX_RETRIES`	10	Retries per storage request
`storage.retry_timeout_secs`	`REGISTRY10X_STORAGE_RETRY_TIMEOUT_SECS`	180	Deadline for one request including retries
`storage.connect_timeout_secs`	`REGISTRY10X_STORAGE_CONNECT_TIMEOUT_SECS`	30	TCP connect timeout
`storage.request_timeout_secs`	`REGISTRY10X_STORAGE_REQUEST_TIMEOUT_SECS`	300	Total timeout for one request attempt

# Registry on a slow link to its bucket: smaller parts finish sooner
kubectl -n registry10x set env deploy/registry10x-registry \
  REGISTRY10X_STORAGE_MULTIPART_THRESHOLD_MB=32 \
  REGISTRY10X_STORAGE_MULTIPART_PART_SIZE_MB=8 \
  REGISTRY10X_STORAGE_MULTIPART_CONCURRENCY=8

Diagnostics for large-push failures are in docs/troubleshooting.md, alongside the 429 and 413 sections.

Storage Backends & Tuning

Stable

Storage is an abstraction over the object_store crate, so the same registry binary runs on a laptop volume or a multi-region bucket without a code change.

Backend	Value	Key settings
Local filesystem	`filesystem`	`root`, PVC size
Amazon S3 / MinIO / S3-compatible	`s3`	`bucket`, `region`, `endpoint`, `encrypt`
Google Cloud Storage	`gcs`	`bucket`, `keyfile`
Azure Blob Storage	`azure`	`container`, `accountName`

💾 Storage behaviour

Content-addressable layout shared by every backend
Per-deployment backend selection, no rebuild required
Multipart writes for large objects (see above)
Circuit-breaker and retry wrapping on every storage call

🗑 Lifecycle

Manifest and tag deletion through the OCI API
Retention policies configurable per project
Online garbage collection refcounting (see Incubating)
Audit trail for every push, pull and delete

High Availability & Resilience

Stable

Both services are stateless, so availability is a matter of running more replicas. What sits underneath is the part that matters: retries, breakers and failover that distinguish a real fault from ordinary traffic.

♻ Stateless scale-out

Registry and auth scale horizontally behind one service
HorizontalPodAutoscaler for both deployments
PodDisruptionBudgets and pod anti-affinity
No sticky sessions — any replica serves any request

🔌 Circuit breakers

Per-dependency breakers with configurable thresholds
Classified failures — NotFound counts as healthy
Half-open probing before a breaker closes again
Breaker state exported as metrics

🔄 Retries & timeouts

Exponential backoff with jitter
Explicit connect, request and retry deadlines
Per-part retry budgets on large uploads
Bounded so a slow dependency cannot queue indefinitely

🚦 Rate limiting

Per-tenant and per-IP request limits
Separate limit for token issuance
Env-overridable at runtime without a config file
Per-tenant quotas from the Tenant CRD

🩺 Health & readiness

/health endpoint for probes and load balancers
Region health loop with latency and failure counts
Automatic read failover to healthy regions
Health surfaced in the dashboard

💿 Data durability

Object storage durability for blobs and manifests
Async replication to secondary regions
Multipart uploads aborted cleanly on failure
Digest verification on write and on read

Multi-Region Replication

Stable

Manifests and blobs replicate across regions on an event-driven pipeline, with failover handled by the same resilience layer the storage path uses.

🌍 Replication engine

Async and semi-sync replication modes
Event-driven — a push enqueues replication work
Primary and secondary region roles
Backoff and retry on partitioned links

🛟 Failover

Health check loop per region
Reads served from the nearest healthy region
Circuit breaker isolates a failing region
Failover state visible in the dashboard and metrics

# Three-replica registry with multi-region enabled
helm install registry10x oci://ghcr.io/bwalia/charts/registry10x \
  --set registry.replicas=3 \
  --set auth.replicas=2 \
  --set multiRegion.enabled=true \
  --set multiRegion.localRegion=us-east-1 \
  --set multiRegion.isPrimary=true

Pull-Through Cache

Stable

Mirror upstream registries transparently to cut egress costs and survive upstream rate limits.

🔍 Upstreams

Docker Hub, GHCR, GCR, Quay.io, registry.k8s.io
Authenticated upstreams to avoid Docker Hub pull limits
Any additional OCI-compliant upstream by config
Zero client-side changes — plain docker pull

💽 Cache behaviour

Automatic manifest and blob caching on first pull
TTL-based cache invalidation
containerd mirror configuration supported
Cached images are scanned like any other image

Authentication & Identity

Enterprise

Zero static credentials in production. Identity comes from your existing provider, and every token is short-lived and scoped.

🏢 SSO / OIDC

Authorization Code flow with PKCE
Azure AD / Entra ID, Okta, Google, any OIDC provider
Auto-provision users on first login
Group-to-role mapping, per tenant and per project

🤖 Machine identity

CI/CD OIDC federation: GitHub Actions, GitLab CI, Kubernetes SAs
Robot accounts with configurable expiry
API keys (nck_) with per-permission scoping
No static secrets in pipelines

🔑 Tokens

Docker-compatible token exchange at /auth/token
RS256 / EdDSA signed JWTs, JWKS published for verification
Short-lived by default (5–15 minutes)
Refresh tokens with instant revocation

📋 SCIM 2.0 provisioning

RFC 7644 Users and Groups endpoints
Pre-provision access before first login
Instant deprovisioning on offboarding
Group membership changes sync without a new login

🔐 Key management

HashiCorp Vault integration (Transit + KV)
Mounted PEM keypair or Vault-managed signing keys
Key rotation without downtime
Bootstrap admin for first-run setup only

🛡 Authorization

RBAC roles: Admin, Maintainer, Reader
AccessPolicy CRD with subjects, resources, actions, conditions
Scope-checked per repository operation
IP restrictions per tenant

Multi-Tenancy & Governance

Enterprise

One registry serves many teams, each with its own quotas, policies and isolation, declared as Kubernetes resources so they live in Git alongside everything else.

CRD	Scope	Purpose
`Tenant`	Cluster	Top-level org with quotas, IP restrictions, OIDC mapping
`Project`	Namespace	Groups repositories; visibility, retention, scanning policy
`AccessPolicy`	Namespace	Fine-grained RBAC: subjects, resources, actions, conditions
`TokenPolicy`	Namespace	Token lifetime, rotation, revocation, robot accounts
`ImageVerificationPolicy`	Namespace	Require cosign signatures on pull; block / warn / log

📏 Quotas

Max storage bytes and max repositories per tenant
Pull and push rate per minute
Enforced at the API edge
Usage telemetry recorded per tenant

📝 Audit

HMAC-chained registry audit (see Chained Registry Audit)
Categories include push, delete, sign, and policy
Filterable audit view in the dashboard
Webhook notifications for registry events

Vulnerability Scanning & SBOM

Stable

Scanning is built into the registry, not bolted on. Every push is scanned in-process by registry10x-scanner — no Trivy daemon, no Grype shell-out, no external scanner pod.

🔎 SBOM extraction

OS packages: dpkg, apk, rpm
Language deps: npm, PyPI, Cargo, Go modules
ELF metadata fallback for un-packaged binaries
CycloneDX SBOM output, exportable to S3

📚 Vulnerability data

OSV as the bootstrap source
Registry10xVulnDb: local Postgres mirror of OSV, GHSA and NVD
Ecosystem-aware matchers: apk, deb, rpm, semver, PEP 440, Go
Scheduled ingest via /admin/vulndb/ingest

⚖ Noise control & policy

CycloneDX VEX 1.4 ingestion (not_affected / fixed / under_investigation)
Suppressions with audit trail
Pass/fail policy gate with per-severity thresholds
Standalone policy evaluation endpoint

📡 Delivery

Results in Redis with a 1-hour TTL, keyed by digest
WebSocket progress stream for live UIs
Cross-image CVE search
Slack notifications and nightly PDF reports

Endpoint	Purpose
`GET /v2/scan/live/{digest}`	Poll-until-complete scan result
`POST /v2/scan`	Trigger a scan for a (repo, tag)
`GET /v2/scan/{id}/report`	Rendered scan report
`GET /v2/scan/{id}/sbom`	CycloneDX SBOM
`GET /v2/cve/search`	Cross-image CVE search
`POST /v2/cve/suppress`	Manage suppressions (audited)
`POST /v2/vex`	Ingest CycloneDX VEX statements
`POST /v2/policy/evaluate`	Standalone policy evaluation
`POST /v2/export/s3/{id}`	Export a scan to an S3 prefix

Nightly automation runs as an in-cluster CronJob (03:17 UTC) that enumerates /v2/_catalog and scans every (repo, tag), rolls the results into a PDF, and uploads it to Slack.

AI-Assisted Remediation

Stable

CVE commentary is generated by a local LLM through Ollama, so image contents and findings never leave your infrastructure.

🤖 Analysis

Per-CVE root cause and impact summary
On-demand, not on every push — no idle GPU cost
Runs against a local Ollama endpoint
No third-party API calls

🔧 Fix suggestions

Dockerfile rewrite suggestions (/v2/scan/{id}/dockerfile-fix)
One-click GitHub PR review comment posting
Findings linked back to the SBOM component
Report export for audit evidence

Dashboard & Operations

Stable

A dark-themed operations dashboard is embedded in the registry binary and served at /dashboard — nothing extra to deploy.

📊 System metrics

CPU, RAM and disk with colour-coded thresholds
HA region health, latency and failure counts
Push/pull activity over time
Live refresh

📦 Image browser

Repository search with wildcards
Tag lists, tag counts and digests
Per-image scan findings panel
Deep links into scan reports

👥 Identity & audit

Users, groups and service accounts
SSO logins and group-role mappings
Robot account status and expiry
Filterable audit log

Observability

Stable

Prometheus metrics, structured logs and OpenTelemetry traces, with Grafana dashboards in the repo.

Metric	Meaning
`registry10x_http_requests_total`	Request count by method, path and status
`registry10x_http_request_duration_seconds`	Request latency histogram
`registry10x_storage_operations_total`	Storage operations by backend
`registry10x_auth_tokens_issued_total`	Token issuance count

📈 Metrics & tracing

Registry metrics on :9090, auth on :9091
ServiceMonitor for Prometheus Operator
OpenTelemetry OTLP trace export
Grafana dashboards under deploy/observability

📃 Logging

Structured JSON logs
Request IDs correlated across services
Storage errors logged with the failing object path
Troubleshooting guide keyed to real log lines

Kubernetes & Deployment

Stable

Helm is the primary path, but the same binary runs under Docker Compose, Ansible on bare metal, or directly on a host.

☸ Helm chart

OCI chart at oci://ghcr.io/bwalia/charts/registry10x
CRDs for Tenant, Project, AccessPolicy, TokenPolicy
HPA, PDB, anti-affinity, ServiceMonitor
Ingress with TLS termination

🐳 Other targets

Docker Compose stack for local development
Ansible playbooks for native deployment
Single binary with a YAML/TOML config file
Env-var overrides for the settings k8s needs to tune

🛡 Cluster hardening

NetworkPolicy templates
Non-root containers, read-only root filesystem
Distroless/scratch image variant
Secrets sourced from Vault or the cluster secret store

CI/CD Integration

Stable

Ready-to-copy examples for every major platform, all under examples/ in the repository.

Platform	Auth method	Example
GitHub Actions	OIDC (zero secrets)	`examples/github-actions/push-image.yml`
GitLab CI	OIDC	`examples/gitlab-ci/push-image.yml`
Jenkins	Token-based	`examples/jenkins/Jenkinsfile`
Tekton	ServiceAccount	`examples/tekton/push-task.yml`
ArgoCD	Image updater	`examples/argocd/image-updater.yml`

Platform & Packaging

Stable

📥 Distribution

Docker Hub: bwalia/registry10x
GHCR: ghcr.io/bwalia/registry10x
Tags: latest, vX.Y.Z, X.Y, edge
linux/amd64 and linux/arm64, arm64 built on native runners

🦀 Build

Cargo workspace of focused crates
Standard image plus a scratch/distroless variant
CI gate: fmt, clippy -D warnings, full test suite
Apache-2.0 licensed, open source

Incubating

In development

These crates exist in the workspace with their traits, models and database schema landed, and are being completed in slices. They are listed for transparency — do not plan a production rollout around them yet.

Capability	Crate	Landed so far
Online garbage collection	`registry10x-gc`	Live blob refcount writer, atomic with manifest mutations
Lazy pull / partial layers	`registry10x-lazy`	TOC indexer trait and media types (eStargz, zstd-chunked, SOCI)
P2P pull acceleration	`registry10x-peer`	Peer mesh trait, in-process mesh, membership store
Usage & cost telemetry	`registry10x-cost`	Usage event model and Postgres recorder
Build attestation (SLSA/DSSE)	`registry10x-attest`	DSSE envelope parser, SLSA level extractor, attestation store
Ephemeral repos & TTL tags	`registry10x-ephemeral`	TTL header parser, per-project default cap, schema
Auto-rebuild on base CVE patch	`registry10x-rebuild`	Lineage extractor; subscriptions + CI dispatch still incubating
Promotion API	(design 006)	Depends on signing + audit; cross-repo copy next
Admission gate	(design 002)	Depends on signing verify-on-pull + scanner policy
Registry migration import	`registry10x-import`	Source trait plus an OCI Distribution adapter; Nexus/Harbor/ACR next
Typed artifacts	`registry10x-artifact-types`	Validator registry and Helm chart validator
Registry AI agent	`registry10x-pilot`	Tool registry with four read-only tools; MCP surface next

Full Feature List

Contents

What’s New — October 2026

🔐 Cosign image signing + ImageVerificationPolicy New

📝 Tamper-evident chained audit log New

Image Signing & Verify-on-Pull

Chained Registry Audit

Registry Core & OCI Distribution API

Large Layers & Blob Uploads

Storage Backends & Tuning

High Availability & Resilience

Multi-Region Replication

Pull-Through Cache

Authentication & Identity

Multi-Tenancy & Governance

Vulnerability Scanning & SBOM

AI-Assisted Remediation

Dashboard & Operations

Observability

Kubernetes & Deployment

CI/CD Integration

Platform & Packaging

Incubating

Run it yourself