build(agent): new-agents-4#58ba63 iteration

This commit is contained in:
agent-58ba63c88b4c9625 2026-04-21 11:21:03 +02:00
parent d95e7ff16a
commit 4f6b7f1226
4 changed files with 111 additions and 26 deletions

View File

@ -1,24 +1,28 @@
# MLTrail: Verifiable Provenance Ledger for Federated ML Experiments (MVP)
# MLTrail Verifiable Provenance Ledger (MVP)
This repository contains a minimal, working MVP of MLTrail, a light-weight,
open-source ledger platform for recording machine-learning experiments across
organizations. It demonstrates core ideas from the original concept: an append-only
hash-chained ledger, compact contract records (Experiment, Run, Dataset, Model,
Environment, EvaluationMetric, Policy), delta-sync primitives, and lightweight adapters.
This repository contains a minimal, productionoriented MVP for a verifiable provenance ledger
tailored to federated machine learning experiments. It offers a compact, append-only hash-chain ledger
with a contract-driven schema registry and a delta-sync protocol to keep distributed networks eventually consistent.
What you get in this MVP:
- Core ledger with cryptographic hash chaining (no external dependencies required)
- Data contracts (Experiment, Run, Dataset, Model, Environment, EvaluationMetric, Policy)
- Reproducibility helpers (environment fingerprint) and a small governance hook
- Two starter adapters (MLFlow-like and WandB-like) to publish records
- A minimal contract registry for schemas and conformance tests scaffold
- Basic delta-sync primitive to simulate cross-partition reconciliation
- CLI/test scaffold for local verification
Key components (MVP):
- Data contracts and schemas for Experiment, Run, Dataset, Model, Environment, EvaluationMetric, Policy.
- Append-only ledger with hash-chain, genesis block, and delta-sync capability.
- Lightweight contract registry (GoC bridge) concepts to enable canonical primitives and replay protection.
- Adapters (MLFlow-like, WandB-like) that publish to the ledger.
- Reproducibility groundwork via environment hashing and deterministic record layout.
- Governance and security scaffolding via signatures (plausible placeholder for MVP).
How to run the MVP locally (quickstart):
- Install Python 3.9+ and run tests with pytest
- See test files under tests/ for guidance
How to use (quick start):
- Run tests and build: ./test.sh
- Publish a registry entry: use the ledger to add a ContractRegistryEntry payload.
- Explore delta-sync: create two ledgers, publish on one, and retrieve deltas on the other with delta_sync.
This is a foundational MVP intended for stepping stones into a broader ecosystem and governance model. Extend it to implement more sophisticated delta-sync, secure anchoring, and adapter ecosystems as needed.
Note: This is an MVP and intentionally lightweight. It focuses on correct data flow and verifiable append-only logs
while keeping the surface area small for rapid iteration and extension (adapters, delta-sync, and governance).
Hook the package into a Python packaging workflow via pyproject.toml (provided).
For contributors: see mltrail_verifiable_provenance_ledger_for/goC_bridge.py for structure of the canonical bridge and
mltrail_verifiable_provenance_ledger_for/ledger.py for the delta-sync primitives.
Version: 0.1.0
"README.md" hooks into packaging via pyproject.toml readme field.

View File

@ -0,0 +1,44 @@
from __future__ import annotations
from typing import Dict, Any
import time
import json
class GoCBridge:
"""Lightweight Canonical Graph of Contracts bridge (GoC).
This is a minimal, language-agnostic bridge outline to map MLTrail primitives
(Experiment, Run, Dataset, Model, Environment, EvaluationMetric, Policy) into
a canonical, replay-protected form. It is intentionally lightweight and intended
for MVP wiring with adapters and the delta-sync protocol.
"""
@staticmethod
def canonical_payload(contract_type: str, payload: Dict[str, Any], version: str = "1.0") -> Dict[str, Any]:
"""Wrap a contract payload into a canonical registry entry payload.
The output is a deterministic structure that can be embedded into records
stored in the ledger as a registry entry.
"""
return {
"canonical_type": contract_type,
"payload": payload,
"version": version,
"timestamp": int(time.time()),
}
@staticmethod
def sign_payload(payload: Dict[str, Any], key: str = "") -> Dict[str, Any]:
"""Placeholder signer to illustrate signatures in the GoC bridge.
In a real deployment, this would produce a cryptographic signature over
the payload. Here we attach a deterministic mock signature for replay protection.
"""
# Very small fake signature; replace with proper crypto in production
ts = payload.get("timestamp", 0)
sig = f"sig:{ts}"
return {"payload": payload, "signature": sig}
@staticmethod
def to_json(entry: Dict[str, Any]) -> str:
return json.dumps(entry, sort_keys=True)

View File

@ -1,12 +1,6 @@
#!/usr/bin/env bash
set -euo pipefail
echo "Running Python tests with pytest..."
# Ensure the repository root is on PYTHONPATH so tests can import the package
export PYTHONPATH="${PYTHONPATH:+$PYTHONPATH:}$(pwd)"
# Run tests and packaging build as a quick quality gate
pytest -q
echo "Building Python package..."
python3 -m build
echo "ALL TESTS PASSED"

43
tests/test_registry.py Normal file
View File

@ -0,0 +1,43 @@
import json
from mltrail_verifiable_provenance_ledger_for.ledger import Ledger
def test_contract_registry_entry_can_be_stored():
ledger = Ledger()
payload = {
"registry_name": "GoC-Bridge-Registry",
"primitives": [
"Experiment",
"Run",
"Dataset",
"Model",
"Environment",
"EvaluationMetric",
"Policy",
],
"version": "1.0",
}
block = ledger.add_record("ContractRegistryEntry", payload)
# Basic sanity checks about the added block and its data
assert block.index == 1
assert block.data[0]["type"] == "ContractRegistryEntry"
# Ensure the payload is preserved inside the contract record
stored = block.data[0]
assert stored["registry_name"] == payload["registry_name"]
assert stored["primitives"] == payload["primitives"]
def test_delta_sync_basic():
ledger_a = Ledger()
ledger_b = Ledger()
# Add a single registry entry in ledger_a
ledger_a.add_record("ContractRegistryEntry", {"registry_name": "GoC", "version": "1.0"})
# Use ledger_b's head hash as remote head to fetch delta from ledger_a
delta = __import__("mltrail_verifiable_provenance_ledger_for.ledger", fromlist=["delta_sync"]).delta_sync(ledger_a, ledger_b.head_hash())
# Should contain the single block from ledger_a (excluding genesis)
assert isinstance(delta, list)
assert len(delta) == 1
assert delta[0]["index"] == 1