starfield-datastore

A validated artifact cache for the space-data stack. Resolve large, named files through local storage, a configured mirror, and an explicitly enabled upstream archive.

RustMITValidated artifact cache

Start building Read the docs

Trace an artifact request

ORBITALCOMMONS / STARFIELD-DATASTORE

The data behind the discovery.

Star catalogs, planetary maps, and ephemeris kernels can be large and expensive to fetch. starfield-datastore gives those artifacts a common identity, validates their content, and resolves requests through a predictable cache chain.

Local first

Reuse an artifact from a content-addressed local store before reaching for the network.

Validate on access

Rehash and validate cached content. Corruption is reported; repair is explicit.

Explicit upstream

Use a configured mirror by default. Direct archive access is a deliberate opt-in.

Follow an artifact through the cache

Explore how a request is resolved. Choose a scenario to see which layers participate and where the request stops.

Artifact resolutionInteractive illustration

01 · Local diskValidated hit
02 · MirrorNot contacted
03 · UpstreamNot contacted

The cached artifact is rehashed and validated, then its local path is returned.

An illustration of the resolution policy, with no network requests. The optional ephemeris server requires separately deployed infrastructure; it is not hosted by this static website.

Configure your data layer

Library

cargo add starfield-datastore

Install local cache-management commands with the CLI feature:

Terminal

cargo install starfield-datastore --features cli --locked

Resolve a catalog artifact

Rust · describe and resolve an artifact

use starfield_datastore::{
    Artifact, ArtifactKey, ContentCheck, Datastore, Source,
};

fn main() -> starfield_datastore::Result<()> {
    let artifact = Artifact::new(
        ArtifactKey::new("archive/example.bin")?,
        vec![Source::new("https://archive.example/example.bin")],
    ).with_check(ContentCheck::default_binary());

    let store = Datastore::from_env()?;
    let path = store.get(&artifact)?;
    println!("{}", path.display());
    Ok(())
}

The example archive URL is a placeholder. Replace it with your artifact’s real source and configure a mirror or explicitly permit upstream downloads. Add a known digest and artifact-specific checks for stronger validation.

Control where your data comes from

Setting	Behavior
`STARFIELD_CACHE_DIR`	Override the local cache directory.
`STARFIELD_MIRROR`	Set the ephemeris server base URL.
`STARFIELD_ALLOW_UPSTREAM=1`	Permit direct archive downloads.
`STARFIELD_OFFLINE=1`	Use local files only.

The default library features are mirror-http and progress. S3 and server dependencies are opt-in. Returned paths stay valid until an explicit remove or garbage-collection operation; fetches do not evict another consumer’s files.

Read the design → · Deploy your own mirror →

Examples and feature notes reviewed against starfield-datastore 0.1.3. Source & contributions →