Package lifecycle

Infer, review, write, read, and validate an SDP.

Choose the entry point

Use create_sdp() for normal work. It infers metadata, optionally seeds semantic suggestions, writes the package, and adds a review checklist.

Use infer_salmon_datapackage_artifacts() when you need to inspect or edit the inferred DataFrames before writing. Use write_salmon_datapackage() only when the metadata tables are already assembled.

Multi-table packages

resources = {
    "catch": catch_df,
    "effort": effort_df,
}

path = create_sdp(
    resources,
    path="survey-sdp",
    dataset_id="survey",
    seed_semantics=False,
)

Each mapping key becomes a table id and each resource is written under data/.

Inspect before writing

from metasalmonpy import (
    infer_salmon_datapackage_artifacts,
    write_salmon_datapackage,
)

artifacts = infer_salmon_datapackage_artifacts(
    resources,
    dataset_id="survey",
    seed_semantics=False,
)

artifacts["dataset_meta"].loc[0, "title"] = "Annual salmon survey"
artifacts["dataset_meta"].loc[0, "description"] = (
    "Catch and effort observations from the annual survey."
)

path = write_salmon_datapackage(
    resources=artifacts["resources"],
    dataset_meta=artifacts["dataset_meta"],
    table_meta=artifacts["table_meta"],
    dict_df=artifacts["dict"],
    codes=artifacts["codes"],
    path="survey-sdp",
)

The returned artifact mapping also carries semantic_suggestions and semantic_llm_assessments when those stages are enabled.

Safe overwrite behavior

overwrite=True replaces an existing directory only when metasalmonpy can recognize it as an owned package. It refuses to delete arbitrary non-package directories. This check is based on the package sentinel or the required SDP metadata files.

create_sdp(
    resources,
    path="survey-sdp",
    dataset_id="survey",
    overwrite=True,
)

Compatibility aliases

create_salmon_datapackage() and create_salmon_datapackage_from_data() remain available for older callers. New code should use write_salmon_datapackage() and create_sdp() respectively.

Publication boundary

Publish the entire package directory, not individual metadata files. A package is ready only after:

  • validate_salmon_datapackage(path, require_iris=True) passes;
  • no MISSING or REVIEW: values remain;
  • metadata and semantic choices have received human review; and
  • any EDH XML has been rebuilt from the final metadata.

Retain semantic-IRI dereference evidence

To check the exact HTTP identifiers already selected in the package, run:

from metasalmonpy import verify_sdp_semantic_iris

results = verify_sdp_semantic_iris("survey-sdp")

This makes live GET requests in a subprocess with a 30-second deadline per attempt, including startup, DNS, connection and redirects. It reads the final response headers and closes the response without downloading its body; an expired worker is killed and reaped. Injected test requesters keep their own execution behavior. HTTP408, 429, 5xx and classified transient transport failures retry at most three times. Candidate suggestions, non-HTTP identifiers and arbitrary data URLs are excluded. Manifest-bound SSSOM semantic references are checked after their manifest and hashes validate. No model provider is called.

The default HTTPX/HTTPcore backend waits through HTTP103 Early Hints for the complete final headers. Its public manual redirect flow preserves client cookies, origin-bound credentials and target-host netrc credentials without reading redirect bodies. Proxy/NO_PROXY routing follows the existing Requests configuration rules, including CIDR and leading-dot bypass entries; cookies survive a switch between proxy and direct connections. Existing REQUESTS_CA_BUNDLE/CURL_CA_BUNDLE aliases and HTTPX’s SSL_CERT_FILE/SSL_CERT_DIR configure certificate verification. No private Requests or HTTP connection shim is used. The worker deadline also bounds these setup steps.

The complete, deterministically ordered report is written to reproducibility/provenance/semantic-iri-dereference.csv, including final status, URL, redacted error and attempt count. A failed check raises an aggregate ValueError after writing every row, so inspect the report before retrying. You can checksum the CSV as publication evidence. HTTP success, including a fragment’s HTTP200, proves resolution; it does not prove RDF term presence, semantic suitability or approval to publish.

Back to top