Guide 6 — Publish your first dataset#

The tutorial version of the publishing journey: you start with a raw cycler export and finish with validated, linked records, a citable archive, and a place in the Battery Genome registry. Every code cell here executes in CI, against the sample Neware CSV that ships inside the battinfo package — swap in your own files and the recipe is identical.

The five stages:

  1. Convert — instrument files → tidy BDF tables

  2. Identify — name the physical cells you tested

  3. Link — attach each test and its data to its cell

  4. Save & validate — canonical records with stable IRIs

  5. Publish — a DOI on Zenodo + the registry review queue

Prerequisites: Stage 1 uses the processing extra, which brings the batterydf BDF converter and the plotting libraries: pip install "battinfo[processing]". The extra currently installs the batterydf 0.2 release candidate from PyPI. Stage 5 needs credentials and is guarded so the notebook runs fully offline without them.

New words? The plain-language glossary decodes EMMO, IRI, JSON-LD, and the rest in two sentences each.

[1]:

import shutil
from importlib.resources import files
from pathlib import Path

# This notebook runs from its own folder (docs/guides); fresh scratch every run.
SCRATCH = Path("_scratch/guide-06").resolve()
if SCRATCH.exists():
    shutil.rmtree(SCRATCH)
SCRATCH.mkdir(parents=True)

# Your starting point: a folder with raw cycler exports. Here, the sample
# Neware CSV that ships inside the battinfo package (one CC charge + discharge cycle).
sample = files("battinfo.data") / "samples" / "neware-sample.csv"
(SCRATCH / "neware-sample.csv").write_bytes(sample.read_bytes())

import battinfo

ws = battinfo.workspace(root=SCRATCH)

Stage 1 — Convert#

Every instrument speaks its own format; BDF (Battery Data Format) is the one tidy, documented table they all become. NEWARE .ndax, Biologic .mpt, Excel and MATLAB exports are auto-detected by ws.convert(); CSV exports use an explicit pattern.

No reader for your instrument (Arbin, Maccor, an in-house export)? Save a CSV from the instrument software and map its headers yourself: ws.bdf_columns() prints the canonical column names, and ws.convert_csv(path, hints={"Voltage(V)": "voltage_volt", ...}) does the rename. Check the validation report it prints — a measured column listed under extras is unmapped and will be invisible downstream.

[2]:

converted = ws.convert("*.csv")

bdf_file = sorted((SCRATCH / "bdf").glob("*.bdf.csv"))[0]
print()
print("BDF columns:", bdf_file.read_text(encoding="utf-8").splitlines()[0])

  neware-sample.csv  ->  neware-sample.bdf.csv  (0.0 MB, via neware_csv)
  Left out a column that only numbers the rows: Record. The row order of neware-sample.bdf.csv holds the same information.

Converted 1 file(s) -> <repo>\docs\guides\_scratch\guide-06\bdf

BDF columns: test_time_second,voltage_volt,current_ampere,unix_time_second,cycle_count,step_id,step_time_second,step_charging_capacity_ah,step_discharging_capacity_ah

Stage 2 — Identify#

Your measurements are about specific physical cells. Search the registry for the cell product you tested; if you are offline (or the cell is not registered yet), describe it yourself — the same object works either way.

[3]:

hits = ws.search("molicel inr21700 p45b")

if hits:
    spec = hits[0]           # found in the registry — reuse its identity
else:
    # Offline / unregistered: describe the product yourself.
    spec = battinfo.CellSpec(
        manufacturer="Molicel",
        model="INR21700-P45B",
        format="cylindrical",
        chemistry="Li-ion",
        nominal_capacity={"value": 4.5, "unit": "Ah"},
    )

# The serial printed on the cell, or a label only your lab uses: a cell's IRI
# comes from its spec and serial, so a bare "S1" would be every lab's S1.
cells = ws.add("cell", spec=spec, serial_numbers=["MYLAB-P45B-001"])

No cell-spec match for 'molicel inr21700 p45b'.
  Tip: ws.template('cell-spec', manufacturer='...', model='...')
  cell: MYLAB-P45B-001  (IRI auto-assigned)

Stage 4 — Save & validate#

Before saving, set two workspace defaults that get stamped onto every record: who made the data, and how anyone may use it.

ws.contributor(orcid, name=...) tags the workspace with a person by ORCID; contributors accumulate, so call it once per author. Each is written onto every record you save (as a Person carrying the ORCID) and into the publication bundle — so a two-author dataset credits both, not just the last one.

ws.license("cc-by-4.0") sets the workspace’s default license. Set it here, at the start — before ws.save() — so every locally saved record carries a license, not just the Zenodo deposit. A per-record ws.add(..., license=...) overrides the default for that record, and the license= you pass to ws.zenodo(...) in Stage 5 wins for the deposit itself (precedence: explicit ws.zenodo license > explicit ws.add license > workspace default). A record saved without any license still validates, but the gold-standard panel nudges you to set one.

ws.save() then validates every record against the canonical JSON Schemas and writes them with stable w3id.org/battinfo/... IRIs, minted deterministically from each record’s identity — re-running an identical save is a no-op, never a duplicate.

[5]:

# The workspace default license — stamped onto every record from here on.
ws.license("cc-by-4.0")

# Credit both authors by ORCID — contributors accumulate across calls.
ws.contributor("0000-0002-1825-0097", name="Ada Lovelace")
ws.contributor("0000-0001-5109-3700", name="Alan Turing")

ws.save()

records_root = SCRATCH / ".battinfo/records"
for record_file in sorted(records_root.rglob("*.json")):
    print(f"{record_file.parent.name}/{record_file.name}")

Workspace license: cc-by-4.0
  Saved to .battinfo/workspace.json; stamped onto records on ws.save().
Workspace contributor: Ada Lovelace (https://orcid.org/0000-0002-1825-0097)
  Saved to .battinfo/workspace.json; stamped onto records on ws.save().
Workspace contributors (2):
  - Ada Lovelace (https://orcid.org/0000-0002-1825-0097)
  - Alan Turing (https://orcid.org/0000-0001-5109-3700)
  Saved to .battinfo/workspace.json; stamped onto records on ws.save().
Saved 4 record(s) under <repo>\docs\guides\_scratch\guide-06\.battinfo\records:
  cell spec   Molicel INR21700-P45B                [created]  .battinfo\records\cell-spec\cell-spec-vgwn-7exg-09tt-zwq3.json
  cell        MYLAB-P45B-001                       [created]  .battinfo\records\cell-instance\cell-fhcx-9qtr-3vep-wahx.json
  test        MYLAB-P45B-001 cycling               [created]  .battinfo\records\test\test-04bx-fcf2-hxm2-rtva.json
  dataset     MYLAB-P45B-001 data                  [created]  .battinfo\records\dataset\dataset-s34m-fyym-c479-hfge.json

  Next: ws.list(verbose=True) to inspect, or ws.publish() to publish.
cell-instance/cell-fhcx-9qtr-3vep-wahx.json
cell-spec/cell-spec-vgwn-7exg-09tt-zwq3.json
dataset/dataset-s34m-fyym-c479-hfge.json
test/test-04bx-fcf2-hxm2-rtva.json

[6]:

# Look inside one: the cell record, linked to its spec, stamped with provenance.
import json
cell_record_file = next((records_root / "cell-instance").glob("*.json"))
cell_record = json.loads(cell_record_file.read_text(encoding="utf-8"))
print(json.dumps(cell_record["cell_instance"], indent=2)[:400])
print("...")
print("provenance:", json.dumps(cell_record["provenance"], indent=2))

{
  "id": "https://w3id.org/battinfo/cell/fhcx-9qtr-3vep-wahx",
  "short_id": "fhcx9q",
  "cell_spec_id": "https://w3id.org/battinfo/spec/vgwn-7exg-09tt-zwq3",
  "name": "MYLAB-P45B-001",
  "serial_number": "MYLAB-P45B-001"
}
...
provenance: {
  "source_type": "measurement",
  "retrieved_at": 1791981082,
  "battinfo_version": "0.8.0"
}

Stage 5 — Publish#

The payoff, in two halves. The DOI needs no registry account, so it is the primary path here, and it comes first: the registry takes only records whose data files are already online.

  • A citable DOI (Zenodo) — primary. Needs only a Zenodo token, no registry account. ws.zenodo() archives the records and data files; pass creators=[...] to credit the authors on the deposit itself; without it, the contributors you tagged in Stage 4 are its creators. Either way their ORCIDs travel into the archived metadata. The deposit inherits the license you set with ws.license(...) in Stage 4; pass license= here only to choose a different one for the archive (an explicit ws.zenodo(license=...) wins over the workspace default). It leaves a reviewable draft by default — click Publish on Zenodo (or pass publish=True) to mint the DOI. Use a Zenodo sandbox token for a dry run (sandbox=True); drop it for the real thing. A sandbox deposit leaves the records pointing at the files on this computer, and the workspace remembers the sandbox record in .battinfo/zenodo-sandbox.json, apart from the record on zenodo.org in .battinfo/zenodo.json, so nothing needs deleting before the real deposit.

  • The Battery Genome registry (``ws.publish()``) — optional. Submits your records to the review queue, where a curator promotes them to the public index. This needs a registry API key (ws.login(api_key="bk_...")). During the soft launch keys are granted by the operators — start from battinfo.org/publish and say who you are and what you plan to publish. The registry refuses records that still point at files on this computer, since it would publish those paths as the download links. Deposit on Zenodo first, which points each record at its deposited copy, or call ws.publish(zenodo=True), which does both in one go.

Both cells are guarded no-ops until you provide the credential, so the notebook runs fully offline without them. They ask the workspace with ws.has_credential(...), which also finds a token kept in the workspace’s .battinfo/credentials file, not only one in the environment.

[7]:

# DOI path (primary) — Zenodo only, no registry key needed.
# The deposit inherits the workspace license from Stage 4; creators credit the
# authors on the deposit. A click on Zenodo (or publish=True) mints the DOI.
# Here we leave a reviewable draft.
if ws.has_credential("ZENODO_SANDBOX_TOKEN"):
    deposit = ws.zenodo(
        sandbox=True,
        creators=[
            {"name": "Lovelace, Ada", "orcid": "0000-0002-1825-0097"},
            {"name": "Turing, Alan", "orcid": "0000-0001-5109-3700"},
        ],
    )
    print("Zenodo draft:", deposit.draft_url)
else:
    print("No ZENODO_SANDBOX_TOKEN set - skipping the DOI dry run.")
    print("Get a token at https://sandbox.zenodo.org, then re-run this cell.")

No ZENODO_SANDBOX_TOKEN set - skipping the DOI dry run.
Get a token at https://sandbox.zenodo.org, then re-run this cell.

[8]:

# Registry path — needs the API key granted by the operators (link above).
if ws.has_credential("BATTINFO_API_KEY"):
    outcomes = ws.publish(note="My first BattINFO dataset")
    ws.status()
else:
    print("Records are saved and validated locally.")
    print("To submit to the registry: ws.login(api_key='bk_...') then re-run.")

Records are saved and validated locally.
To submit to the registry: ws.login(api_key='bk_...') then re-run.

What you have now#

_scratch/guide-06/
├── neware-sample.csv           # your raw export (untouched)
├── bdf/neware-sample.bdf.csv   # the tidy, shareable table
└── .battinfo/records/ # validated, linked records with stable IRIs
    ├── cell-spec/  ├── cell-instance/  ├── test/  └── dataset/

Re-run this notebook from the top: same IRIs, no duplicates — that determinism is what makes the workflow safe to automate.

Next steps