Guide 6 — Publish your first dataset#

The tutorial version of the publishing journey: you start with a raw cycler export and finish with validated, linked records, a citable archive, and a place in the Battery Genome registry. Every code cell here executes in CI, against the sample Neware CSV that ships with the repository — swap in your own files and the recipe is identical.

The five stages:

  1. Convert — instrument files → tidy BDF tables

  2. Identify — name the physical cells you tested

  3. Link — attach each test and its data to its cell

  4. Save & validate — canonical records with stable IRIs

  5. Publish — a DOI on Zenodo + the registry review queue

Prerequisites: Stage 1 uses the processing extra — the batterydf BDF converter. BattINFO is not on PyPI until the 0.8 release, so install from source (see Installation); batterydf is not on PyPI yet either, so pip install "battinfo[processing]" cannot resolve it. Clone the repo and run uv sync --all-extras (how this notebook is executed in CI), or pip install -e . plus the processing dependencies directly: pip install "git+https://github.com/battery-data-alliance/battery-data-format.git" matplotlib plotly. Stage 5 needs credentials and is guarded so the notebook runs fully offline without them.

New words? The plain-language glossary decodes EMMO, IRI, JSON-LD, and the rest in two sentences each.

[1]:

import os
import shutil
from pathlib import Path

# This notebook runs from its own folder (docs/guides); fresh scratch every run.
SCRATCH = Path("_scratch/guide-06").resolve()
if SCRATCH.exists():
    shutil.rmtree(SCRATCH)
SCRATCH.mkdir(parents=True)

# Your starting point: a folder with raw cycler exports. Here, the sample
# Neware CSV that ships with the repo (one CC charge + discharge cycle).
shutil.copy(Path("data/neware-sample.csv"), SCRATCH / "neware-sample.csv")

import battinfo

ws = battinfo.workspace(root=SCRATCH)

Stage 1 — Convert#

Every instrument speaks its own format; BDF (Battery Data Format) is the one tidy, documented table they all become. NEWARE .ndax, Biologic .mpt, Excel and MATLAB exports are auto-detected by ws.convert(); CSV exports use an explicit pattern.

No reader for your instrument (Arbin, Maccor, an in-house export)? Save a CSV from the instrument software and map its headers yourself: ws.bdf_columns() prints the canonical column names, and ws.convert_csv(path, hints={"Voltage(V)": "voltage_volt", ...}) does the rename. Check the validation report it prints — a measured column listed under extras is unmapped and will be invisible downstream.

[2]:

converted = ws.convert("*.csv")

bdf_file = sorted((SCRATCH / "bdf").glob("*.bdf.csv"))[0]
print()
print("BDF columns:", bdf_file.read_text(encoding="utf-8").splitlines()[0])

  neware-sample.csv  ->  neware-sample.bdf.csv  (0.0 MB, via neware_csv)
  WARNING: 1 source column(s) were NOT mapped to BDF and are missing from neware-sample.bdf.csv:
    - Record
    (reader plugin: neware_csv -- if that looks wrong, the file was likely detected as the wrong instrument format)
    Fix: ws.bdf_columns() lists the canonical names, then ws.convert_csv('neware-sample.csv', hints={'<source column>': '<bdf name>'}).

Converted 1 file(s) -> <repo>\docs\guides\_scratch\guide-06\bdf
  DATA-LOSS WARNING: 1 file(s) had unmapped source columns (details above) -- the converted files do NOT contain those columns.

BDF columns: test_time_second,voltage_volt,current_ampere,unix_time_second,cycle_count,step_id,step_time_second,step_charging_capacity_ah,step_discharging_capacity_ah

Stage 2 — Identify#

Your measurements are about specific physical cells. Search the registry for the cell product you tested; if you are offline (or the cell is not registered yet), describe it yourself — the same object works either way.

[3]:

hits = ws.search("molicel inr21700 p45b")

if hits:
    spec = hits[0]           # found in the registry — reuse its identity
else:
    # Offline / unregistered: describe the product yourself.
    spec = battinfo.CellSpec(
        manufacturer="Molicel",
        model="INR21700-P45B",
        format="cylindrical",
        chemistry="Li-ion",
        nominal_capacity={"value": 4.5, "unit": "Ah"},
    )

cells = ws.add("cell", spec=spec, serial_numbers=["S1"])

No cell-spec match for 'molicel inr21700 p45b'.
  Tip: ws.template('cell-spec', manufacturer='...', model='...')
  cell: S1  (IRI auto-assigned)

Stage 4 — Save & validate#

Before saving, credit the authors. ws.contributor(orcid, name=...) tags the workspace with a person by ORCID; contributors accumulate, so call it once per author. Each is written onto every record you save (as a Person carrying the ORCID) and into the publication bundle — so a two-author dataset credits both, not just the last one.

ws.save() then validates every record against the canonical JSON Schemas and writes them with stable w3id.org/battinfo/... IRIs, minted deterministically from each record’s identity — re-running an identical save is a no-op, never a duplicate.

[5]:

# Credit both authors by ORCID — contributors accumulate across calls.
ws.contributor("0000-0002-1825-0097", name="Ada Lovelace")
ws.contributor("0000-0001-5109-3700", name="Alan Turing")

ws.save()

records_root = SCRATCH / ".battinfo/records"
for record_file in sorted(records_root.rglob("*.json")):
    print(f"{record_file.parent.name}/{record_file.name}")

Workspace contributor: Ada Lovelace (https://orcid.org/0000-0002-1825-0097)
  Saved to .battinfo/workspace.json; stamped onto records on ws.save().
Workspace contributors (2):
  - Ada Lovelace (https://orcid.org/0000-0002-1825-0097)
  - Alan Turing (https://orcid.org/0000-0001-5109-3700)
  Saved to .battinfo/workspace.json; stamped onto records on ws.save().
Saved 4 record(s) under <repo>\docs\guides\_scratch\guide-06\.battinfo\records:
  cell spec   Molicel INR21700-P45B                [created]  .battinfo\records\cell-spec\cell-spec-fpdr-z1qt-23v0-stzx.json
  cell        S1                                   [created]  .battinfo\records\cell-instance\cell-d6tp-jgqy-a3s9-4qem.json
  test        S1 cycling                           [created]  .battinfo\records\test\test-9f4c-mxny-yhym-2zh4.json
  dataset     S1 data                              [created]  .battinfo\records\dataset\dataset-4pkb-x4v6-mqb9-tq2w.json

  Next: ws.list(verbose=True) to inspect, or ws.publish() to publish.
cell-instance/cell-d6tp-jgqy-a3s9-4qem.json
cell-spec/cell-spec-fpdr-z1qt-23v0-stzx.json
dataset/dataset-4pkb-x4v6-mqb9-tq2w.json
test/test-9f4c-mxny-yhym-2zh4.json

[6]:

# Look inside one: the cell record, linked to its spec, stamped with provenance.
import json
cell_record_file = next((records_root / "cell-instance").glob("*.json"))
cell_record = json.loads(cell_record_file.read_text(encoding="utf-8"))
print(json.dumps(cell_record["cell_instance"], indent=2)[:400])
print("...")
print("provenance:", json.dumps(cell_record["provenance"], indent=2))

{
  "id": "https://w3id.org/battinfo/cell/d6tp-jgqy-a3s9-4qem",
  "short_id": "d6tpjg",
  "cell_spec_id": "https://w3id.org/battinfo/spec/fpdr-z1qt-23v0-stzx",
  "name": "S1",
  "serial_number": "S1"
}
...
provenance: {
  "source_type": "measurement",
  "retrieved_at": 1785428066,
  "battinfo_version": "0.7.0"
}

Stage 5 — Publish#

The payoff. The two halves are independent, and the DOI needs no registry account — so it is the primary path here:

  • A citable DOI (Zenodo) — primary. Needs only a Zenodo token, no registry account. ws.zenodo() archives the records and data files; pass creators=[...] to credit the authors on the deposit itself, and the ORCIDs you tagged in Stage 4 travel into the archived metadata too. It leaves a reviewable draft by default — click Publish on Zenodo (or pass publish=True) to mint the DOI. Use a Zenodo sandbox token for a dry run (sandbox=True); drop it for the real thing.

  • The Battery Genome registry (``ws.publish()``) — optional. Submits your records to the review queue, where a curator promotes them to the public index. This needs a registry API key (ws.login(api_key="bk_...")). During the soft launch keys are granted by the operators — start from battinfo.org/publish and say who you are and what you plan to publish.

Both cells are guarded no-ops until you provide the credential, so the notebook runs fully offline without them.

[7]:

# DOI path (primary) — Zenodo only, no registry key needed.
# creators credit the authors on the deposit; a click on Zenodo (or publish=True)
# mints the DOI. Here we leave a reviewable draft.
if os.environ.get("ZENODO_SANDBOX_TOKEN"):
    deposit = ws.zenodo(
        sandbox=True,
        license="cc-by-4.0",
        creators=[
            {"name": "Lovelace, Ada", "orcid": "0000-0002-1825-0097"},
            {"name": "Turing, Alan", "orcid": "0000-0001-5109-3700"},
        ],
    )
    print("Zenodo draft:", deposit.draft_url)
else:
    print("No ZENODO_SANDBOX_TOKEN set - skipping the DOI dry run.")
    print("Get a token at https://sandbox.zenodo.org, then re-run this cell.")

No ZENODO_SANDBOX_TOKEN set - skipping the DOI dry run.
Get a token at https://sandbox.zenodo.org, then re-run this cell.

[8]:

# Registry path — needs the API key granted by the operators (link above).
if os.environ.get("BATTINFO_API_KEY"):
    outcomes = ws.publish(note="My first BattINFO dataset")
    ws.status()
else:
    print("Records are saved and validated locally.")
    print("To submit to the registry: ws.login(api_key='bk_...') then re-run.")

Records are saved and validated locally.
To submit to the registry: ws.login(api_key='bk_...') then re-run.

What you have now#

_scratch/guide-06/
├── neware-sample.csv           # your raw export (untouched)
├── bdf/neware-sample.bdf.csv   # the tidy, shareable table
└── .battinfo/records/ # validated, linked records with stable IRIs
    ├── cell-spec/  ├── cell-instance/  ├── test/  └── dataset/

Re-run this notebook from the top: same IRIs, no duplicates — that determinism is what makes the workflow safe to automate.

Next steps