Network Tools API¶

The calvin.network package provides Python tools for working with the calvin-network-data repository. It replaces the legacy Node.js calvin-network-tools (cnf CLI) entirely in Python.

Top-level imports:

from calvin.network import load_network, build_matrix, export_matrix

loader¶

Crawls the data repository, reads node.geojson / link.json files, resolves $ref pointers to CSV data, and returns a Network object.

Network data loader.

Crawls the calvin-network-data repository, reads node.geojson and link.json files, resolves $ref pointers to CSV timeseries, and returns a structured network representation.

class calvin.network.loader.Bound(type: str, bound: Any = None)¶

Bases: object

A single bound definition on a link or node.

bound: Any = None¶

property is_constant: bool¶

property is_equality: bool¶

property is_monthly: bool¶

property is_timeseries: bool¶

type: str¶

class calvin.network.loader.CostDef(type: str, cost: float | None = None, costs: dict[str, Any] | None = None)¶

Bases: object

Cost definition for a link or sink.

cost: float | None = None¶

costs: dict[str, Any] | None = None¶

type: str¶

class calvin.network.loader.Network(nodes: dict[str, ~calvin.network.loader.NetworkNode]=<factory>, links: list[NetworkLink] = <factory>, regions: dict[str, list[str]]=<factory>)¶

Bases: object

The full water network.

links: list[NetworkLink]¶

nodes: dict[str, NetworkNode]¶

regions: dict[str, list[str]]¶

class calvin.network.loader.NetworkLink(prmname: str, description: str = '', link_type: str = '', origin: str = '', terminus: str = '', amplitude: float = 1.0, disabled: bool = False, filepath: str = '', flow: list[list] | None = None, bounds: list[Bound] = <factory>, costs: CostDef | None = None, properties: dict = <factory>)¶

Bases: object

A directed link (edge) in the water network.

amplitude: float = 1.0¶

bounds: list[Bound]¶

costs: CostDef | None = None¶

description: str = ''¶

disabled: bool = False¶

filepath: str = ''¶

flow: list[list] | None = None¶

link_type: str = ''¶

origin: str = ''¶

prmname: str¶

properties: dict¶

terminus: str = ''¶

class calvin.network.loader.NetworkNode(prmname: str, description: str = '', node_type: str = '', disabled: bool = False, filepath: str = '', initialstorage: float | None = None, endingstorage: float | None = None, areacapfactor: float = 0.0, inflows: dict[str, list[list]] | None=None, sinks: list[dict] | None = None, flow: list[list] | None = None, storage: list[list] | None = None, evaporation: list[list] | None = None, bounds: list[Bound] = <factory>, costs: CostDef | None = None, properties: dict = <factory>)¶

Bases: object

A node in the water network.

areacapfactor: float = 0.0¶

bounds: list[Bound]¶

costs: CostDef | None = None¶

description: str = ''¶

disabled: bool = False¶

endingstorage: float | None = None¶

evaporation: list[list] | None = None¶

filepath: str = ''¶

flow: list[list] | None = None¶

inflows: dict[str, list[list]] | None = None¶

initialstorage: float | None = None¶

node_type: str = ''¶

prmname: str¶

properties: dict¶

sinks: list[dict] | None = None¶

storage: list[list] | None = None¶

calvin.network.loader.load_network(data_path: str | Path, nodes: list[str] | None = None, regions: list[str] | None = None) → Network¶

Load the calvin-network-data repository into a Network object.

Parameters¶

data_pathstr or Path: Path to the data/ directory inside calvin-network-data.
nodeslist of str, optional: If provided, only include these nodes (by prmname) and their connecting links. Pass None for the full network.
regionslist of str, optional: If provided, include all nodes within the named regions.

Returns¶

Network: Populated network with nodes, links, and region mappings.

matrix¶

Builds the time-expanded i,j,k,cost,amplitude,lower_bound,upper_bound matrix from a loaded Network object.

Matrix builder.

Transforms a Network object into the time-expanded i,j,k,cost,amplitude, lower_bound,upper_bound matrix consumed by the CALVIN Pyomo solver.

This is the Python replacement for the entire nodejs/matrix/ directory in calvin-network-tools.

class calvin.network.matrix.CostSegment(cost: float = 0.0, lb: float = 0.0, ub: float | None = None)¶

Bases: object

A single piecewise cost segment.

cost: float = 0.0¶

lb: float = 0.0¶

ub: float | None = None¶

class calvin.network.matrix.StepBounds(lb: float = 0.0, ub: float | None = None, lb_defined: bool = False)¶

Bases: object

Resolved bounds for a single timestep.

lb: float = 0.0¶

lb_defined: bool = False¶

ub: float | None = None¶

calvin.network.matrix.build_annual_matrix(network: Network, start: str = '1921-10', stop: str = '2003-09', max_ub: float = 1000000.0, sep: str = '.', add_debug: bool = False, debug_cost: float = 20000000.0, constrain_ending: str = 'none', node_lb_overrides: dict | None = None) → DataFrame¶

Build the annual time-expanded network matrix from a Network object.

Each timestep corresponds to one water year, identified by its September 30 end-date. Flows are aggregated totals (TAF/year); storage is tracked at end-of-September with compound evaporation across the 12 monthly steps.

Parameters¶

networkNetwork: Loaded network from load_network().
startstr: Start date as YYYY-MM (default: 1921-10 → first WY ending 1922-09-30).
stopstr: Stop date as YYYY-MM (default: 2003-09 → last WY ending 2003-09-30).
max_ubfloat: Replace unbounded upper bounds with this value.
sepstr: Separator between node name and date (default: “.”).
add_debugbool: If True, add debug source/sink links for infeasibility diagnosis.
debug_costfloat: Cost assigned to debug links.
constrain_endingstr: Controls ending-storage constraints on node→FINAL links. Same semantics as build_matrix().
node_lb_overridesdict, optional: Same semantics as build_matrix().

Returns¶

pd.DataFrame: Matrix with columns: i, j, k, cost, amplitude, lower_bound, upper_bound.

calvin.network.matrix.build_matrix(network: Network, start: str = '1921-10', stop: str = '2003-09', max_ub: float = 1000000.0, sep: str = '.', add_debug: bool = False, debug_cost: float = 20000000.0, constrain_ending: str = 'none', node_lb_overrides: dict | None = None) → DataFrame¶

Build the time-expanded network matrix from a Network object.

Parameters¶

networkNetwork

Loaded network from load_network().

startstr

Start date as YYYY-MM (default: 1921-10).

stopstr

Stop date as YYYY-MM (default: 2003-09).

max_ubfloat

Replace unbounded upper bounds with this value.

sepstr

Separator between node name and date (default: “.”).

add_debugbool

If True, add debug source/sink links for infeasibility diagnosis.

debug_costfloat

Cost assigned to debug links.

constrain_endingstr

Controls ending-storage constraints on node→FINAL links:

"none" (default) — GW nodes unconstrained; SR nodes fixed to their endingstorage value if defined.
"gw" — GW nodes: lb=initial storage, ub=unconstrained (no net overdraft, but aquifer recovery above initial is allowed). SR nodes unchanged.
"all" — All storage nodes (GW + SR) fixed to ending/initial storage (lb=ub). Used by COSVF one-year runs.

node_lb_overridesdict, optional

Mapping of node name → physical lower bound. For each entry, the storage lower bound is clamped to this value before piecewise cost reconciliation, overriding any higher LBT-derived bound. Used to allow drought drawdown to the physical dead-pool level for nodes whose LBT data reflects an operational target (e.g. SR_SHA, SR_CLE).

Returns¶

pd.DataFrame: Matrix with columns: i, j, k, cost, amplitude, lower_bound, upper_bound.

calvin.network.matrix.export_matrix(df: DataFrame, output: str | Path | None = None, fmt: str = 'csv') → str | None¶

Export a matrix DataFrame to file or string.

Parameters¶

dfpd.DataFrame: Matrix from build_matrix().
outputstr, Path, or None: Output file path. If None, returns the formatted string.
fmtstr: Output format: “csv” or “tsv”.

Returns¶

str or None: Formatted matrix string if output is None.

query¶

List and find nodes, links, and regions in the data repository.

Network listing and query utilities.

Provides functions to list nodes, links, and query the network data repository — replacing the cnf list command.

calvin.network.query.find_link(data_path: str | Path, prmname: str) → dict | None¶

Find and return the full data for a single link by prmname.

Returns the raw link.json dict, or None if not found.

calvin.network.query.find_node(data_path: str | Path, prmname: str) → dict | None¶

Find and return the full data for a single node by prmname.

Returns the raw GeoJSON properties dict, or None if not found.

calvin.network.query.list_items(data_path: str | Path, item_type: Literal['all', 'nodes', 'links'] = 'all', filter_names: list[str] | None = None) → DataFrame¶

List all nodes and/or links in the data repository.

Parameters¶

data_pathstr or Path: Path to the data/ directory.
item_typestr: What to list: “all”, “nodes”, or “links”.
filter_nameslist of str, optional: Only include items whose prmname matches one of these.

Returns¶

pd.DataFrame: Table with columns: prmname, type, kind (node/link), path, description.

calvin.network.query.list_regions(data_path: str | Path) → list[dict]¶

List all regions in the data repository.

Returns a list of dicts with keys: name, path.

validate¶

Validates data repository structure, schemas, and referential integrity.

Network data validator.

Validates node.geojson and link.json files against expected schemas and checks for structural issues in the network.

class calvin.network.validate.ValidationError(path: str, message: str, severity: str = 'error')¶

Bases: object

A single validation error.

message: str¶

path: str¶

severity: str = 'error'¶

class calvin.network.validate.ValidationResult(errors: list[ValidationError] = <factory>, node_count: int = 0, link_count: int = 0, region_count: int = 0)¶

Bases: object

Result of validating a network data repository.

errors: list[ValidationError]¶

property is_valid: bool¶

link_count: int = 0¶

node_count: int = 0¶

region_count: int = 0¶

summary() → str¶

calvin.network.validate.validate_data(data_path: str | Path) → ValidationResult¶

Validate a calvin-network-data repository.

Checks: - All node.geojson files have required fields (prmname, type) - All link.json files have required fields (prmname, origin, terminus) - Node types are recognized - Link types are recognized - Bound types are valid - Cost types are valid - $ref targets exist - Links reference existing nodes - No orphaned nodes (warning) - Storage nodes have initialstorage/endingstorage

Parameters¶

data_pathstr or Path: Path to the data/ directory.

Returns¶

ValidationResult

apply_changes¶

Import data updates from Excel workbooks or CSV files into the data repository.

Apply external changes to the network data repository.

Supports importing data from: - Excel workbooks (exported from the web app) - CSV files with multi-column timeseries

Replaces the cnf apply-changes app and cnf apply-changes csv commands.

calvin.network.apply_changes.apply_csv(csv_path: str | Path, data_path: str | Path, prop: Literal['flow', 'storage', 'inflows'] = 'flow', dry_run: bool = False) → list[str]¶

Import timeseries data from a CSV file into the data repository.

The CSV has columns date, PRMNAME1, PRMNAME2, .... Each column’s data is written to the corresponding node’s CSV file.

Parameters¶

csv_pathstr or Path: Path to the input CSV.
data_pathstr or Path: Path to the data/ directory.
propstr: Which property to update: “flow”, “storage”, or “inflows”.
dry_runbool: If True, only report what would change.

Returns¶

list of str: Paths of files written.

calvin.network.apply_changes.apply_excel(excel_path: str | Path, data_path: str | Path, dry_run: bool = False) → list[str]¶

Import data from an Excel workbook into the data repository.

Each sheet in the workbook represents one node/link. The first row contains metadata: prmname/path/to/file. Subsequent rows contain the timeseries data to write.

Parameters¶

excel_pathstr or Path: Path to the .xlsx file.
data_pathstr or Path: Path to the data/ directory.
dry_runbool: If True, only report what would change without writing files.

Returns¶

list of str: Paths of files that were written (or would be written).

prepare¶

Prepare input files for the COSVF limited-foresight model.

COSVF data preparation.

Reads the calvin-network-data repository and produces the five input files required by the COSVF limited-foresight CALVIN model:

links.csv — single water-year network matrix
r-dict.json — reservoir dictionary with penalty properties
inflows.csv — external inflows for the full period of analysis
variable-constraints.csv — time-varying link bounds
cosvf-params.csv — default penalty parameters (initial EA values)

calvin.network.prepare.prepare_cosvf(data_path: str | Path, output_dir: str | Path, *, start: str = '1921-10', stop: str = '2003-09', wy1_start: str = '1921-10', wy1_stop: str = '1922-09', r_type1: list[str] | None = None, r_type2: list[str] | None = None, network: Network | None = None, node_lb_overrides: dict[str, float] | None = None, storage_persuasion_cost: float = -0.02, connector_cost: float = 0.01) → Path¶

Prepare all input files for a COSVF limited-foresight CALVIN run.

Reads directly from calvin-network-data via the network module and writes the five required files into output_dir.

Parameters¶

data_pathstr or Path: Path to the data/ directory in calvin-network-data.
output_dirstr or Path: Destination directory for the generated files. Created if it does not already exist.
startstr: Start of the full period of analysis, YYYY-MM (default 1921-10).
stopstr: End of the full period of analysis, YYYY-MM (default 2003-09).
wy1_startstr: Start of the first (template) water year, YYYY-MM (default 1921-10).
wy1_stopstr: End of the first water year, YYYY-MM (default 1922-09).
r_type1list of str, optional: Surface reservoirs with quadratic COSVF penalties. Defaults to DEFAULT_R_TYPE1.
r_type2list of str, optional: Groundwater basins with linear COSVF penalties. Defaults to DEFAULT_R_TYPE2.
networkNetwork, optional: Pre-loaded network object. If None, the network will be loaded from data_path.

Returns¶

Path: The resolved output_dir path.

calvin.network.prepare.prepare_cosvf_astep(data_path: str | Path, output_dir: str | Path, *, start: str = '1921-10', stop: str = '2003-09', wy1_start: str = '1921-10', wy1_stop: str = '1922-09', r_type1: list[str] | None = None, r_type2: list[str] | None = None, network: Network | None = None, node_lb_overrides: dict[str, float] | None = None, storage_persuasion_cost: float = -0.02, connector_cost: float = 0.01) → Path¶

Prepare all input files for an annual-step COSVF limited-foresight run.

Produces the same five-file schema as prepare_cosvf() but with annual timesteps (one water year per step, Sep 30 end-dates).

Parameters¶

data_pathstr or Path: Path to the data/ directory in calvin-network-data.
output_dirstr or Path: Destination directory. Created if it does not already exist.
startstr: Start of the full period of analysis, YYYY-MM (default 1921-10).
stopstr: End of the full period of analysis, YYYY-MM (default 2003-09).
wy1_startstr: Start of the first (template) water year, YYYY-MM (default 1921-10).
wy1_stopstr: End of the first water year, YYYY-MM (default 1922-09).
r_type1list of str, optional: Surface reservoirs with quadratic COSVF penalties.
r_type2list of str, optional: Groundwater basins with linear COSVF penalties.
networkNetwork, optional: Pre-loaded network object.

Returns¶

Path: The resolved output_dir path.

calvin.network.prepare.prepare_pf_astep(data_path: str | Path, output_dir: str | Path, *, start: str = '1921-10', stop: str = '2003-09', network: Network | None = None, node_lb_overrides: dict[str, float] | None = None, connector_cost: float = 0.01) → Path¶

Prepare annual-step perfect foresight links file.

Builds a links.csv with 82 annual timesteps (Sep 30 end-dates) covering the full CALVIN period of analysis.

Parameters¶

data_pathstr or Path: Path to the data/ directory in calvin-network-data.
output_dirstr or Path: Destination directory for the generated links.csv. Created if it does not already exist.
startstr: Start of the analysis period, YYYY-MM (default 1921-10).
stopstr: End of the analysis period, YYYY-MM (default 2003-09).
networkNetwork, optional: Pre-loaded network object.

Returns¶

Path: The resolved output_dir path.

cli¶

Command-line interface for all network tools.

CLI entry point for calvin network tools.

Replaces the cnf Node.js CLI. Usage:

python -m calvin.network.cli matrix --data /path/to/data ...
python -m calvin.network.cli list --data /path/to/data
python -m calvin.network.cli validate --data /path/to/data
python -m calvin.network.cli apply-changes csv --file changes.csv --data /path/to/data

Or if installed as a console script (see pyproject.toml):

calvin-network matrix --data /path/to/data ...

calvin.network.cli.main(argv: list[str] | None = None)¶