COSVF Evolutionary Algorithm API¶

class calvin.cosvfea.COSVF(pwd, log_name='calvin-cosvf', console_level=20)¶

Bases: CALVIN

Instantiate COSVF model as a child class of calvin.CALVIN for annual COSVF optimization.

Parameters:

pwd –
(string) path to directory containing COSVF input files. Generate these files with calvin.network.prepare.prepare_cosvf() or the CLI equivalent:
```
python -m calvin.network.cli prepare-cosvf \
    --data /path/to/calvin-network-data/data \
    --output ./my-models/calvin-cosvf
```
Required files in the directory:
1. links.csv — single water-year network matrix (i,j,k,cost,amplitude,lower_bound,upper_bound)
2. cosvf-params.csv — penalty parameters (r,param,value)
3. r-dict.json — reservoir dictionary with penalty properties
4. inflows.csv — external inflows for the full period (date,j,flow_taf)
5. variable-constraints.csv — time-varying link bounds (date,i,j,k,lower_bound,upper_bound)
See calvin.network.prepare.prepare_cosvf() for details on each file.
log_name – (string) name for the global logger. Log file is written to the specified pwd path.
console_level – (int) logging level for the console handler (default logging.INFO). Pass logging.WARNING in EA worker processes to suppress per-solve console output while still writing full DEBUG detail to the log file.

Returns:

COSVF CALVIN model object

add_ag_region_sinks()¶

Hack to get rid of surplus water at no cost from agricultural regions. Called internally when model is initialized.

Returns:: nothing, but modifies the model object

apply_ic(ic)¶

Set initial storage conditions.

Parameters:: ic – (dict) initial storage values
Returns:: nothing, but modifies the model object

assign_cosvf_penalties()¶

Assign the COSVF values to links on the model.

Returns:: nothing, but modifies COSVF CALVIN model object

compute_gw_overdraft(model_df)¶

Calculate overdraft of all groundwater reservoirs that have costs

Parameters:: model_df – (Pandas dataframe) dataframe of cost, upper bound, and flows from the solved annual Pyomo CALVIN instance
Returns:: (float) total groundwater overdraft of all groundwater reservoirs

compute_network_costs(model_df)¶

Calculate costs of LF model run for evolutionary alogrithm.

Parameters:: model_df – (Pandas dataframe) dataframe of cost, upper bound, and flows from the solved CALVIN instance
Returns short_costs:: (float) total costs for shorted links
Returns op_costs:: (float) total costs over operational links

cosvf_construct_piecewise_penalties(r)¶

Create piecewise costs for penalties on end-of-year storage for rtype1 (quadratic) and rtype2 (linear) COSVF penalties.

Parameters:: r – (str) reservoir id (e.g. “SR_DNP”)
Returns r_b:: (list) storage breakpoints
Returns r_k:: (list) and corresponding slopes (marginal values)

cosvf_fit_from_params(pmin, pmax, eop_min, eop_max, k_count)¶

Determine piecewise costs for COSVF

Parameters:

pmin – (float) penalty representing willingness to pay for an additional unit of storage that would encroach the rain-flood conservation pool
pmax – (float) penalty representing willingness to pay for an additional unit of storage below the minimum operating bound
eop_min – (float) end-of-year storage minimum bound
eop_max – (float) end-of-year storage carryover capacity
k_count – (int) number of piecewise links

Returns x:

(numpy.ndarray) array of storage values

Returns y:

(numpy.ndarray) array of penalty values as function of storage values

cosvf_marginal_piecewise(x, y)¶

Calculate slope (cost) and breakpoints (k) for the fitted piecewise quadratic COSVF

Parameters:

x – (numpy.ndarray) array of storage values
y – (numpy.ndarray) array of penalty values for x array of storage values

Returns r_b:

(list) storage breakpoints

Returns r_k:

(list) and corresponding slopes (marginal values)

cosvf_solve(solver='highs', nproc=1, resultdir=None, pcosvf=None, show_progress=False)¶

Solve COSVF CALVIN model for full period of analysis

Parameters:

solver – (string) solver name. glpk, cplex, cbc, gurobi, highs.
nproc – (int) number of processors assigned to model solver instance
resultdir – (path) directory to write out results. If None (default), the assumption is that the user is running in evolutionary mode
pcosvf – (list) If None (default) the COSVF parameters loaded when constructing the COSVF CALVIN instance (cosvf-params.csv) will be used. Otherwise, and specifically for evolutionary mode, the argument is the list of $P_{min}$ and $P_{max}$ for quadratic carryover penalty curves on surface water reservoirs and $P_{GW}$ for linear penalty on groundwater reservoirs, where the order of the penalty parameters for each reservoir must match the order of reservoirs in the r_dict.json.
show_progress – (bool) display a tqdm progress bar in the console (default False). Requires tqdm to be installed; silently disabled if it is not.

Returns:

tuple of (f1, f2, f3) fitness values

Performance notes

Gurobi / CPLEX / HiGHS: automatically uses Pyomo’s APPSI persistent interface, which keeps the LP in the solver’s memory between years and pushes only changed bounds as deltas. Dual simplex is selected so that the previously feasible basis is re-optimised rather than restarted from scratch.
CBC: uses the file-based interface with warm-start basis handoff. After the first year the previous optimal basis is fed back (basisIn), LP presolve is disabled (presolve off), and dual simplex is selected (dualSimplex). Together these typically halve solve time on years 2–82 versus cold-starting with primal simplex.
GLPK: unchanged (GLPK does not support basis warm-start via Pyomo).

cosvf_update_inflows(wy)¶

Update link inflows to reflect the current water year under analysis.

Parameters:: wy – (int) current water year under evaluation.
Returns:: nothing, but modifies CALVIN model object

cosvf_update_initial_storage(eop)¶

Update initial storages in COSVF annual mode

Parameters:: eop – (dict) dictionary of reservoir nodes with the end of year storage from the previous water year’s solution
Returns:: nothing, but modifies CALVIN model object

cosvf_update_variable_bounds(wy)¶

Update link lower/upper bounds to reflect the current water year under analysis.

Parameters:: wy – (int) current water year under evaluation.
Returns:: nothing, but modifies CALVIN model object

create_cosvf_links()¶

Create k-links for the storage nodes that define the carryover penalties.

Returns:: nothing, but modifies links dataframe

create_pyomo_model(**kwargs)¶

Create the pyomo model for COSVF mode.

The COSVF instance of CALVIN uses CALVIN’s create_pyomo_model but with cosvf_mode parameter always on. The only difference is whether debug links will be used or not. When debug_mode is used with COSVF, the debug links are assigned the default (or user specified) `debug_cost of 2e7 $/af; however, all other cost links are left with costs as is. See calvin.create_pyomo_model init_params function.

Returns:: nothing

eop_constraint_multiplier(x)¶

Set end-of-period storage constraints as a fraction of maximum available storage. Needed for limited foresight (annual) optimization.

Parameters:: x – (float) fraction of maximum storage to set lower bound
Returns:: nothing, but modifies the model object

fix_debug_flows(tol=1e-07)¶

Find infeasible constraints where debug flows occur. Fix them by either raising the UB (DBUGSNK) or lowering the LB (DBUGSRC).

Parameters:: tol – (float) Tolerance to identify nonzero debug flows
Returns run_again:: (boolean) whether debug mode needs to run again
Returns vol:: (float) total volume of constraint changes
Returns total_debug:: (float) total debug flow volume in this iteration also modifies the model object.

get_bound_adjustments()¶

Return a dataframe of links whose bounds were modified by fix_debug_flows, showing the initial and final lower/upper bounds and the net delta.

Returns:: DataFrame with columns i, j, k, lb_init, lb_final, lb_delta, ub_init, ub_final, ub_delta — only rows where at least one bound changed.

inflow_multiplier(x)¶

Multiply all network inflows by a constant.

Parameters:: x – (float) value to multiply inflows
Returns:: nothing, but modifies the model object

model_to_dataframe()¶

Converts the model to a pandas dataframe. Useful for computing objective values (costs) without having to postprocess.

Returns model_df:: (Pandas dataframe) Dataframe of upper_bound, cost, and flow (solution) values for each link

networkcheck()¶: Confirm constraint feasibility for the model object. (No inputs or outputs) :raises: ValueError when infeasibilities are identified.

no_gw_overdraft()¶

Impose constraints to prevent groundwater overdraft

(not currently implemented)

remove_debug_links()¶

Remove debug links from model object.

Returns:: dataframe of links, excluding debug links.

solve_pyomo_model(solver='highs', nproc=1, debug_mode=False, maxiter=10)¶

Solve Pyomo model (must be called after create_pyomo_model)

Parameters:

solver – (string) solver name. glpk, cplex, cbc, gurobi.
nproc – (int) number of processors. 1=serial.
debug_mode – (boolean) Whether to run in debug mode. Use when there may be infeasibilities in the network.
maxiter – (int) maximum iterations for debug mode.

Returns:

nothing, but assigns results to self.model.solutions.

Raises:

RuntimeError, if problem is found to be infeasible.

calvin.cosvfea.cosvf_check_bounds(rtype1_start_idx, init)¶

Check bounds of indiviudal’s COSVF Pmin and Pmax array during evolution.

Two checks:

Minimum penalty is zero
Pmax cannot be greater than Pmin

Parameters:: rtype1_start_idx – (int) position on individual parameter list at which rtype1 begin
Returns decorator:: (func) a decorator function that is applied after individual mating or mutation

calvin.cosvfea.cosvf_ea_main(toolbox, n_gen, mu, pwd, cxpb=1, mutpb=1, seed=None, log_name='calvin-cosvf-ea', checkpoint=None)¶

Main evolutionary algorithm using NSGA-III selection.

Parameters:

toolbox – (object) the DEAP toolbox constructed using cosvf_ea_toolbox
n_gen – (object) number of evolutionary generations to conduct (stopping criteria)
mu – (int) number of individuals in the evolutionary population
pwd – (path) directory to save evolutionary results and checkpoints
cxpb – (float) [0,1] probability of mating two individuals (consecutive pairs in pop)
mutpb – (float) [0,1] probability of mutating an individual
seed – (int) random seed (will assign random integer b/t 1 and 100 if not specified)
log_name – (string) global logger name to use, log file will save to pwd
checkpoint – (path) checkpoint file of previous EA to continue running

Returns:

nothing, but outputs evolutionary results to CSV and a pickled checkpoint

calvin.cosvfea.cosvf_ea_toolbox(cosvf_evaluate, nrtype, mu, nobj=3, cx_eta=10.0, mut_eta=40.0, mutind_pb=0.5)¶

Create a DEAP toolbox with the NSGA-III selection evolutionary algorithm.

Parameters:

cosvf_evaluate – (func) this function, which must be defined in the “main” run file, constructs a COSVF CALVIN model object, taking COSVF params as the argument for the model solve
nrtype – (list) [(int), (int)] a list with number of type 1 (quadratic) COSVF reservoirs as the first entry and number of type 2 (linear) COSVF reservoirs as the second entry
mu – (int) number of individuals in the population
nobj – (int) number of objectives
cx_eta – (float) Likeness degree of the simulated binary bounded crossover. High eta –> children close to parents; Low eta –> children far from parents
mut_eta – (float) Likeness degree of the polynomial bounded mutation.
mutind_pb – (float) [0,1] probability of mutating a parameter within a given individual.

Returns:

a DEAP toolbox for the evolutionary search

calvin.cosvfea.logbook_to_csv(logbook, pwd, seed)¶

Convert the logbook of evolutionary history to a CSV file.

Parameters:

logbook – (object) DEAP logbook of evolutionary history
pwd – (path) directory to save evolutionary results and checkpoints
seed – (int) random seed used in EA generations

Returns:

nothing, but outputs CSV file cosvf-ea-history-seed[number].csv

calvin.cosvfea.mu_from_pdiv(pdiv, nobj=3)¶: Get population count based on divisions per objective for NSGA-III

calvin.cosvfea.pdiv_from_mu(mu, nobj=3)¶: Get divisions per objective for NSGA-III from a population count.