CSV_MERGE

Merge multiple CSV files into one. Each row will be tagged at least with the original filename in the FILENAME column.

Additionally, if realization, iteration and ensemble name can be inferred from the paths, it will be added to the REAL, ITER and ENSEMBLE and ENSEMBLESET columns.

The columns in the ensembles need not be the same. Similar column names will be merged, differing column names will be padded (with NaN) in the resulting dataset where they don’t exist.

Do not assume anything on the ordering of columns after merging.

usage: csv_merge [-h] [-o OUTPUT] [--memoryconservative]
                 [--dropconstantcolumns] [--filecolumn FILECOLUMN] [-v]
                 [--debug] [--version]
                 csvfiles [csvfiles ...]

Positional Arguments

csvfiles

input csv files

Named Arguments

-o, --output

name of output csv file. Use - or stdout to dump output to stdout.

Default: “merged.csv”

--memoryconservative, -m

Conserve memory while merging at the expense of speed. Default is to use up to twice as much memory as the size of the final CSV. Do not use unless normal mode fails.

Default: False

--dropconstantcolumns

Drop (delete) constant columns in the merged dataset

Default: False

--filecolumn

Name of column containing original filename

Default: “FILENAME”

-v, --verbose

Verbose output

Default: False

--debug

Debug output, more verbose than –verbose

Default: False

--version

show program’s version number and exit