compress

climatebenchpress.compressor.scripts.compress

compress 

compress(
    basepath: Path = Path(),
    exclude_dataset: Container[str] = tuple(),
    include_dataset: None | Container[str] = None,
    exclude_compressor: Container[str] = tuple(),
    include_compressor: None | Container[str] = None,
    data_loader_basepath: None | Path = None,
    chunked: bool = False,
    progress: bool = True,
)

Compress datasets with compressors.

Parameters:
  • basepath (Path, default: Path() ) –

    Compressed dataset will be stored in basepath / compressed-datasets.

  • exclude_dataset (Container[str], default: tuple() ) –

    Datasets to exclude from compression.

  • include_dataset (None | Container[str], default: None ) –

    Datasets to include in compression. If None, all datasets are included. If specified, only datasets in include_dataset will be compressed.

  • exclude_compressor (Container[str], default: tuple() ) –

    Compressors to exclude from compression.

  • include_compressor (None | Container[str], default: None ) –

    Compressors to include in compression. If None, all compressors are included. If specified, only compressors in include_compressor will be used.

  • data_loader_basepath (None | Path, default: None ) –

    Base path for the data loader datasets. If None, defaults to basepath / .. / data-loader. Input datasets will be loaded from data_loader_basepath / datasets.

  • chunked (bool, default: False ) –

    Whether to chunk the input data.

  • progress (bool, default: True ) –

    Whether to show a progress bar during compression.