iris.fileformats.cf.dataset#

The format-agnostic description of a CF-conforming array store.

API reference

CFDataset and CFDatasetVariable are what the rest of the CF layer is written against. They are deliberately small: only what the CFVariable classes, the CF loader and the CF saver actually need, with one implementation per storage format.

The property attributes is the only route to a CF attribute, and every storage property is a named, typed member.

class iris.fileformats.cf.dataset.CFDataset[source]#

Bases: ABC

A CF-conforming array store, open for reading or writing.

__enter__()[source]#

Return this dataset, for use as a context manager.

Return type:: CFDataset

__exit__(*exc_info)[source]#

Close this dataset on leaving the context, whatever happened in it.

Return type:: None

abstract property attributes: MutableMapping#: The store’s global attributes.

abstract close()[source]#

Release the store’s resources.

Return type:: None

abstract property closed: bool#: Whether close() has already released the store.

abstract create_dimension(name, size)[source]#

Declare a dimension of the given length.

size=None requests an unlimited dimension. A store with no such concept raises NotImplementedError.

Parameters:

name (str)
size (int | None)

Return type:

None

abstract create_variable(name, dtype, dimensions=(), *, fill_value=None, **storage_settings)[source]#

Create and return a new variable.

**storage_settings is storage-specific: zlib, complevel and chunksizes for netCDF; compressors, chunks and shards for Zarr.

Parameters:: name (str)
Return type:: CFDatasetVariable

abstract property dimensions: Mapping#: The store’s dimension lengths, by name.

abstract finalise()[source]#

Perform the store’s one-shot completion step, after every write.

Deliberately not part of close(): a worker that writes one region of a store must close its handle without performing a completion step that only one process may perform.

Return type:: None

abstract property format: str#

The concrete format of this dataset.

Examples: NETCDF4, NETCDF3_CLASSIC, ZARR_V2 or ZARR_V3.

What the loader, the saver and LOAD_PROBLEMS say when they have to name the format. Reconstructing it from the implementation class would be the type test this interface exists to forbid.

abstract property location: str#: The store’s path or URL, for messages and data proxies.

abstract property mode: str#

r, r+, a, w or w-.

Type:: The mode the store was opened in

abstract property storage_settings: Mapping#: The store-wide settings this store holds, in the backend’s spelling.

abstract supports_dtype(dtype)[source]#

Whether this store can hold a variable of this dtype.

The capability question the saver asks before it narrows an integer type. Asking it of the store, rather than testing the format, is what keeps the saver free of format names.

Parameters:: dtype (numpy.dtype) – The dtype a variable would be created with.
Return type:: bool

abstract sync()[source]#

Flush buffered writes to the store.

Return type:: None

abstract property variables: Mapping#: The store’s variables, by name.

class iris.fileformats.cf.dataset.CFDatasetVariable[source]#

Bases: ABC

One named array in a CF-conforming dataset.

abstract __getitem__(keys)[source]#

Return the indexed portion of the variable’s data.

Return type:: ndarray

__len__()[source]#

Return the length of the variable’s leading dimension.

Return type:: int

abstract __setitem__(keys, values)[source]#

Write values into the indexed portion of the variable.

Return type:: None

abstract property attributes: MutableMapping#: The variable’s CF attributes, materialised once at construction.

abstract property chunking: tuple | None#: The variable’s storage chunk shape, or None when unchunked.

deprecated_netcdf_member(name)[source]#

Return a netCDF4-only member of the object backing this variable.

The one-release compatibility route for code that reached netCDF4 API through a CFVariable. A store with no backing netCDF4 object raises AttributeError.

Parameters:: name (str)
Return type:: Any

abstract property dimensions: tuple#: The names of the dimensions this variable spans, in order.

abstract property dtype: dtype#: A numpy.dtype, or str for a variable-length string.

abstract property fill_value: Any#: The store’s own fill value, or None if it has none.

abstract property format: str#

The concrete format of the dataset holding this variable.

Repeated here from the dataset so that a variable can answer for itself, as location is. For messages and diagnostics only: generic code asks for a capability, never for a format.

abstract property location: str#: The path or URL of the dataset holding this variable.

abstract property name: str#: The variable’s name within its dataset.

property ndim: int#: The number of dimensions the variable spans.

abstract read_data(chunking_policy)[source]#

Return this variable’s data, lazily where appropriate.

The one route by which the CF loader obtains arrays, so a new storage format mainly needs a new implementation of this method. An implementation is expected to share the CF-layer chunking helpers in iris.fileformats.cf.loader - _get_actual_dtype for a lazy result’s dtype and _LAZYVAR_MIN_BYTES for the small-array cutoff - as NetCDFDatasetVariable does, rather than reinventing them. Read _LAZYVAR_MIN_BYTES through the module object (loader._LAZYVAR_MIN_BYTES), not by from ... import _LAZYVAR_MIN_BYTES, so that tests can patch the cutoff.

chunking_policy is called with no arguments and returns the (chunks, dims_fixed) pair for a lazy result. Call it only once the result is known to be lazy, otherwise it may raise. Under from_file() it raises for a variable the store has not chunked, and we want to avoid this when a variable is small enough to read whole.

Parameters:: chunking_policy (Callable[[], tuple])
Return type:: Any

abstract property shape: tuple#: The variable’s shape.

abstract property size: int#: The total number of elements in the variable.

abstract property storage_settings: Mapping#

The storage settings this variable holds, in the backend’s spelling.

zlib, complevel, chunksizes for netCDF; chunks, shards, compressors, filters, serializer, zarr_format for Zarr. Readable, never interpreted: no generic code may branch on a key. The typed, guaranteed-present member is chunking.

abstract write_handle()[source]#

Return a picklable object supporting __setitem__ on this variable.

This is what a Dask worker receives as a da.store target, so it must survive pickling and must remain usable after the dataset it came from has been closed.

Return type:: Any

class iris.fileformats.cf.dataset.TrackedAttributes(source, *, ignored=())[source]#

Bases: MutableMapping

A variable’s attributes, recording which of them have been looked up.

Single-key lookups record: __getitem__(), get() and __contains__(). Bulk access does not: iteration, keys(), values(), items() and len() leave the record alone. untracked reads a single key without recording it.

Wrap source, treating any ignored name it holds as already read.

Parameters:

source (MutableMapping)
ignored (Iterable[str])

__contains__(key)[source]#

Return whether key is present, recording a hit as read.

Parameters:: key (object)
Return type:: bool

__delitem__(key)[source]#

Remove key, and any record of it having been read.

Parameters:: key (str)
Return type:: None

__getitem__(key)[source]#

Return key’s value, recording it as read.

Parameters:: key (str)
Return type:: Any

__iter__()[source]#

Return an iterator over the attribute names, recording nothing.

Return type:: Iterator[str]

__len__()[source]#

Return the number of attributes, recording nothing.

Return type:: int

__repr__()[source]#

Return a string representation, recording nothing.

Return type:: str

__setitem__(key, value)[source]#

Set key’s value, recording nothing.

Parameters:

key (str)
value (Any)

Return type:

None

clear() → None. Remove all items from D.#

get(k[, d]) → D[k] if k in D, else d. d defaults to None.#

items()[source]#

Return a view of the attribute pairs, recording nothing.

Return type:: ItemsView

keys()[source]#

Return a view of the attribute names, recording nothing.

Return type:: KeysView

pop(k[, d]) → v, remove specified key and return the corresponding#: value. If key is not found, d is returned if given, otherwise KeyError is raised.

popitem() → (k, v), remove and return some (key, value) pair#: as a 2-tuple; but raise KeyError if D is empty.

property read: frozenset#: The names looked up since construction or the last reset().

reset()[source]#

Forget every recorded lookup, re-seeding from the ignored names.

Return type:: None

setdefault(k[, d]) → D.get(k,d), also set D[k]=d if k not in D#

property unread: frozenset#: The names present but not looked up since the last reset().

property untracked: Mapping#: A read-only view of the same attributes that records nothing.

update([E, ]**F) → None. Update D from mapping/iterable E and F.#

If E present and has a .keys() method, does:: for k in E.keys(): D[k] = E[k]
If E present and lacks .keys() method, does:: for (k, v) in E: D[k] = v
In either case, this is followed by:: for k, v in F.items(): D[k] = v

values()[source]#

Return a view of the attribute values, recording nothing.

Return type:: ValuesView