v3.6 (18 May 2023)#
This document explains the changes made to Iris for this release (View all changes.)
v3.6 Release Highlights
We’re so excited about our recent support for delayed saving of lazy data to netCDF (PR #5191) that we’re celebrating this important step change in behaviour with its very own dedicated release 🥳
By using iris.save(..., compute=False)
you can now save to multiple NetCDF files
in parallel. See the new compute
keyword in iris.fileformats.netcdf.save()
.
This can share and reuse any common (lazy) result computations, and it makes much
better use of resources during any file-system waiting (i.e., it can use such periods
to progress the other saves).
Usage example:
# Create output files with delayed data saving.
delayeds = [
iris.save(cubes, filepath, compute=False)
for cubes, filepath in zip(output_cubesets, output_filepaths)
]
# Complete saves in parallel.
dask.compute(*delayeds)
This advance also includes another substantial benefit, because NetCDF saves can now use a Dask.distributed scheduler. With Distributed you can parallelise the saves across a whole cluster. Whereas previously, the NetCDF saving only worked with a “threaded” scheduler, limiting it to a single CPU.
We’re so super keen for the community to leverage the benefit of this new feature within Iris that we’ve brought this release forward several months. As a result, this minor release of Iris is intentionally light in content. However, there are some other goodies available for you to enjoy, such as:
Performing lazy arithmetic with an Iris
Cube
and adask.array.Array
, andVarious improvements to our documentation resulting from adoption of sphinx-design and sphinx-apidoc.
As always, get in touch with us on GitHub, particularly if you have any feedback with regards to delayed saving, or have any issues or feature requests for improving Iris. Enjoy!
v3.6.1 (26 June 2023)#
v3.6.1 Patches
📢 Announcements
Welcome and congratulations to @sloosvel who made their first contribution to Iris! 🎉
The patches in this release of Iris include:
✨ Features
@rcomer rewrote
broadcast_to_shape()
so it now handles lazy data. This pull-request has been included to support PR #5341. (PR #5307) [pre-v3.7.0
]
🐛 Bugs Fixed
@stephenworsley fixed
convert_units()
to allow unit conversion of lazy data when using a Distributed scheduler. (Issue #5347, PR #5349)@schlunma fixed a bug in the concatenation of cubes with aux factories which could lead to a KeyError due to dependencies that have not been properly updated. (Issue #5339, PR #5340)
@schlunma fixed a bug which realized all weights during weighted aggregation. Now weighted aggregation is fully lazy again. (Issue #5338, PR #5341)
🚀 Performance Enhancements
@sloosvel improved
concatenate_cube()
andconcatenate()
to ensure that lazy auxiliary coordinate points and bounds are not realized. This change now allows cubes with high-resolution auxiliary coordinates to concatenate successfully whilst using a minimal in-core memory footprint. (Issue #5115, PR #5142)
Note that, the above contribution labelled with pre-v3.7.0
is part of the
forthcoming Iris v3.7.0
release, but requires to be included in this patch
release.
📢 Announcements#
@bjlittle added the community Contributor Covenant code of conduct. (PR #5291)
✨ Features#
@pp-mo and @lbdreyer supported delayed saving of lazy data, when writing to the netCDF file format. See delayed netCDF saves. Also with significant input from @fnattino. (PR #5191)
@rcomer tweaked binary operations so that dask arrays may safely be passed to arithmetic operations and
mask_cube()
. (PR #4929)
🐛 Bugs Fixed#
💣 Incompatible Changes#
N/A
🚀 Performance Enhancements#
N/A
🔥 Deprecations#
N/A
🔗 Dependencies#
📚 Documentation#
@tkknight migrated to sphinx-design over the legacy sphinx-panels. (PR #5127)
@tkknight updated the
make
target forhelp
and addedlivehtml
to auto generate the documentation when changes are detected during development. (PR #5258)@tkknight updated the Installing a Development Version instructions to use
pip
. (PR #5273)@tkknight removed the legacy custom sphinx extensions that generate the API documentation. Instead use a less complex approach via sphinx-apidoc. (PR #5264)
@trexfeathers re-wrote the Releases documentation for clarity, and wrote a step-by-step Release Do-Nothing Script for the release process. (PR #5134)
@trexfeathers and @tkknight added a dark-mode friendly logo. (PR #5278)
💼 Internal#
@bjlittle added the codespell pre-commit
git-hook
to automate spell checking within the code-base. (PR #5186)@bjlittle and @trexfeathers (reviewer) added a check-manifest GitHub Action and pre-commit
git-hook
to automate verification of assets bundled within asdist
and binarywheel
of our scitools-iris PyPI package. (PR #5259)@rcomer removed a now redundant copying workaround from Resolve testing. (PR #5267)
@bjlittle and @trexfeathers (reviewer) migrated
setup.cfg
topyproject.toml
, as motivated by PEP-0621. (PR #5262)@bjlittle adopted pypa/build recommended best practice to build a binary
wheel
from thesdist
. (PR #5266)@trexfeathers enabled on-demand benchmarking of Pull Requests; see here. (PR #5286)