iris.io.format_picker

A module to provide convenient file format identification through a combination of filename extension and file based magic numbers.

To manage a collection of FormatSpecifications for loading:

import iris.io.format_picker as fp
import matplotlib.pyplot as plt
fagent = fp.FormatAgent()
png_spec = fp.FormatSpecification('PNG image', fp.MagicNumber(8),
                                  0x89504E470D0A1A0A,
                                  handler=lambda filename: plt.imread(filename),
                                  priority=5
                                  )
fagent.add_spec(png_spec)

To identify a specific format from a file:

with open(png_filename, 'rb') as png_fh:
    handling_spec = fagent.get_spec(png_filename, png_fh)

In the example, handling_spec will now be the png_spec previously added to the agent.

Now that a specification has been found, if a handler has been given with the specification, then the file can be handled:

handler = handling_spec.handler
if handler is None:
   raise ValueError('File cannot be handled.')
else:
   result = handler(filename)

The calling sequence of handler is dependent on the function given in the original specification and can be customised to your project’s needs.

class iris.io.format_picker.DataSourceObjectProtocol

Bases: FileElement

A FileElement that simply returns the URI entry itself.

This enables a arbitrary non-string data object to be passed, subject to subsequent checks on the object itself (specified in the handler).

Constructs a new file element, which may require a file buffer.

Kwargs:

• requires_fh - Whether this FileElement needs a file buffer.

get_element(basename, file_handle)

Called when identifying the element of a file that this FileElement is representing.

class iris.io.format_picker.FileElement(requires_fh=True)

Bases: object

Represents a specific aspect of a FileFormat which can be identified using the given element getter function.

Constructs a new file element, which may require a file buffer.

Kwargs:

• requires_fh - Whether this FileElement needs a file buffer.

get_element(basename, file_handle)

Called when identifying the element of a file that this FileElement is representing.

class iris.io.format_picker.FileExtension(requires_fh=True)

Bases: FileElement

A FileElement that returns the extension from the filename.

Constructs a new file element, which may require a file buffer.

Kwargs:

• requires_fh - Whether this FileElement needs a file buffer.

get_element(basename, file_handle)

Called when identifying the element of a file that this FileElement is representing.

class iris.io.format_picker.FormatAgent(format_specs=None)

Bases: object

The FormatAgent class is the containing object which is responsible for identifying the format of a given file by interrogating its children FormatSpecification instances.

Typically a FormatAgent will be created empty and then extended with the FormatAgent.add_spec() method:

agent = FormatAgent()
agent.add_spec(NetCDF_specification)

Less commonly, this can also be written:

agent = FormatAgent(NetCDF_specification)

add_spec(format_spec)

Add a FormatSpecification instance to this agent for format consideration.

get_spec(basename, buffer_obj)

Pick the first FormatSpecification which can handle the given filename and file/buffer object.

Note

buffer_obj may be None when a seekable file handle is not feasible (such as over the http protocol). In these cases only the format specifications which do not require a file handle are tested.

class iris.io.format_picker.FormatSpecification(format_name, file_element, file_element_value, handler=None, priority=0, constraint_aware_handler=False)

Bases: object

Provides the base class for file type definition.

Every FormatSpecification instance has a name which can be accessed with the FormatSpecification.name property and a FileElement, such as filename extension or 32-bit magic number, with an associated value for format identification.

Constructs a new FormatSpecification given the format_name and particular FileElements

Args:

• format_name - string name of fileformat being described

• file_element - FileElement instance of the element which identifies this FormatSpecification

• file_element_value - The value that the file_element should take if a file matches this FormatSpecification

Kwargs:

• handler - function which will be called when the specification has been identified and is required to handler a format.

  If None, then the file can still be identified but no handling can be done.

• priority - Integer giving a priority for considering this specification where higher priority means sooner consideration.

property file_element

property file_element_value

property handler

The handler function of this FileFormat. (Read only)

property name

The name of this FileFormat. (Read only)

class iris.io.format_picker.LeadingLine(requires_fh=True)

Bases: FileElement

A FileElement that returns the first line from the file.

Constructs a new file element, which may require a file buffer.

Kwargs:

• requires_fh - Whether this FileElement needs a file buffer.

get_element(basename, file_handle)

Called when identifying the element of a file that this FileElement is representing.

class iris.io.format_picker.MagicNumber(num_bytes, offset=None)

Bases: FileElement

A FileElement that returns a byte sequence in the file.

Constructs a new file element, which may require a file buffer.

Kwargs:

• requires_fh - Whether this FileElement needs a file buffer.

get_element(basename, file_handle)

Called when identifying the element of a file that this FileElement is representing.

len_formats = {4: '>L', 8: '>Q'}

class iris.io.format_picker.UriProtocol

Bases: FileElement

A FileElement that returns the “scheme” and “part” from a URI, using decode_uri().

Constructs a new file element, which may require a file buffer.

Kwargs:

• requires_fh - Whether this FileElement needs a file buffer.

get_element(basename, file_handle)

Called when identifying the element of a file that this FileElement is representing.