OMF Python API

Python reader and writer for Open Mining Format version 2, a standard for mining data interchange backed by the Global Mining Guidelines Group.

Warning: These Python bindings are incomplete. The API may experience breaking changes before it is finished.

Missing Parts

• Composite geometry is not supported.

• Writing files is not supported.

• Can only read real files, specified by name. The Rust code is generic now, so it will be possible to support Python io.BytesIO objects as well. Will need to make sure that performance is still acceptable though.

• PyO3 now supports wrapping rich enums. Use that support rather than wrapping each variant as a separate struct.

What is OMF

OMF is an open-source serialization format and library to support data interchange across the entire mining community. Its goal is to standardize file formats and promote collaboration.

This repository provides a file format specification and a Rust library for reading and writing files, plus wrappers to use that library from C and Python.

Getting Started

The Reader object is the starting points for reading files. Project is the root object of the data contained within the file, storing a list of elements, each containing some geometry and a list of attributes.

Supported element geometries are:

• Points.
• Line segments.
• Triangulated surfaces.
• Grid surfaces.
  • Regular or tensor.
  • Any orientation.
• Block models, with optional sub-blocks.
  • Regular or tensor.
  • Any orientation.
  • Regular sub-blocks that lie on a grid within their parent, with octree or arbitrary layout.
  • Free-form sub-blocks that don't lie on any grid.
• Composite elements made out of any of the above. Not yet supported under Python.

Supported attribute data types are:

• Floating-point or signed integer values, including date and date-time values.
• Category values, storing an index used to look up name, color, or other sub-attributes.
• Boolean or filter values.
• 2D and 3D vector values.
• Text values.
• Color values.
• Projected texture images.
• UV mapped texture images.

Attributes values can be valid or null. They can be attached to different parts of each element type, such as the vertices vs. faces of a surface, or the parent blocks vs. sub-blocks of a block model.

Classes

Attribute

Describes data attached to an Element.

Each Element can have zero or more attributes, each attached to different parts of the element and each containing different types of data. On a set of points, one attribute might contain gold assay results and another rock-type classifications.

Attributes

Name	Type	Description
description	str
location	Location
metadata	Any
name	str
units	str

Functions

get_data() -> typing.Any

The attribute data.

AttributeDataBoolean

Boolean or filter data.

Attributes

Name	Type	Description
values	BooleanArray

AttributeDataCategory

Category data.

A name is required for each category, a color is optional, and other values can be attached as sub-attributes.

Attributes

Name	Type	Description
attributes	list[Attribute]
gradient	Optional[GradientArray]
names	NameArray
values	IndexArray

AttributeDataColor

Color data.

Attributes

Name	Type	Description
values	ColorArray

AttributeDataMappedTexture

A texture applied with UV mapping.

Typically applied to surface vertices. Applications may ignore other locations.

Attributes

Name	Type	Description
image	ImageArray
texcoords	TexcoordArray

AttributeDataNumber

Number data with flexible type.

Values can be stored as 32 or 64-bit signed integers, 32 or 64-bit floating point, date, or date-time. Valid dates are approximately ±262,000 years from the common era. Date-time values are written with microsecond accuracy, and times are always in UTC.

Attributes

Name	Type	Description
colormap	Optional[Any]
values	NumberArray

AttributeDataProjectedTexture

A texture defined as a rectangle in space projected along its normal.

Behavior of the texture outside the projected rectangle is not defined. The texture might repeat, clip the element, or itself be clipped to reveal the flat color of the element.

The attribute location must be Projected.

Attributes

Name	Type	Description
height	float
image	ImageArray
orient	Orient2
width	float

AttributeDataText

Text data.

Attributes

Name	Type	Description
values	TextArray

AttributeDataVector

2D or 3D vector data.

Attributes

Name	Type	Description
values	VectorArray

BlockModel

Block model geometry with optional sub-blocks.

Attributes

Name	Type	Description
grid	Any
orient	Orient3
subblocks	Optional[Any]

BooleanArray

Nullable booleans.

Functions

item_count() -> builtins.int

Number of items in the decompressed array. Zero for images.

BoundaryArray

Discrete color-map boundaries.

Functions

item_count() -> builtins.int

Number of items in the decompressed array. Zero for images.

BoundaryType

Bases: Enum

Enum indicating whether a particular boundary is inclusive or exclusive of the threshold value.

Attributes

Name	Type	Description
Less
LessEqual

ColorArray

Nullable colors.

Functions

item_count() -> builtins.int

Number of items in the decompressed array. Zero for images.

Element

Defines a single “object” or “shape” within the OMF file.

Each shape has a name plus other optional metadata, a “geometry” that describes a point-set, surface, etc., and a list of attributes that that exist on that geometry.

Attributes

Name	Type	Description
color	Optional[tuple[int, int, int, int]]
description	str
metadata	Any
name	str

Functions

attributes() -> builtins.list[Attribute]

List of attributes, if any.

geometry() -> typing.Any

The geometry of the element.

FreeformSubblockArray

Parent indices and corners of free-form sub-blocks.

Functions

item_count() -> builtins.int

Number of items in the decompressed array. Zero for images.

FreeformSubblocks

Block model geometry with optional sub-blocks.

Attributes

Name	Type	Description
subblocks	FreeformSubblockArray

GradientArray

Non-nullable colormap or category colors.

Functions

item_count() -> builtins.int

Number of items in the decompressed array. Zero for images.

Grid2Regular

Regularly spaced cells.

Attributes

Name	Type	Description
size	list[float]

Functions

count() -> builtins.list[builtins.int]

Returns the number of cells in each axis.

flat_corner_count() -> builtins.int

Returns the total number of cell corners.

flat_count() -> builtins.int

Returns the total number of cells.

Grid2Tensor

Tensor cells, where each row and column can have a different size.

Attributes

Name	Type	Description
u	ScalarArray
v	ScalarArray

Functions

count() -> builtins.list[builtins.int]

Returns the number of cells in each axis.

flat_corner_count() -> builtins.int

Returns the total number of cell corners.

flat_count() -> builtins.int

Returns the total number of cells.

Grid3Regular

Regularly spaced cells.

Attributes

Name	Type	Description
size	list[float]

Functions

count() -> builtins.list[builtins.int]

Returns the number of blocks in each axis.

flat_corner_count() -> builtins.int

Returns the total number of block corners.

flat_count() -> builtins.int

Returns the total number of blocks.

Grid3Tensor

Tensor cells, where each row, column and layer can have a different size.

Attributes

Name	Type	Description
u	ScalarArray
v	ScalarArray
w	ScalarArray

Functions

count() -> builtins.list[builtins.int]

Returns the number of blocks in each axis.

flat_corner_count() -> builtins.int

Returns the total number of block corners.

flat_count() -> builtins.int

Returns the total number of blocks.

GridSurface

A surface defined by a 2D grid a height on each grid vertex.

Attributes

Name	Type	Description
grid	Any
heights	Optional[ScalarArray]
orient	Orient2

ImageArray

An image in PNG or JPEG encoding.

Functions

item_count() -> builtins.int

Number of items in the decompressed array. Zero for images.

IndexArray

Nullable category index values.

Functions

item_count() -> builtins.int

Number of items in the decompressed array. Zero for images.

Limits

Memory limits for reading OMF files.

Attributes

Name	Type	Description
image_bytes	Optional[int]
image_dim	Optional[int]
json_bytes	Optional[int]
validation	Optional[int]

LineSet

A set of line segments.

Attributes

Name	Type	Description
origin	NDArray[float64]
segments	SegmentArray
vertices	VertexArray

Location

Bases: Enum

Describes what part of the geometry an attribute attaches to.

See the documentation for each Geometry variant for a list of what locations are valid.

Attributes

Name	Type	Description
Categories
Elements
Primitives
Projected
Subblocks
Vertices

NameArray

Non-nullable category names.

Functions

item_count() -> builtins.int

Number of items in the decompressed array. Zero for images.

NumberArray

Nullable number values, floating-point or signed integer.

Functions

item_count() -> builtins.int

Number of items in the decompressed array. Zero for images.

NumberColormapContinuous

A continuous colormap linearly samples a color gradient within a defined range.

A value X% of way between min and max should use the color from X% way down gradient. When that X doesn't land directly on a color use the average of the colors on either side, inverse-weighted by the distance to each.

Values below the minimum use the first color in the gradient array. Values above the maximum use the last.

Attributes

Name	Type	Description
gradient	GradientArray

Functions

range() -> typing.Any

Value range.

NumberColormapDiscrete

A discrete colormap divides the number line into adjacent but non-overlapping ranges and gives a flat color to each range.

Values above the last boundary use end_color.

Attributes

Name	Type	Description
boundaries	BoundaryArray
gradient	GradientArray

Omf1Converter

Converts a OMF1 files to OMF2.

This object allows you to set up the desired parameters then convert one or more files.

Functions

compression() -> builtins.int

Returns the current compression level.

convert(input_path: builtins.str, output_path: builtins.str) -> builtins.list[Problem]

Runs a conversion from one filename to another.

The output file will be created if it does not exist, and truncated if it does. On success the validation warnings are returned.

May be called more than once to convert multiple files with the same parameters.

limits() -> Limits

Returns the current limits.

set_compression(compression: builtins.int) -> None

Set the compression level to use when writing. Range 0-9.

set_limits(limits: Limits) -> None

Set the limits to use during conversion.

OmfException

Bases: Exception

Base class for all OMF exceptions.

OmfFileIoException

Bases: OmfException

Exception raised when a file IO error occurs.

OmfInvalidDataException

Bases: OmfException

Exception raised when an OMF file contains invalid data.

OmfJsonException

Bases: OmfException

Exception raised when a JSON error occurs. Can also be triggered by exceeding the json_bytes safety limit.

OmfLimitExceededException

Bases: OmfException

Exception raised when a safety limit was exceeded.

OmfNotSupportedException

Bases: OmfException

Exception raised when some action is not supported.

OmfValidationFailedException

Bases: OmfException

Exception raised when an OMF validation error occurs.

Orient2

Defines the position and orientation of a 2D plane in 3D space.

Attributes

Name	Type	Description
origin	NDArray[float64]
u	NDArray[float64]
v	NDArray[float64]

Orient3

Defines the position and orientation of a 3D sub-space.

Attributes

Name	Type	Description
origin	NDArray[float64]
u	NDArray[float64]
v	NDArray[float64]
w	NDArray[float64]

PointSet

Point set geometry.

Attributes

Name	Type	Description
origin	NDArray[float64]
vertices	VertexArray

Problem

A single validation problem.

Attributes

Name	Type	Description
field	Optional[str]
name	Optional[str]
reason	str
type_name	str

Functions

is_error() -> builtins.bool

True if the reason is an error, false if it is a warning.

Project

This is the root element of an OMF file, holding global metadata and a list of Elements that describe the objects or shapes within the file.

Attributes

Name	Type	Description
application	str
author	str
coordinate_reference_system	str
description	str
name	str
origin	NDArray[float64]
units	str

Functions

elements() -> builtins.list[Element]

List of elements.

Reader

OMF reader object.

Typical usage pattern is:

• Create the reader object.
• Optional: retrieve the file version with reader.version().
• Optional: adjust the limits with reader.set_limits(...).
• Read the project from the file with reader.project().
• Iterate through the project's contents to find the elements and attributes you want to load.
• For each of those items load the array or image data.

Warning: When loading arrays and images from OMF files, beware of "zip bombs" where data is maliciously crafted to expand to an excessive size when decompressed, leading to a potential denial of service attack. Use the limits provided and check arrays sizes before allocating memory.

Functions

array_booleans(array: BooleanArray) -> tuple[typing.Any, typing.Any]

Read a Boolean array and return a tuple of two numpy arrays: the booleans themselves, and a mask indicating any values that are null.

array_boundaries(array: BoundaryArray) -> builtins.list[tuple[typing.Any, BoundaryType]]

Read a Boundary array and return the values as a list of tuples. The first element in each tuple is an enum value, :const:BoundaryType.Less or :const:BoundaryType.LessEqual, indicating whether the boundary is inclusive or exclusive of the threshold value. The second element in each tuple is the threshold value, which may be :class:numpy.float32, :class:numpy.float64, :class:numpy.int64, :class:numpy.datetime64[D] (for dates), or :class:numpy.datetime64[us] (for datetimes).

array_color(array: ColorArray) -> tuple[numpy.typing.NDArray[numpy.uint8], typing.Any]

Read a Color array and return a tuple of two numpy arrays: the colors represented as (r, g, b, a) rows, and a mask indicating any rows that are null.

array_freeform_subblocks(array: FreeformSubblockArray) -> tuple[numpy.typing.NDArray[numpy.uint32], typing.Any]

Read a FreeformSubblock array and return a tuple of two numpy arrays, the first containing parent block indices as (u, v, w) triplets, and the second containing the corresponding corners as (u_min, v_min, w_min, u_max, v_max, w_max). The type of the indices array is :class:numpy.uint32, and the type of the corners array is :class:numpy.float32 or :class:numpy.float64.

array_gradient(array: GradientArray) -> numpy.typing.NDArray[numpy.uint8]

Read a Gradient array and return the values as a two‐dimensional numpy array whose rows are (r, g, b, a) components.

array_indices(array: IndexArray) -> tuple[numpy.typing.NDArray[numpy.uint32], typing.Any]

Read an Index array and return a tuple of two numpy arrays: the indices themselves, and a mask indicating any values that are null.

array_names(array: NameArray) -> builtins.list[builtins.str]

Read a Name array and return the values as a list of strings.

array_numbers(array: NumberArray) -> tuple[typing.Any, typing.Any]

Read a Number array and return a tuple of two numpy arrays: the values themselves, and a mask indicating any values that are null. The number array’s type is one of :class:numpy.float32, :class:numpy.float64, :class:numpy.int64, :class:numpy.datetime64[D] (for dates), or :class:numpy.datetime64[us] (for datetimes).

array_regular_subblocks(array: RegularSubblockArray) -> tuple[numpy.typing.NDArray[numpy.uint32], numpy.typing.NDArray[numpy.uint32]]

Read a RegularSubblock array and return a tuple of two numpy arrays, the first containing parent block indices as (u, v, w) triplets, and the second containing the corresponding corners as (u_min, v_min, w_min, u_max, v_max, w_max). The type of both arrays is :class:numpy.uint32.

array_scalars(array: ScalarArray) -> typing.Any

Read a Scalar array and return a numpy array containing its values. The array’s type may be :class:numpy.float32 or :class:numpy.float64.

array_segments(array: SegmentArray) -> numpy.typing.NDArray[numpy.uint32]

Read a Segment array and return the values as a two‐dimensional numpy array. The array’s type is :class:numpy.uint32, and its rows are (a, b) indices into a vertex array.

array_texcoords(array: TexcoordArray) -> typing.Any

Read a Texcoord array and return the values as a two‐dimensional numpy array. The array’s type may be :class:numpy.float32 or :class:numpy.float64, and its rows are (u, v) coordinates.

array_text(array: TextArray) -> builtins.list[typing.Optional[builtins.str]]

Read a Text array and return the values as a list of optional strings.

array_triangles(array: TriangleArray) -> numpy.typing.NDArray[numpy.uint32]

Read a Triangle array and return the values as a two‐dimensional numpy array. The array’s type is :class:numpy.uint32, and its rows are (a, b, c) indices into a vertex array.

array_vectors(array: VectorArray) -> tuple[typing.Any, typing.Any]

Read a Vector array and return a tuple of two numpy arrays: the vectors themselves, and a mask indicating any rows that are null. The vector array’s type may be :class:numpy.float32 or :class:numpy.float64, and the vectors may be 2D (x, y) or 3D (x, y, z).

array_vertices(array: VertexArray) -> typing.Any

Read a Vertex array and return the values as a two‐dimensional numpy array. The array’s type may be :class:numpy.float32 or :class:numpy.float64, and its rows are the (x, y, z) coordinates.

image_bytes(array: ImageArray) -> bytes

Read the encoded bytes of an Image and return them as a bytes array. The bytes may be in JPEG or PNG format; use the image header to distinguish the two.

limits() -> Limits

Returns the current limits.

project() -> tuple[Project, builtins.list[Problem]]

Reads, validates, and returns the root Project object from the file.

Fails with an error if an IO error occurs, the json_bytes limit is exceeded, or validation fails. Validation warnings are returned alongside the project if successful or included with the errors if not.

set_limits(limits: Limits) -> None

Sets the memory limits.

These limits prevent the reader from consuming excessive system resources, which might allow denial of service attacks with maliciously crafted files. Running without limits is not recommended.

version() -> builtins.list[builtins.int]

Return the version number of the file, which can only be [2, 0] right now.

RegularSubblockArray

Parent indices and corners of regular sub-blocks.

Functions

item_count() -> builtins.int

Number of items in the decompressed array. Zero for images.

RegularSubblocks

Block model geometry with optional sub-blocks.

Attributes

Name	Type	Description
count	list[int]
mode	Optional[SubblockMode]
subblocks	RegularSubblockArray

ScalarArray

Floating-point scalar values.

Functions

item_count() -> builtins.int

Number of items in the decompressed array. Zero for images.

SegmentArray

Line segments as indices into a vertex array.

Functions

item_count() -> builtins.int

Number of items in the decompressed array. Zero for images.

SubblockMode

Bases: Enum

An optional mode for regular sub-blocks.

Attributes

Name	Type	Description
Full
Octree

Surface

A surface made up of triangles.

Attributes

Name	Type	Description
origin	NDArray[float64]
triangles	TriangleArray
vertices	VertexArray

TexcoordArray

UV texture coordinates.

Functions

item_count() -> builtins.int

Number of items in the decompressed array. Zero for images.

TextArray

Nullable text.

Functions

item_count() -> builtins.int

Number of items in the decompressed array. Zero for images.

TriangleArray

Triangles as indices into a vertex array.

Functions

item_count() -> builtins.int

Number of items in the decompressed array. Zero for images.

VectorArray

Nullable 2D or 3D vectors.

Functions

item_count() -> builtins.int

Number of items in the decompressed array. Zero for images.

VertexArray

Vertex locations in 3D.

Functions

item_count() -> builtins.int

Number of items in the decompressed array. Zero for images.

Functions

detect_omf1(path: builtins.str) -> builtins.bool

Returns true if the path looks more like OMF1 than OMF2.

version() -> builtins.str

Returns the version of the library