Dtypes¶

Data type classes are used to represents ways to interpret binary data. The Dtype class is an abstract base class, but its constructor can be used to conveniently create the correct sub-class.

The concrete data-type classes are:

DtypeSingle: An interpretation of a single value, such as a 32-bit float, or a 10 bit integer.
DtypeArray: Adds an item count to represent an array of values of the same type, such as 1000 u8 or 5 bool flags.
DtypeTuple: An arbitrary sequence of other data types.

These classes all have a from_params method to create them, but also have a particular formatting that can be used in the base Dtype from_string method which will delegate to the correct sub-class.

It’s usually best to create them via a call to Dtype('some_formatted_string').

In places where a Dtype is expected as a parameter you can just supply the string format and it will find the correct type automatically.

Some examples of equivalent types, going from most verbose to least:

DtypeSingle.from_params(DtypeKind.UINT, 8)
DtypeSingle('u8')
Dtype('u8')
'u8'  # When used as a parameter

DtypeArray.from_params(DtypeKind.FLOAT, 16, 20)
DtypeArray('[f16; 20]')
Dtype('[f16; 20]')
'[f16; 20]'  # When used as a parameter

DtypeTuple.from_params([DtypeSingle.from_params(DtypeKind.UINT, 8), DtypeArray.from_params(DtypeKind.FLOAT, 16, 20)])
DtypeTuple.from_params(['u8', '[f16; 20]'])
DtypeTuple('(u8, [f16; 20])')
Dtype('(u8, [f16; 20])')
'(u8, [f16; 20])'  # When used as a parameter

class Dtype(s: str | None = None, /)¶

Bases: ABC

An abstract base class for the DtypeSingle, DtypeArray and DtypeTuple classes.

The __init__ method can be used to construct its sub-classes via a formatted string.

Dtype sub-class instances are immutable. They are often created implicitly via a token string.

>>> a_12bit_int = Dtype('i12')  # Creates a DtypeSingle
>>> five_16_bit_floats = Dtype('[f16; 5]')  # Creates a DtypeArray
>>> a_tuple = Dtype('(bool, u7)')  # Creates a DtypeTuple

abstract evaluate(**kwargs)¶

Create a concrete Dtype using the values provided.

If a Dtype has been defined in terms of expressions for its size or number of items then this method can return a concrete Dtype instance. If the Dtype does not contain any expressions then this method will just return it unchanged.

concrete = Dtype('u32')
e1 = Dtype('u{my_size}')
e2 = Dtype('[u8; {my_items}]')

assert e1.evaluate(my_size=32) == concrete
assert e2.evaluate(my_items=10).bit_length == 80

Return type:: Self

abstract classmethod from_params(*args, **kwargs)¶

Create a new Dtype sub-class from parameters.

See the sub-classes for the parameters needed for each type.

Return type:: Self

classmethod from_string(s, /)¶

Create a new Dtype sub-class from a token string.

Parameters:: s (str) – The formatted string to convert to a Dtype.
Return type:: Self

Some token string examples:

'u12': A DtypeSingle representing an unsigned 12-bit integer.
'[i6; 5]': A DtypeArray of 5 signed 6-bit integers.
'(bool, hex4, f16)': A DtypeTuple of a boolean, a 4-char hex value and a 16-bit float.

As a shortcut the Dtype constructor can be used directly with a token string.

Dtype(s) is equivalent to Dtype.from_string(s).

abstract has_dynamic_size()¶

Return whether the dtype can stretch to fit the available data.

Dynamically sized data types can be used to consume all available data. They can only be used as the final part of compound data types or field types.

>>> d = Dtype('u')
>>> d.has_dynamic_size()
True
>>> d.unpack('0b1')
1
>>> d.unpack('0x00001')
1

Return type:: bool

abstract has_known_size()¶

Return whether the size of the dtype is fully known.

This will be True if the dtype has a known length that doesn’t depend on any parameters or available data, otherwise it will be False.

This will return False for data types with dynamic sizes, and for those whose size depends on an expression without a known value.

>>> Dtype('u32').has_known_size()
True
>>> Dtype('[f16; 4]').has_known_size()
True
>>> Dtype('[u32;]').has_known_size()
False
>>> Dtype('u{x}').has_known_size()
False

Return type:: bool

abstract info()¶

Return a descriptive string with information about the Dtype.

Note that the output is designed to be helpful to users and is not considered part of the API. You should not use the output programmatically as it may change even between point versions.

Return type:: str

abstract pack(value, /)¶

Create and return a new Bits from a value.

The value parameter should be of a type appropriate to the data type.

Return type:: Bits

unpack(b, /)¶

Unpack a Bits to find its value.

The b parameter should be a Bits of the appropriate length, or an object that can be converted to a Bits.

property bit_length: int | None¶

The total length of the data type in bits.

Returns None if the data type doesn’t have a fixed or known length.

>>> Dtype('u12').bit_length
12
>>> Dtype('[u12; 5]').bit_length
60
>>> Dtype('(hex5, bool)').bit_length
21
>>> Dtype('i').bit_length
None

class DtypeSingle(s: str | None = None, /)¶

Bases: Dtype

A data type of a single kind representing a single value.

This is used to represent the simplest data types, such as an integer, float or a hex string.

evaluate(**kwargs)¶

Create a concrete Dtype using the values provided.

If a Dtype has been defined in terms of expressions for its size or number of items then this method can return a concrete Dtype instance. If the Dtype does not contain any expressions then this method will just return it unchanged.

concrete = Dtype('u32')
e1 = Dtype('u{my_size}')
e2 = Dtype('[u8; {my_items}]')

assert e1.evaluate(my_size=32) == concrete
assert e2.evaluate(my_items=10).bit_length == 80

Return type:: Self

classmethod from_params(kind, size=None, endianness=Endianness.UNSPECIFIED)¶

Create a new Dtype from its kind and size.

It’s usually clearer to use the Dtype constructor directly with a dtype str, but this builder will be more efficient and is used internally to avoid string parsing.

Return type:: Self

has_dynamic_size()¶

Return whether the dtype can stretch to fit the available data.

Dynamically sized data types can be used to consume all available data. They can only be used as the final part of compound data types or field types.

>>> d = Dtype('u')
>>> d.has_dynamic_size()
True
>>> d.unpack('0b1')
1
>>> d.unpack('0x00001')
1

Return type:: bool

has_known_size()¶

Return whether the size of the dtype is fully known.

This will be True if the dtype has a known length that doesn’t depend on any parameters or available data, otherwise it will be False.

This will return False for data types with dynamic sizes, and for those whose size depends on an expression without a known value.

>>> Dtype('u32').has_known_size()
True
>>> Dtype('[f16; 4]').has_known_size()
True
>>> Dtype('[u32;]').has_known_size()
False
>>> Dtype('u{x}').has_known_size()
False

Return type:: bool

info()¶

Return a descriptive string with information about the Dtype.

Note that the output is designed to be helpful to users and is not considered part of the API. You should not use the output programmatically as it may change even between point versions.

Return type:: str

pack(value, /)¶

Create and return a new Bits from a value.

The value parameter should be of a type appropriate to the data type.

Return type:: Bits

property endianness: Endianness¶: The endianness of the data type.

property kind: DtypeKind¶

property size: Expression¶

The size of the data type as an Expression.

This is the number used immediately after the data type kind in the dtype string. For example 'u10' has a size of 10.

See also bit_length.

class DtypeArray(s: str | None = None, /)¶

Bases: Dtype

evaluate(**kwargs)¶

Create a concrete Dtype using the values provided.

If a Dtype has been defined in terms of expressions for its size or number of items then this method can return a concrete Dtype instance. If the Dtype does not contain any expressions then this method will just return it unchanged.

concrete = Dtype('u32')
e1 = Dtype('u{my_size}')
e2 = Dtype('[u8; {my_items}]')

assert e1.evaluate(my_size=32) == concrete
assert e2.evaluate(my_items=10).bit_length == 80

Return type:: Self

classmethod from_params(kind, size, items=Expression('None'), endianness=Endianness.UNSPECIFIED)¶

Create a new Dtype from its kind, size and items.

It’s usually clearer to use the Dtype constructor directly with a dtype str, but this builder will be more efficient and is used internally to avoid string parsing.

Return type:: Self

has_dynamic_size()¶

Return whether the dtype can stretch to fit the available data.

Dynamically sized data types can be used to consume all available data. They can only be used as the final part of compound data types or field types.

>>> d = Dtype('u')
>>> d.has_dynamic_size()
True
>>> d.unpack('0b1')
1
>>> d.unpack('0x00001')
1

Return type:: bool

has_known_size()¶

Return whether the size of the dtype is fully known.

This will be True if the dtype has a known length that doesn’t depend on any parameters or available data, otherwise it will be False.

This will return False for data types with dynamic sizes, and for those whose size depends on an expression without a known value.

>>> Dtype('u32').has_known_size()
True
>>> Dtype('[f16; 4]').has_known_size()
True
>>> Dtype('[u32;]').has_known_size()
False
>>> Dtype('u{x}').has_known_size()
False

Return type:: bool

info()¶

Return a descriptive string with information about the Dtype.

Note that the output is designed to be helpful to users and is not considered part of the API. You should not use the output programmatically as it may change even between point versions.

Return type:: str

pack(value, /)¶

Create and return a new Bits from a value.

The value parameter should be of a type appropriate to the data type.

Return type:: Bits

property endianness: Endianness¶: The endianness of the data type stored in the array.

property items: Expression | None | Any¶

The number of items in the data type as an Expression.

For example '[u10; 5]' has 5 items.

An items equal to 0 means it’s an array data type but with items currently unset, while if items is None it is open ended and will consume as many items as possible.

If the number of items is an Expression rather than a constant value then the Expression will be returned.

property kind: DtypeKind¶

property size: Expression¶

The size of the data type as an Expression.

This is the number used immediately after the data type kind in the dtype string. For example '[u10; 5]' has a size of 10.