ImmutableMemory#

class bytesparse.base.ImmutableMemory(start=None, endex=None)[source]#

Immutable virtual memory.

This class is a handy wrapper around blocks, so that it can behave mostly like a bytes, but on sparse chunks of data.

Being immutable, only getters and queries can be performed against instances of this class.

Please look at examples of each method to get a glimpse of the features of this class.

Parameters:

  • start (int) – Optional memory start address. Anything before will be deleted.

  • endex (int) – Optional memory exclusive end address. Anything at or after it will be deleted.

Examples

>>> from bytesparse import Memory

>>> memory = Memory()
>>> memory.to_blocks()
[]

>>> memory = Memory(start=3, endex=10)
>>> memory.bound_span
(3, 10)
>>> memory.write(0, b'Hello, World!')
>>> memory.to_blocks()
[[3, b'lo, Wor']]

>>> memory = Memory.from_bytes(b'Hello, World!', offset=5)
>>> memory.to_blocks()
[[5, b'Hello, World!']]
Method Groups:

Attributes

bound_endex

Bounds exclusive end address.

bound_span

Bounds span addresses.

bound_start

Bounds start address.

content_endex

Exclusive content end address.

content_endin

Inclusive content end address.

content_parts

Number of blocks.

content_size

Actual content size.

content_span

Memory content address span.

content_start

Inclusive content start address.

contiguous

Contains contiguous data.

endex

Exclusive end address.

endin

Inclusive end address.

span

Memory address span.

start

Inclusive start address.

Methods

__init__

block_span

Span of block data.

blocks

Iterates over blocks.

bound

Bounds addresses.

chop

Iterates over chopped blocks.

collapse_blocks

Collapses a generic sequence of blocks.

content_blocks

Iterates over blocks.

content_items

Iterates over content address and value pairs.

content_keys

Iterates over content addresses.

content_values

Iterates over content values.

copy

Creates a deep copy.

count

Counts items.

equal_span

Span of homogeneous data.

extract

Selects items from a range.

find

Index of an item.

from_blocks

Creates a virtual memory from blocks.

from_bytes

Creates a virtual memory from a byte-like chunk.

from_items

Creates a virtual memory from a iterable address/byte mapping.

from_memory

Creates a virtual memory from another one.

from_values

Creates a virtual memory from a byte-like sequence.

fromhex

Creates a virtual memory from an hexadecimal string.

gaps

Iterates over block gaps.

get

Gets the item at an address.

hex

Converts into an hexadecimal string.

hexdump

Textual hex dump.

index

Index of an item.

intervals

Iterates over block intervals.

items

Iterates over address and value pairs.

keys

Iterates over addresses.

peek

Gets the item at an address.

read

Reads data.

readinto

Reads data into a pre-allocated buffer.

rfind

Index of an item, reversed search.

rindex

Index of an item, reversed search.

rvalues

Iterates over values, reversed order.

to_blocks

Exports into blocks.

to_bytes

Exports into bytes.

validate

Validates internal structure.

values

Iterates over values.

view

Creates a view over a range.
abstract __add__(value)[source]#

Concatenates items.

Equivalent to self.copy() += items of a MutableMemory.

See also

MutableMemory.__iadd__()

Examples

>>> from bytesparse import Memory

>>> memory1 = Memory.from_bytes(b'ABC')
>>> memory2 = memory1 + b'xyz'
>>> memory2.to_blocks()
[[0, b'ABCxyz']]

>>> memory1 = Memory.from_blocks([[1, b'ABC']])
>>> memory2 = Memory.from_blocks([[5, b'xyz']])
>>> memory1.content_endex
4
>>> memory3 = memory1 + memory2
>>> memory3.to_blocks()
[[1, b'ABC'], [9, b'xyz']]
abstract __bool__()[source]#

Has any items.

Returns:

bool – Has any items.

Examples

>>> from bytesparse import Memory

>>> memory = Memory()
>>> bool(memory)
False

>>> memory = Memory.from_bytes(b'Hello, World!', offset=5)
>>> bool(memory)
True
abstract __bytes__()[source]#

Creates a bytes clone.

Returns:

bytes – Cloned data.

Raises:

ValueError – Data not contiguous (see contiguous).

Examples

>>> from bytesparse import Memory

>>> memory = Memory()
>>> bytes(memory)
b''

>>> memory = Memory.from_bytes(b'Hello, World!', offset=5)
>>> bytes(memory)
b'Hello, World!'

>>> memory = Memory.from_bytes(b'Hello, World!', offset=5, start=1, endex=20)
>>> bytes(memory)
Traceback (most recent call last):
    ...
ValueError: non-contiguous data within range

>>> memory = Memory.from_blocks([[5, b'ABC'], [9, b'xyz']])
>>> bytes(memory)
Traceback (most recent call last):
    ...
ValueError: non-contiguous data within range
classmethod __class_getitem__()#

Represent a PEP 585 generic type

E.g. for t = list[int], t.__origin__ is list and t.__args__ is (int,).

abstract __contains__(item)[source]#

Checks if some items are contained.

Parameters:

item (items) – Items to find. Can be either some byte string or an integer.

Returns:

bool – Item is contained.

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

9

10

11

[A

B

C]

[1

2

3]

[x

y

z]

 
>>> memory = Memory.from_blocks([[1, b'ABC'], [5, b'123'], [9, b'xyz']])
>>> b'23' in memory
True
>>> ord('y') in memory
True
>>> b'$' in memory
False
abstract __copy__()[source]#

Creates a shallow copy.

Returns:

ImmutableMemory – Shallow copy.

abstract __deepcopy__()[source]#

Creates a deep copy.

Returns:

ImmutableMemory – Deep copy.

abstract __eq__(other)[source]#

Equality comparison.

Parameters:

other (Memory) –

Data to compare with self.

If it is a ImmutableMemory, all of its blocks must match.

If it is a bytes, a bytearray, or a memoryview, it is expected to match the first and only stored block.

Otherwise, it must match the first and only stored block, via iteration over the stored values.

Returns:

bool – self is equal to other.

Examples

>>> from bytesparse import Memory

>>> data = b'Hello, World!'
>>> memory = Memory.from_bytes(data)
>>> memory == data
True
>>> memory.shift(1)
>>> memory == data
True

>>> data = b'Hello, World!'
>>> memory = Memory.from_bytes(data)
>>> memory == list(data)
True
>>> memory.shift(1)
>>> memory == list(data)
True
abstract __getitem__(key)[source]#

Gets data.

Parameters:

key (slice or int) – Selection range or address. If it is a slice with bytes-like step, the latter is interpreted as the filling pattern.

Returns:

items – Items from the requested range.

Note

This method is typically not optimized for a slice where its step is an integer greater than 1.

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

9

10

[A

B

C

D]

[$]

[x

y

z]

65

66

67

68

36

120

121

122

 
>>> memory = Memory.from_blocks([[1, b'ABCD'], [6, b'$'], [8, b'xyz']])
>>> memory[9]  # -> ord('y') = 121
121
>>> memory[:3]._blocks
[[1, b'AB']]
>>> memory[3:10]._blocks
[[3, b'CD'], [6, b'$'], [8, b'xy']]
>>> bytes(memory[3:10:b'.'])
b'CD.$.xy'
>>> memory[memory.endex]
None
>>> bytes(memory[3:10:3])
b'C$y'
>>> memory[3:10:2]._blocks
[[3, b'C'], [6, b'y']]
>>> bytes(memory[3:10:2])
Traceback (most recent call last):
    ...
ValueError: non-contiguous data within range
__hash__ = None#
abstract __init__(start=None, endex=None)[source]#
abstract __iter__()[source]#

Iterates over values.

Iterates over values between start and endex.

Yields:

int – Value as byte integer, or None.

Examples

>>> from bytesparse import Memory

>>> memory = Memory.from_blocks([[5, b'ABC'], [9, b'xyz']])
>>> list(memory)
[65, 66, 67, None, 120, 121, 122]
abstract __len__()[source]#

Actual length.

Computes the actual length of the stored items, i.e. (endex - start). This will consider any bounds being active.

Returns:

int – Memory length.

Examples

>>> from bytesparse import Memory

>>> memory = Memory()
>>> len(memory)
0

>>> memory = Memory(start=3, endex=10)
>>> len(memory)
7

>>> memory = Memory.from_blocks([[1, b'ABC'], [9, b'xyz']])
>>> len(memory)
11

>>> memory = Memory.from_blocks([[3, b'ABC'], [9, b'xyz']], start=1, endex=15)
>>> len(memory)
14
abstract __mul__(times)[source]#

Concatenates a repeated copy.

Equivalent to self.copy() *= items of a MutableMemory.

See also

MutableMemory.__imul__()

Examples

>>> from bytesparse import Memory

>>> memory1 = Memory.from_bytes(b'ABC', offset=2)
>>> memory2 = memory1 * 3
>>> memory2.to_blocks()
[[0, b'ABCABCABC']]

>>> memory1 = Memory.from_blocks([[1, b'ABC']])
>>> memory2 = memory1 * 3
>>> memory2.to_blocks()
[[1, b'ABCABCABC']]
abstract __or__(value)[source]#

Merges memories.

Equivalent to self.copy() |= items of a MutableMemory.

See also

MutableMemory.__ior__()

Examples

>>> from bytesparse import Memory

>>> memory1 = Memory.from_blocks([[1, b'ABC']])
>>> memory2 = Memory.from_blocks([[5, b'xyz']])
>>> memory3 = memory1 | memory2
>>> memory3.to_blocks()
[[1, b'ABC'], [5, b'xyz']]

>>> memory1 = Memory.from_bytes(b'ABC', offset=2)
>>> memory2 = memory1 | b'xyz'
>>> memory2.to_blocks()
[[0, b'xyzBC']]
abstract __repr__()[source]#

Return repr(self).

abstract __reversed__()[source]#

Iterates over values, reversed order.

Iterates over values between start and endex, in reversed order.

Yields:

int – Value as byte integer, or None.

Examples

>>> from bytesparse import Memory

>>> memory = Memory.from_blocks([[5, b'ABC'], [9, b'xyz']])
>>> list(memory)
[65, 66, 67, None, 120, 121, 122]
>>> list(reversed(memory))
[122, 121, 120, None, 67, 66, 65]
abstract __str__()[source]#

String representation.

If content_size is lesser than STR_MAX_CONTENT_SIZE, then the memory is represented as a list of blocks.

If exceeding, it is equivalent to __repr__().

Returns:

str – String representation.

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

9

10

[A

B

C]

[x

y

z]

 
>>> memory = Memory.from_blocks([[1, b'ABC'], [7, b'xyz']])
>>> str(memory)
<[[1, b'ABC'], [7, b'xyz']]>
classmethod __subclasshook__(C)#

Abstract classes can override this to customize issubclass().

This is invoked early on by abc.ABCMeta.__subclasscheck__(). It should return True, False or NotImplemented. If it returns NotImplemented, the normal algorithm is used. Otherwise, it overrides the normal algorithm (and the outcome is cached).

__weakref__#

list of weak references to the object (if defined)

abstract _block_index_at(address)[source]#

Locates the block enclosing an address.

Returns the index of the block enclosing the given address.

Parameters:

address (int) – Address of the target item.

Returns:

int – Block index if found, None otherwise.

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

9

10

11

[A

B

C

D]

[$]

[x

y

z]

0

0

0

0

1

2

2

2

 
>>> memory = Memory.from_blocks([[1, b'ABCD'], [6, b'$'], [8, b'xyz']])
>>> [memory._block_index_at(i) for i in range(12)]
[None, 0, 0, 0, 0, None, 1, None, 2, 2, 2, None]
abstract _block_index_endex(address)[source]#

Locates the last block before an address range.

Returns the index of the last block whose end address is lesser than or equal to address.

Useful to find the termination block index in a ranged search.

Parameters:

address (int) – Exclusive end address of the scanned range.

Returns:

int – First block index before address.

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

9

10

11

[A

B

C

D]

[$]

[x

y

z]

0

1

1

1

1

1

2

2

3

3

3

3

 
>>> memory = Memory.from_blocks([[1, b'ABCD'], [6, b'$'], [8, b'xyz']])
>>> [memory._block_index_endex(i) for i in range(12)]
[0, 1, 1, 1, 1, 1, 2, 2, 3, 3, 3, 3]
abstract _block_index_start(address)[source]#

Locates the first block inside an address range.

Returns the index of the first block whose start address is greater than or equal to address.

Useful to find the initial block index in a ranged search.

Parameters:

address (int) – Inclusive start address of the scanned range.

Returns:

int – First block index since address.

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

9

10

11

[A

B

C

D]

[$]

[x

y

z]

0

0

0

0

0

1

1

2

2

2

2

3

 
>>> memory = Memory.from_blocks([[1, b'ABCD'], [6, b'$'], [8, b'xyz']])
>>> [memory._block_index_start(i) for i in range(12)]
[0, 0, 0, 0, 0, 1, 1, 2, 2, 2, 2, 3]
abstract block_span(address)[source]#

Span of block data.

It searches for the biggest chunk of data adjacent to the given address.

If the address is within a gap, its bounds are returned, and its value is None.

If the address is before or after any data, bounds are None.

Parameters:

address (int) – Reference address.

Returns:

tuple – Start bound, exclusive end bound, and reference value.

Examples

>>> from bytesparse import Memory

>>> memory = Memory()
>>> memory.block_span(0)
(None, None, None)

~~~

0

1

2

3

4

5

6

7

8

9

10

[A

B

B

B

C]

[C

C

D]

65

66

66

66

67

67

67

68

 
>>> memory = Memory.from_blocks([[0, b'ABBBC'], [7, b'CCD']])
>>> memory.block_span(2)
(0, 5, 66)
>>> memory.block_span(4)
(0, 5, 67)
>>> memory.block_span(5)
(5, 7, None)
>>> memory.block_span(10)
(10, None, None)
abstract blocks(start=None, endex=None)[source]#

Iterates over blocks.

Iterates over data blocks within an address range.

Parameters:

  • start (int) – Inclusive start address. If None, start is considered.

  • endex (int) – Exclusive end address. If None, endex is considered.

Yields:

(start, memoryview) – Start and data view of each block/slice.

See also

intervals() to_blocks()

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

9

10

[A

B]

[x]

[1

2

3]

 
>>> memory = Memory.from_blocks([[1, b'AB'], [5, b'x'], [7, b'123']])
>>> [[s, bytes(d)] for s, d in memory.blocks()]
[[1, b'AB'], [5, b'x'], [7, b'123']]
>>> [[s, bytes(d)] for s, d in memory.blocks(2, 9)]
[[2, b'B'], [5, b'x'], [7, b'12']]
>>> [[s, bytes(d)] for s, d in memory.blocks(3, 5)]
[]
abstract bound(start, endex)[source]#

Bounds addresses.

It bounds the given addresses to stay within memory limits. None is used to ignore a limit for the start or endex directions.

In case of stored data, content_start and content_endex are used as bounds.

In case of bounds limits, bound_start or bound_endex are used as bounds, when not None.

In case start and endex are in the wrong order, one clamps the other if present (see the Python implementation for details).

Returns:

tuple of int – Bounded start and endex, closed interval.

Examples

>>> from bytesparse import Memory

>>> Memory().bound(None, None)
(0, 0)
>>> Memory().bound(None, 100)
(0, 100)

~~~

0

1

2

3

4

5

6

7

8

[A

B

C]

[x

y

z]

 
>>> memory = Memory.from_blocks([[1, b'ABC'], [5, b'xyz']])
>>> memory.bound(0, 30)
(0, 30)
>>> memory.bound(2, 6)
(2, 6)
>>> memory.bound(None, 6)
(1, 6)
>>> memory.bound(2, None)
(2, 8)

~~~

0

1

2

3

4

5

6

7

8

[[[

[A

B

C]

)))

 
>>> memory = Memory.from_blocks([[3, b'ABC']], start=1, endex=8)
>>> memory.bound(None, None)
(1, 8)
>>> memory.bound(0, 30)
(1, 8)
>>> memory.bound(2, 6)
(2, 6)
>>> memory.bound(2, None)
(2, 8)
>>> memory.bound(None, 6)
(1, 6)
abstract property bound_endex: int | None#

Bounds exclusive end address.

Any data at or after this address is automatically discarded. Disabled if None.

Examples

>>> from bytesparse import Memory

>>> memory = Memory.from_bytes(b'Hello, World!', offset=5)
>>> memory.bound_endex = 10
>>> memory.to_blocks()
[[5, b'Hello']]

>>> memory = Memory.from_bytes(b'Hello, World!', offset=5, endex=10)
>>> memory.to_blocks()
[[5, b'Hello']]
Type:

int

abstract property bound_span: Tuple[int | None, int | None]#

Bounds span addresses.

A tuple holding bound_start and bound_endex.

Notes

Assigning None to MutableMemory.bound_span sets both bound_start and bound_endex to None (equivalent to (None, None)).

Examples

>>> from bytesparse import Memory

>>> memory = Memory.from_bytes(b'Hello, World!', offset=5)
>>> memory.bound_span = (7, 13)
>>> memory.to_blocks()
[[7, b'llo, W']]
>>> memory.bound_span = None
>>> memory.bound_span
(None, None)

>>> memory = Memory.from_bytes(b'Hello, World!', offset=5, start=7, endex=13)
>>> memory.to_blocks()
[[7, b'llo, W']]
Type:

tuple of int

abstract property bound_start: int | None#

Bounds start address.

Any data before this address is automatically discarded. Disabled if None.

Examples

>>> from bytesparse import Memory

>>> memory = Memory.from_bytes(b'Hello, World!', offset=5)
>>> memory.bound_start = 10
>>> memory.to_blocks()
[[10, b', World!']]

>>> memory = Memory.from_bytes(b'Hello, World!', offset=5, start=10)
>>> memory.to_blocks()
[[10, b', World!']]
Type:

int

abstract chop(width, start=None, endex=None, align=False)[source]#

Iterates over chopped blocks.

The provided range is split into sub-ranges of a fixed width. For each sub-range, it yields views of the contained block chunks.

Parameters:

  • width (int) – Sub-range width.

  • start (int) – Inclusive start address. If None, start is considered.

  • endex (int) – Exclusive end address. If None, endex is considered.

  • align (bool) – Sub-ranges are aligned to width.

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

9

[A

B

C]

[x

y

z]

[A

B]

[C]

[x

y]

[z]

[A]

[B

C]

[x

y]

[z]

 
>>> memory = Memory.from_blocks([[1, b'ABC'], [6, b'xyz']])
>>> chopping = memory.chop(2, align=False)
>>> [(address, bytes(view)) for address, view in chopping]
[(1, b'AB'), (3, b'C'), (6, b'xy'), (8, b'z')]
>>> chopping = memory.chop(2, align=True)
>>> [(address, bytes(view)) for address, view in chopping]
[(1, b'A'), (2, b'BC'), (6, b'xy'), (8, b'z')]
abstract classmethod collapse_blocks(blocks)[source]#

Collapses a generic sequence of blocks.

Given a generic sequence of blocks, writes them in the same order, generating a new sequence of non-contiguous blocks, sorted by address.

Parameters:

blocks (sequence of blocks) – Sequence of blocks to collapse.

Returns:

list of blocks – Collapsed block list.

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

9

[0

1

2

3

4

5

6

7

8

9]

[A

B

C

D]

[E

F]

[$]

[x

y

z]

[$

B

C

E

F

5

x

y

z

9]

 
>>> blocks = [
...     [0, b'0123456789'],
...     [0, b'ABCD'],
...     [3, b'EF'],
...     [0, b'$'],
...     [6, b'xyz'],
... ]
>>> Memory.collapse_blocks(blocks)
[[0, b'$BCEF5xyz9']]

~~~

0

1

2

3

4

5

6

7

8

9

[0

1

2]

[A

B]

[x

y

z]

[$]

[0

$

2]

[A

B

x

y

z]

 
>>> blocks = [
...     [0, b'012'],
...     [4, b'AB'],
...     [6, b'xyz'],
...     [1, b'$'],
... ]
>>> Memory.collapse_blocks(blocks)
[[0, b'0$2'], [4, b'ABxyz']]
abstract content_blocks(block_index_start=None, block_index_endex=None, block_index_step=None)[source]#

Iterates over blocks.

Iterates over data blocks within a block index range.

Parameters:

  • block_index_start (int) – Inclusive block start index. A negative index is referred to content_parts. If None, 0 is considered.

  • block_index_endex (int) – Exclusive block end index. A negative index is referred to content_parts. If None, content_parts is considered.

  • block_index_step (int) – Block index step, which can be negative. If None, 1 is considered.

Yields:

(start, memoryview) – Start and data view of each block/slice.

See also

content_parts

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

9

10

[A

B]

[x]

[1

2

3]

 
>>> memory = Memory.from_blocks([[1, b'AB'], [5, b'x'], [7, b'123']])
>>> [[s, bytes(d)] for s, d in memory.content_blocks()]
[[1, b'AB'], [5, b'x'], [7, b'123']]
>>> [[s, bytes(d)] for s, d in memory.content_blocks(1, 2)]
[[5, b'x']]
>>> [[s, bytes(d)] for s, d in memory.content_blocks(3, 5)]
[]
>>> [[s, bytes(d)] for s, d in memory.content_blocks(block_index_start=-2)]
[[5, b'x'], [7, b'123']]
>>> [[s, bytes(d)] for s, d in memory.content_blocks(block_index_endex=-1)]
[[1, b'AB'], [5, b'x']]
>>> [[s, bytes(d)] for s, d in memory.content_blocks(block_index_step=2)]
[[1, b'AB'], [7, b'123']]
abstract property content_endex: int#

Exclusive content end address.

This property holds the exclusive end address of the memory content. By default, it is the current maximmum exclusive end address of the last stored block.

If the memory has no data and no bounds, start is returned.

Bounds considered only for an empty memory.

Examples

>>> from bytesparse import Memory

>>> Memory().content_endex
0
>>> Memory(endex=8).content_endex
0
>>> Memory(start=1, endex=8).content_endex
1

~~~

0

1

2

3

4

5

6

7

8

[A

B

C]

[x

y

z]

 
>>> memory = Memory.from_blocks([[1, b'ABC'], [5, b'xyz']])
>>> memory.content_endex
8

~~~

0

1

2

3

4

5

6

7

8

[A

B

C]

)))

 
>>> memory = Memory.from_blocks([[1, b'ABC']], endex=8)
>>> memory.content_endex
4
Type:

int

abstract property content_endin: int#

Inclusive content end address.

This property holds the inclusive end address of the memory content. By default, it is the current maximmum inclusive end address of the last stored block.

If the memory has no data and no bounds, start minus one is returned.

Bounds considered only for an empty memory.

Examples

>>> from bytesparse import Memory

>>> Memory().content_endin
-1
>>> Memory(endex=8).content_endin
-1
>>> Memory(start=1, endex=8).content_endin
0

~~~

0

1

2

3

4

5

6

7

8

[A

B

C]

[x

y

z]

 
>>> memory = Memory.from_blocks([[1, b'ABC'], [5, b'xyz']])
>>> memory.content_endin
7

~~~

0

1

2

3

4

5

6

7

8

[A

B

C]

)))

 
>>> memory = Memory.from_blocks([[1, b'ABC']], endex=8)
>>> memory.content_endin
3
Type:

int

abstract content_items(start=None, endex=None)[source]#

Iterates over content address and value pairs.

Parameters:

  • start (int) – Inclusive start address. If None, start is considered.

  • endex (int) – Exclusive end address. If None, endex is considered.

Yields:

int – Content address and value pairs.

See also

meth:content_keys meth:content_values

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

9

10

[A

B]

[x]

[1

2

3]

 
>>> memory = Memory.from_blocks([[1, b'AB'], [5, b'x'], [7, b'123']])
>>> dict(memory.content_items())
{1: 65, 2: 66, 5: 120, 7: 49, 8: 50, 9: 51}
>>> dict(memory.content_items(2, 9))
{2: 66, 5: 120, 7: 49, 8: 50}
>>> dict(memory.content_items(3, 5))
{}
abstract content_keys(start=None, endex=None)[source]#

Iterates over content addresses.

Parameters:

  • start (int) – Inclusive start address. If None, start is considered.

  • endex (int) – Exclusive end address. If None, endex is considered.

Yields:

int – Content addresses.

See also

meth:content_items meth:content_values

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

9

10

[A

B]

[x]

[1

2

3]

 
>>> memory = Memory.from_blocks([[1, b'AB'], [5, b'x'], [7, b'123']])
>>> list(memory.content_keys())
[1, 2, 5, 7, 8, 9]
>>> list(memory.content_keys(2, 9))
[2, 5, 7, 8]
>>> list(memory.content_keys(3, 5))
[]
abstract property content_parts: int#

Number of blocks.

Returns:

int – The number of blocks.

Examples

>>> from bytesparse import Memory

>>> Memory().content_parts
0

~~~

0

1

2

3

4

5

6

7

8

[A

B

C]

[x

y

z]

 
>>> memory = Memory.from_blocks([[1, b'ABC'], [5, b'xyz']])
>>> memory.content_parts
2

~~~

0

1

2

3

4

5

6

7

8

[A

B

C]

)))

 
>>> memory = Memory.from_blocks([[1, b'ABC']], endex=8)
>>> memory.content_parts
1
abstract property content_size: int#

Actual content size.

Returns:

int – The sum of all block lengths.

Examples

>>> from bytesparse import Memory

>>> Memory().content_size
0
>>> Memory(start=1, endex=8).content_size
0

~~~

0

1

2

3

4

5

6

7

8

[A

B

C]

[x

y

z]

 
>>> memory = Memory.from_blocks([[1, b'ABC'], [5, b'xyz']])
>>> memory.content_size
6

~~~

0

1

2

3

4

5

6

7

8

[A

B

C]

)))

 
>>> memory = Memory.from_blocks([[1, b'ABC']], endex=8)
>>> memory.content_size
3
abstract property content_span: Tuple[int, int]#

Memory content address span.

A tuple holding both content_start and content_endex.

Examples

>>> from bytesparse import Memory

>>> Memory().content_span
(0, 0)
>>> Memory(start=1).content_span
(1, 1)
>>> Memory(endex=8).content_span
(0, 0)
>>> Memory(start=1, endex=8).content_span
(1, 1)

~~~

0

1

2

3

4

5

6

7

8

[A

B

C]

[x

y

z]

 
>>> memory = Memory.from_blocks([[1, b'ABC'], [5, b'xyz']])
>>> memory.content_span
(1, 8)
Type:

tuple of int

abstract property content_start: int#

Inclusive content start address.

This property holds the inclusive start address of the memory content. By default, it is the current minimum inclusive start address of the first stored block.

If the memory has no data and no bounds, 0 is returned.

Bounds considered only for an empty memory.

Examples

>>> from bytesparse import Memory

>>> Memory().content_start
0
>>> Memory(start=1).content_start
1
>>> Memory(start=1, endex=8).content_start
1

~~~

0

1

2

3

4

5

6

7

8

[A

B

C]

[x

y

z]

 
>>> memory = Memory.from_blocks([[1, b'ABC'], [5, b'xyz']])
>>> memory.content_start
1

~~~

0

1

2

3

4

5

6

7

8

[[[

[x

y

z]

 
>>> memory = Memory.from_blocks([[5, b'xyz']], start=1)
>>> memory.content_start
5
Type:

int

abstract content_values(start=None, endex=None)[source]#

Iterates over content values.

Parameters:

  • start (int) – Inclusive start address. If None, start is considered.

  • endex (int) – Exclusive end address. If None, endex is considered.

Yields:

int – Content values.

See also

meth:content_items meth:content_keys

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

9

10

[A

B]

[x]

[1

2

3]

 
>>> memory = Memory.from_blocks([[1, b'AB'], [5, b'x'], [7, b'123']])
>>> list(memory.content_values())
[65, 66, 120, 49, 50, 51]
>>> list(memory.content_values(2, 9))
[66, 120, 49, 50]
>>> list(memory.content_values(3, 5))
[]
abstract property contiguous: bool#

Contains contiguous data.

The memory is considered to have contiguous data if there is no empty space between blocks.

If bounds are defined, there must be no empty space also towards them.

Examples

>>> from bytesparse import Memory

>>> memory = Memory()
>>> memory.contiguous
True

>>> memory = Memory.from_bytes(b'Hello, World!', offset=5)
>>> memory.contiguous
True

>>> memory = Memory.from_bytes(b'Hello, World!', offset=5, start=1, endex=20)
>>> memory.contiguous
False

>>> memory = Memory.from_blocks([[5, b'ABC'], [9, b'xyz']])
>>> memory.contiguous
False
Type:

bool

abstract copy()[source]#

Creates a deep copy.

Returns:

ImmutableMemory – Deep copy.

Examples

>>> from bytesparse import Memory

>>> memory1 = Memory()
>>> memory2 = memory1.copy()
>>> memory2.bound_span
(None, None)
>>> memory2.to_blocks()
[]

>>> memory1 = Memory(start=1, endex=20)
>>> memory2 = memory1.copy()
>>> memory2.bound_span
(1, 20)
>>> memory2.to_blocks()
[]

>>> memory1 = Memory.from_bytes(b'Hello, World!', offset=5)
>>> memory2 = memory1.copy()
>>> memory2.to_blocks()
[[5, b'Hello, World!']]

>>> memory1 = Memory.from_bytes(b'Hello, World!', offset=5, start=1, endex=20)
>>> memory2 = memory1.copy()
>>> memory2.bound_span
(1, 20)
>>> memory2.to_blocks()
[[5, b'Hello, World!']]
>>> memory2.bound_span = (2, 19)
>>> memory1 == memory2
True

>>> memory1 = Memory.from_blocks([[5, b'ABC'], [9, b'xyz']])
>>> memory2 = memory1.copy()
[[5, b'ABC'], [9, b'xyz']]
>>> memory1 == memory2
True
abstract count(item, start=None, endex=None)[source]#

Counts items.

Parameters:

  • item (items) – Reference value to count.

  • start (int) – Inclusive start of the searched range. If None, start is considered.

  • endex (int) – Exclusive end of the searched range. If None, endex is considered.

Returns:

int – The number of items equal to value.

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

9

10

11

[A

B

C]

[B

a

t]

[t

a

b]

 
>>> memory = Memory.from_blocks([[1, b'ABC'], [5, b'Bat'], [9, b'tab']])
>>> memory.count(b'a')
2
abstract property endex: int#

Exclusive end address.

This property holds the exclusive end address of the virtual space. By default, it is the current maximmum exclusive end address of the last stored block.

If bound_endex not None, that is returned.

If the memory has no data and no bounds, start is returned.

Examples

>>> from bytesparse import Memory

>>> Memory().endex
0

~~~

0

1

2

3

4

5

6

7

8

[A

B

C]

[x

y

z]

 
>>> memory = Memory.from_blocks([[1, b'ABC'], [5, b'xyz']])
>>> memory.endex
8

~~~

0

1

2

3

4

5

6

7

8

[A

B

C]

)))

 
>>> memory = Memory.from_blocks([[1, b'ABC']], endex=8)
>>> memory.endex
8
Type:

int

abstract property endin: int#

Inclusive end address.

This property holds the inclusive end address of the virtual space. By default, it is the current maximmum inclusive end address of the last stored block.

If bound_endex not None, that minus one is returned.

If the memory has no data and no bounds, start is returned.

Examples

>>> from bytesparse import Memory

>>> Memory().endin
-1

~~~

0

1

2

3

4

5

6

7

8

[A

B

C]

[x

y

z]

 
>>> memory = Memory.from_blocks([[1, b'ABC'], [5, b'xyz']])
>>> memory.endin
7

~~~

0

1

2

3

4

5

6

7

8

[A

B

C]

)))

 
>>> memory = Memory.from_blocks([[1, b'ABC']], endex=8)
>>> memory.endin
7
Type:

int

abstract equal_span(address)[source]#

Span of homogeneous data.

It searches for the biggest chunk of data adjacent to the given address, with the same value at that address.

If the address is within a gap, its bounds are returned, and its value is None.

If the address is before or after any data, bounds are None.

Parameters:

address (int) – Reference address.

Returns:

tuple – Start bound, exclusive end bound, and reference value.

Examples

>>> from bytesparse import Memory

>>> memory = Memory()
>>> memory.equal_span(0)
(None, None, None)

~~~

0

1

2

3

4

5

6

7

8

9

10

[A

B

B

B

C]

[C

C

D]

65

66

66

66

67

67

67

68

 
>>> memory = Memory.from_blocks([[0, b'ABBBC'], [7, b'CCD']])
>>> memory.equal_span(2)
(1, 4, 66)
>>> memory.equal_span(4)
(4, 5, 67)
>>> memory.equal_span(5)
(5, 7, None)
>>> memory.equal_span(10)
(10, None, None)
abstract extract(start=None, endex=None, pattern=None, step=None, bound=True)[source]#

Selects items from a range.

Parameters:

  • start (int) – Inclusive start of the extracted range. If None, start is considered.

  • endex (int) – Exclusive end of the extracted range. If None, endex is considered.

  • pattern (items) – Optional pattern of items to fill the emptiness.

  • step (int) – Optional address stepping between bytes extracted from the range. It has the same meaning of Python’s slice.step, but negative steps are ignored. Please note that a step greater than 1 could take much more time to process than the default unitary step.

  • bound (bool) – The selected address range is applied to the resulting memory as its bounds range. This retains information about any initial and final emptiness of that range, which would be lost otherwise.

Returns:

ImmutableMemory – A copy of the memory from the selected range.

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

9

10

11

[A

B

C

D]

[$]

[x

y

z]

 
>>> memory = Memory.from_blocks([[1, b'ABCD'], [6, b'$'], [8, b'xyz']])
>>> memory.extract()._blocks
[[1, b'ABCD'], [6, b'$'], [8, b'xyz']]
>>> memory.extract(2, 9)._blocks
[[2, b'BCD'], [6, b'$'], [8, b'x']]
>>> memory.extract(start=2)._blocks
[[2, b'BCD'], [6, b'$'], [8, b'xyz']]
>>> memory.extract(endex=9)._blocks
[[1, b'ABCD'], [6, b'$'], [8, b'x']]
>>> memory.extract(5, 8).span
(5, 8)
>>> memory.extract(pattern=b'.')._blocks
[[1, b'ABCD.$.xyz']]
>>> memory.extract(pattern=b'.', step=3)._blocks
[[1, b'AD.z']]
abstract find(item, start=None, endex=None)[source]#

Index of an item.

Parameters:

  • item (items) – Value to find. Can be either some byte string or an integer.

  • start (int) – Inclusive start of the searched range. If None, endex is considered.

  • endex (int) – Exclusive end of the searched range. If None, endex is considered.

Returns:

int – The index of the first item equal to value, or -1.

Warning

If the memory allows negative addresses, index() is more appropriate, because it raises ValueError if the item is not found.

See also

index()

abstract classmethod from_blocks(blocks, offset=0, start=None, endex=None, copy=True, validate=True)[source]#

Creates a virtual memory from blocks.

Parameters:

  • blocks (list of blocks) – A sequence of non-overlapping blocks, sorted by address.

  • offset (int) – Some address offset applied to all the blocks.

  • start (int) – Optional memory start address. Anything before will be deleted.

  • endex (int) – Optional memory exclusive end address. Anything at or after it will be deleted.

  • copy (bool) – Forces copy of provided input data.

  • validate (bool) – Validates the resulting ImmutableMemory object.

Returns:

ImmutableMemory – The resulting memory object.

Raises:

ValueError – Some requirements are not satisfied.

See also

to_blocks()

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

[A

B

C]

[x

y

z]

 
>>> blocks = [[1, b'ABC'], [5, b'xyz']]
>>> memory = Memory.from_blocks(blocks)
>>> memory.to_blocks()
[[1, b'ABC'], [5, b'xyz']]
>>> memory = Memory.from_blocks(blocks, offset=3)
>>> memory.to_blocks()
[[4, b'ABC'], [8, b'xyz']]

~~~

>>> # Loads data from an Intel HEX record file
>>> # NOTE: Record files typically require collapsing!
>>> import hexrec.records as hr  # noqa
>>> blocks = hr.load_blocks('records.hex')
>>> memory = Memory.from_blocks(Memory.collapse_blocks(blocks))
>>> memory
    ...
abstract classmethod from_bytes(data, offset=0, start=None, endex=None, copy=True, validate=True)[source]#

Creates a virtual memory from a byte-like chunk.

Parameters:

  • data (byte-like data) – A byte-like chunk of data (e.g. bytes, bytearray, memoryview).

  • offset (int) – Start address of the block of data.

  • start (int) – Optional memory start address. Anything before will be deleted.

  • endex (int) – Optional memory exclusive end address. Anything at or after it will be deleted.

  • copy (bool) – Forces copy of provided input data into the underlying data structure.

  • validate (bool) – Validates the resulting ImmutableMemory object.

Returns:

ImmutableMemory – The resulting memory object.

Raises:

ValueError – Some requirements are not satisfied.

See also

to_bytes()

Examples

>>> from bytesparse import Memory

>>> memory = Memory.from_bytes(b'')
>>> memory.to_blocks()
[]

~~~

0

1

2

3

4

5

6

7

8

[A

B

C

x

y

z]

 
>>> memory = Memory.from_bytes(b'ABCxyz', 2)
>>> memory.to_blocks()
[[2, b'ABCxyz']]
abstract classmethod from_items(items, offset=0, start=None, endex=None, validate=True)[source]#

Creates a virtual memory from a iterable address/byte mapping.

Parameters:

  • items (iterable address/byte mapping) – An iterable mapping of address to byte values. Values of None are translated as gaps. When an address is stated multiple times, the last is kept.

  • offset (int) – An address offset applied to all the values.

  • start (int) – Optional memory start address. Anything before will be deleted.

  • endex (int) – Optional memory exclusive end address. Anything at or after it will be deleted.

  • validate (bool) – Validates the resulting ImmutableMemory object.

Returns:

ImmutableMemory – The resulting memory object.

Raises:

ValueError – Some requirements are not satisfied.

See also

to_bytes()

Examples

>>> from bytesparse import Memory

>>> memory = Memory.from_values({})
>>> memory.to_blocks()
[]

~~~

0

1

2

3

4

5

6

7

8

[A

Z]

[x]

 
>>> items = [
...     (0, ord('A')),
...     (1, ord('B')),
...     (3, ord('x')),
...     (1, ord('Z')),
... ]
>>> memory = Memory.from_items(items, offset=2)
>>> memory.to_blocks()
[[2, b'AZ'], [5, b'x']]
abstract classmethod from_memory(memory, offset=0, start=None, endex=None, copy=True, validate=True)[source]#

Creates a virtual memory from another one.

Parameters:

  • memory (Memory) – A ImmutableMemory to copy data from.

  • offset (int) – Some address offset applied to all the blocks.

  • start (int) – Optional memory start address. Anything before will be deleted.

  • endex (int) – Optional memory exclusive end address. Anything at or after it will be deleted.

  • copy (bool) – Forces copy of provided input data into the underlying data structure.

  • validate (bool) – Validates the resulting MemorImmutableMemory object.

Returns:

ImmutableMemory – The resulting memory object.

Raises:

ValueError – Some requirements are not satisfied.

Examples

>>> from bytesparse import Memory

>>> memory1 = Memory.from_bytes(b'ABC', 5)
>>> memory2 = Memory.from_memory(memory1)
>>> memory2._blocks
[[5, b'ABC']]
>>> memory1 == memory2
True
>>> memory1 is memory2
False
>>> memory1._blocks is memory2._blocks
False

~~~

>>> memory1 = Memory.from_bytes(b'ABC', 10)
>>> memory2 = Memory.from_memory(memory1, -3)
>>> memory2._blocks
[[7, b'ABC']]
>>> memory1 == memory2
False

~~~

>>> memory1 = Memory.from_bytes(b'ABC', 10)
>>> memory2 = Memory.from_memory(memory1, copy=False)
>>> all((b1[1] is b2[1])  # compare block data
...     for b1, b2 in zip(memory1._blocks, memory2._blocks))
True
abstract classmethod from_values(values, offset=0, start=None, endex=None, validate=True)[source]#

Creates a virtual memory from a byte-like sequence.

Parameters:

  • values (iterable byte-like sequence) – An iterable sequence of byte values. Values of None are translated as gaps.

  • offset (int) – An address offset applied to all the values.

  • start (int) – Optional memory start address. Anything before will be deleted.

  • endex (int) – Optional memory exclusive end address. Anything at or after it will be deleted.

  • validate (bool) – Validates the resulting ImmutableMemory object.

Returns:

ImmutableMemory – The resulting memory object.

Raises:

ValueError – Some requirements are not satisfied.

See also

to_bytes()

Examples

>>> from bytesparse import Memory

>>> memory = Memory.from_values(range(0))
>>> memory.to_blocks()
[]

~~~

0

1

2

3

4

5

6

7

8

[A

B

C

D

E]

 
>>> memory = Memory.from_values(range(ord('A'), ord('F')), offset=2)
>>> memory.to_blocks()
[[2, b'ABCDE']]
abstract classmethod fromhex(string)[source]#

Creates a virtual memory from an hexadecimal string.

Parameters:

string (str) – Hexadecimal string.

Returns:

ImmutableMemory – The resulting memory object.

Examples

>>> from bytesparse import Memory

>>> memory = Memory.fromhex('')
>>> bytes(memory)
b''

~~~

>>> memory = Memory.fromhex('48656C6C6F2C20576F726C6421')
>>> bytes(memory)
b'Hello, World!'
abstract gaps(start=None, endex=None)[source]#

Iterates over block gaps.

Iterates over gaps emptiness bounds within an address range. If a yielded bound is None, that direction is infinitely empty (valid before or after global data bounds).

Parameters:

  • start (int) – Inclusive start address. If None, start is considered.

  • endex (int) – Exclusive end address. If None, endex is considered.

Yields:

pair of addresses – Block data interval boundaries.

See also

intervals()

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

9

10

[A

B]

[x]

[1

2

3]

 
>>> memory = Memory.from_blocks([[1, b'AB'], [5, b'x'], [7, b'123']])
>>> list(memory.gaps())
[(None, 1), (3, 5), (6, 7), (10, None)]
>>> list(memory.gaps(0, 11))
[(0, 1), (3, 5), (6, 7), (10, 11)]
>>> list(memory.gaps(*memory.span))
[(3, 5), (6, 7)]
>>> list(memory.gaps(2, 6))
[(3, 5)]
abstract get(address, default=None)[source]#

Gets the item at an address.

Returns:

int – The item at address, default if empty.

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

9

10

11

[A

B

C

D]

[$]

[x

y

z]

 
>>> memory = Memory.from_blocks([[1, b'ABCD'], [6, b'$'], [8, b'xyz']])
>>> memory.get(3)  # -> ord('C') = 67
67
>>> memory.get(6)  # -> ord('$') = 36
36
>>> memory.get(10)  # -> ord('z') = 122
122
>>> memory.get(0)  # -> empty -> default = None
None
>>> memory.get(7)  # -> empty -> default = None
None
>>> memory.get(11)  # -> empty -> default = None
None
>>> memory.get(0, 123)  # -> empty -> default = 123
123
>>> memory.get(7, 123)  # -> empty -> default = 123
123
>>> memory.get(11, 123)  # -> empty -> default = 123
123
abstract hex(*args)[source]#

Converts into an hexadecimal string.

Parameters:

  • sep (str) – Separator string between bytes. Defaults to an emoty string if not provided. Available since Python 3.8.

  • bytes_per_sep (int) – Number of bytes grouped between separators. Defaults to one byte per group. Available since Python 3.8.

Returns:

str – Hexadecimal string representation.

Raises:

ValueError – Data not contiguous (see contiguous).

Examples

>>> from bytesparse import Memory

>>> Memory().hex() == ''
True

~~~

>>> memory = Memory.from_bytes(b'Hello, World!')
>>> memory.hex()
48656c6c6f2c20576f726c6421
>>> memory.hex('.')
48.65.6c.6c.6f.2c.20.57.6f.72.6c.64.21
>>> memory.hex('.', 4)
48.656c6c6f.2c20576f.726c6421
abstract hexdump(start=None, endex=None, columns=16, addrfmt='{:08X} ', bytefmt=' {:02X}', headfmt=None, charmap='................................ !"#$%&\\'()*+, -./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\\\]^_`abcdefghijklmnopqrstuvwxyz{|}~................................................................................................................................. ><', emptystr=' --', beforestr=' >>', afterstr=' <<', charsep='  |', charend='|', stream=Ellipsis)[source]#

Textual hex dump.

This function generates a hex dump of the bytes within the specified range.

If stream is not None, the hex dump is written on it, otherwise it is returned as a str.

The default output is similar to that of hexdump or xxd commands, with some degree of tweaking. In case more customized formatting is desired, a dedicated custom function can be written by carefully looping over values().

Parameters:

  • start (int) – Inclusive start of the searched range. If None, start is considered.

  • endex (int) – Exclusive end of the searched range. If None, endex is considered.

  • columns (int) – Number of byte columns per row.

  • addrfmt (str) – Address formatting string.

  • bytefmt (str) – Byte formatting string.

  • headfmt (str) – Header offset formatting string. If Ellipsis, it applies that of bytefmt. If None, no header row is generated.

  • charmap (mapping) –

    Mapping to convert a byte integer into a string character. If None, no character data are appended to each row.

    The table is structured this way:

    • The initial 256 bytes map actual byte values.

    • Index 0x100 represents an empty byte (None).

    • Index 0x101 represents a byte before start.

    • Index 0x102 represents a byte after endex.

  • emptystr (str) – Placeholder for an empty byte (None value).

  • beforestr (str) – Placeholder for a byte before bound_start.

  • afterstr (str) – Placeholder for a byte after bound_endex.

  • charsep (str) – Separator between byte data and character data.

  • charend (str) – Separator after character data.

  • stream (IO stream) – Stream to write text onto. If Ellipsis, it uses sys.stdout. If not None, the function returns None.

Returns:

str – Textual hex dump, if stream is None.

Examples

>>> from bytesparse import Memory

>>> memory = Memory.from_blocks([[1, b'ABC'], [6, b'xyz']], offset=0xDA7A)
>>> memory.hexdump()
0000DA7B  41 42 43 -- -- 78 79 7A -- -- -- -- -- -- -- --  |ABC  xyz        |
>>> memory.hexdump(stream=None)
'0000DA7B  41 42 43 -- -- 78 79 7A -- -- -- -- -- -- -- --  |ABC  xyz        |'
>>> memory.hexdump(start=0xDA7A, charmap=None)
0000DA7A  -- 41 42 43 -- -- 78 79 7A -- -- -- -- -- -- --
>>> memory.hexdump(start=0xDA7A)
0000DA7A  -- 41 42 43 -- -- 78 79 7A -- -- -- -- -- -- --  | ABC  xyz       |
>>> memory.hexdump(start=0xDA70)
0000DA70  -- -- -- -- -- -- -- -- -- -- -- 41 42 43 -- --  |           ABC  |
0000DA80  78 79 7A -- -- -- -- -- -- -- -- -- -- -- -- --  |xyz             |
>>> memory.bound_span = (0xDA78, 0xDA88)
>>> memory.hexdump(start=0xDA70)
0000DA70  >> >> >> >> >> >> >> >> -- -- -- 41 42 43 -- --  |>>>>>>>>   ABC  |
0000DA80  78 79 7A -- -- -- -- -- << << << << << << << <<  |xyz     <<<<<<<<|
>>> memory.hexdump(start=0xDA70, headfmt=...)
          00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
0000DA70  >> >> >> >> >> >> >> >> -- -- -- 41 42 43 -- --  |>>>>>>>>   ABC  |
0000DA80  78 79 7A -- -- -- -- -- << << << << << << << <<  |xyz     <<<<<<<<|
>>> memory.hexdump(start=0xDA78, endex=0xDA84, columns=4)
0000DA78  -- -- -- 41  |   A|
0000DA7C  42 43 -- --  |BC  |
0000DA80  78 79 7A --  |xyz |
abstract index(item, start=None, endex=None)[source]#

Index of an item.

Parameters:

  • item (items) – Value to find. Can be either some byte string or an integer.

  • start (int) – Inclusive start of the searched range. If None, start is considered.

  • endex (int) – Exclusive end of the searched range. If None, endex is considered.

Returns:

int – The index of the first item equal to value.

Raises:

ValueError – Item not found.

See also

find()

abstract intervals(start=None, endex=None)[source]#

Iterates over block intervals.

Iterates over data boundaries within an address range.

Parameters:

  • start (int) – Inclusive start address. If None, start is considered.

  • endex (int) – Exclusive end address. If None, endex is considered.

Yields:

pair of addresses – Block data interval boundaries.

See also

blocks() gaps()

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

9

10

[A

B]

[x]

[1

2

3]

 
>>> memory = Memory.from_blocks([[1, b'AB'], [5, b'x'], [7, b'123']])
>>> list(memory.intervals())
[(1, 3), (5, 6), (7, 10)]
>>> list(memory.intervals(2, 9))
[(2, 3), (5, 6), (7, 9)]
>>> list(memory.intervals(3, 5))
[]
abstract items(start=None, endex=None, pattern=None)[source]#

Iterates over address and value pairs.

Iterates over address and value pairs, from start to endex. Implemets the interface of dict.

Parameters:

  • start (int) – Inclusive start address. If None, start is considered.

  • endex (int) – Exclusive end address. If None, endex is considered. If Ellipsis, the iterator is infinite.

  • pattern (items) – Pattern of values to fill emptiness.

Yields:

int – Range address and value pairs.

Examples

>>> from bytesparse import Memory

>>> from itertools import islice
>>> memory = Memory()
>>> list(memory.items(endex=8))
[(0, None), (1, None), (2, None), (3, None), (4, None), (5, None), (6, None), (7, None)]
>>> list(memory.items(3, 8))
[(3, None), (4, None), (5, None), (6, None), (7, None)]
>>> list(islice(memory.items(3, ...), 7))
[(3, None), (4, None), (5, None), (6, None), (7, None), (8, None), (9, None)]

~~~

0

1

2

3

4

5

6

7

8

9

[A

B

C]

[x

y

z]

65

66

67

120

121

122

 
>>> memory = Memory.from_blocks([[1, b'ABC'], [6, b'xyz']])
>>> list(memory.items())
[(1, 65), (2, 66), (3, 67), (4, None), (5, None), (6, 120), (7, 121), (8, 122)]
>>> list(memory.items(3, 8))
[(3, 67), (4, None), (5, None), (6, 120), (7, 121)]
>>> list(islice(memory.items(3, ...), 7))
[(3, 67), (4, None), (5, None), (6, 120), (7, 121), (8, 122), (9, None)]
abstract keys(start=None, endex=None)[source]#

Iterates over addresses.

Iterates over addresses, from start to endex. Implemets the interface of dict.

Parameters:

  • start (int) – Inclusive start address. If None, start is considered.

  • endex (int) – Exclusive end address. If None, endex is considered. If Ellipsis, the iterator is infinite.

Yields:

int – Range address.

Examples

>>> from bytesparse import Memory

>>> from itertools import islice
>>> memory = Memory()
>>> list(memory.keys())
[]
>>> list(memory.keys(endex=8))
[0, 1, 2, 3, 4, 5, 6, 7]
>>> list(memory.keys(3, 8))
[3, 4, 5, 6, 7]
>>> list(islice(memory.keys(3, ...), 7))
[3, 4, 5, 6, 7, 8, 9]

~~~

0

1

2

3

4

5

6

7

8

9

[A

B

C]

[x

y

z]

 
>>> memory = Memory.from_blocks([[1, b'ABC'], [6, b'xyz']])
>>> list(memory.keys())
[1, 2, 3, 4, 5, 6, 7, 8]
>>> list(memory.keys(endex=8))
[1, 2, 3, 4, 5, 6, 7]
>>> list(memory.keys(3, 8))
[3, 4, 5, 6, 7]
>>> list(islice(memory.keys(3, ...), 7))
[3, 4, 5, 6, 7, 8, 9]
abstract peek(address)[source]#

Gets the item at an address.

Returns:

int – The item at address, None if empty.

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

9

10

11

[A

B

C

D]

[$]

[x

y

z]

 
>>> memory = Memory.from_blocks([[1, b'ABCD'], [6, b'$'], [8, b'xyz']])
>>> memory.peek(3)  # -> ord('C') = 67
67
>>> memory.peek(6)  # -> ord('$') = 36
36
>>> memory.peek(10)  # -> ord('z') = 122
122
>>> memory.peek(0)
None
>>> memory.peek(7)
None
>>> memory.peek(11)
None
abstract read(address, size)[source]#

Reads data.

Reads a chunk of data from an address, with a given size. Data within the range is required to be contiguous.

Parameters:

  • address (int) – Start address of the chunk to read.

  • size (int) – Chunk size.

Returns:

memoryview – A view over the addressed chunk.

Raises:

ValueError – Data not contiguous (see contiguous).

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

9

10

11

[A

B

C

D]

[$]

[x

y

z]

 
>>> memory = Memory.from_blocks([[1, b'ABCD'], [6, b'$'], [8, b'xyz']])
>>> bytes(memory.read(2, 3))
b'BCD'
>>> bytes(memory.read(9, 1))
b'y'
>>> memory.read(4, 3)
Traceback (most recent call last):
    ...
ValueError: non-contiguous data within range
>>> memory.read(0, 6)
Traceback (most recent call last):
    ...
ValueError: non-contiguous data within range
abstract readinto(address, buffer)[source]#

Reads data into a pre-allocated buffer.

Provided a pre-allocated writable buffer (e.g. a bytearray or a memoryview slice of it), this method reads a chunk of data from an address, with the size of the target buffer. Data within the range is required to be contiguous.

Parameters:

  • address (int) – Start address of the chunk to read.

  • buffer (writable) – Pre-allocated buffer to fill with data.

Returns:

int – Number of bytes read.

Raises:

ValueError – Data not contiguous (see contiguous).

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

9

10

11

[A

B

C

D]

[$]

[x

y

z]

 
>>> memory = Memory.from_blocks([[1, b'ABCD'], [6, b'$'], [8, b'xyz']])
>>> buffer = bytearray(3)
>>> memory.readinto(2, buffer)
3
>>> buffer
bytearray(b'BCD')
>>> view = memoryview(buffer)
>>> memory.readinto(9, view[1:2])
1
>>> buffer
bytearray(b'ByD')
>>> memory.readinto(4, buffer)
Traceback (most recent call last):
    ...
ValueError: non-contiguous data within range
>>> memory.readinto(0, bytearray(6))
Traceback (most recent call last):
    ...
ValueError: non-contiguous data within range
abstract rfind(item, start=None, endex=None)[source]#

Index of an item, reversed search.

Parameters:

  • item (items) – Value to find. Can be either some byte string or an integer.

  • start (int) – Inclusive start of the searched range. If None, start is considered.

  • endex (int) – Exclusive end of the searched range. If None, endex is considered.

Returns:

int – The index of the last item equal to value, or -1.

Warning

If the memory allows negative addresses, rindex() is more appropriate, because it raises ValueError if the item is not found.

See also

rindex()

abstract rindex(item, start=None, endex=None)[source]#

Index of an item, reversed search.

Parameters:

  • item (items) – Value to find. Can be either some byte string or an integer.

  • start (int) – Inclusive start of the searched range. If None, start is considered.

  • endex (int) – Exclusive end of the searched range. If None, endex is considered.

Returns:

int – The index of the last item equal to value.

Raises:

ValueError – Item not found.

Warning

If the memory allows negative addresses, index() is more appropriate, because it raises ValueError if the item is not found.

See also

rfind()

abstract rvalues(start=None, endex=None, pattern=None)[source]#

Iterates over values, reversed order.

Iterates over values, from endex to start.

Parameters:

  • start (int) – Inclusive start address. If None, start is considered. If Ellipsis, the iterator is infinite.

  • endex (int) – Exclusive end address. If None, endex is considered.

  • pattern (items) – Pattern of values to fill emptiness.

Yields:

int – Range values.

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

9

A

B

C

D

A

65

66

67

68

65

 
>>> from itertools import islice
>>> memory = Memory()
>>> list(memory.rvalues(endex=8))
[None, None, None, None, None, None, None, None]
>>> list(memory.rvalues(3, 8))
[None, None, None, None, None]
>>> list(islice(memory.rvalues(..., 8), 7))
[None, None, None, None, None, None, None]
>>> list(memory.rvalues(3, 8, b'ABCD'))
[65, 68, 67, 66, 65]

~~~

0

1

2

3

4

5

6

7

8

9

[A

B

C]

<1

2>

[x

y

z]

65

66

67

120

121

122

65

66

67

49

50

120

121

122

 
>>> memory = Memory.from_blocks([[1, b'ABC'], [6, b'xyz']])
>>> list(memory.rvalues())
[122, 121, 120, None, None, 67, 66, 65]
>>> list(memory.rvalues(3, 8))
[121, 120, None, None, 67]
>>> list(islice(memory.rvalues(..., 8), 7))
[121, 120, None, None, 67, 66, 65]
>>> list(memory.rvalues(3, 8, b'0123'))
[121, 120, 50, 49, 67]
abstract property span: Tuple[int, int]#

Memory address span.

A tuple holding both start and endex.

Examples

>>> from bytesparse import Memory

>>> Memory().span
(0, 0)
>>> Memory(start=1, endex=8).span
(1, 8)

~~~

0

1

2

3

4

5

6

7

8

[A

B

C]

[x

y

z]

 
>>> memory = Memory.from_blocks([[1, b'ABC'], [5, b'xyz']])
>>> memory.span
(1, 8)
Type:

tuple of int

abstract property start: int#

Inclusive start address.

This property holds the inclusive start address of the virtual space. By default, it is the current minimum inclusive start address of the first stored block.

If bound_start not None, that is returned.

If the memory has no data and no bounds, 0 is returned.

Examples

>>> from bytesparse import Memory

>>> Memory().start
0

~~~

0

1

2

3

4

5

6

7

8

[A

B

C]

[x

y

z]

 
>>> memory = Memory.from_blocks([[1, b'ABC'], [5, b'xyz']])
>>> memory.start
1

~~~

0

1

2

3

4

5

6

7

8

[[[

[x

y

z]

 
>>> memory = Memory.from_blocks([[5, b'xyz']], start=1)
>>> memory.start
1
Type:

int

abstract to_blocks(start=None, endex=None)[source]#

Exports into blocks.

Exports data blocks within an address range, converting them into standalone bytes objects.

Parameters:

  • start (int) – Inclusive start address. If None, start is considered.

  • endex (int) – Exclusive end address. If None, endex is considered.

Returns:

list of blocks – Exported data blocks.

See also

blocks() from_blocks()

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

9

10

[A

B]

[x]

[1

2

3]

 
>>> memory = Memory.from_blocks([[1, b'AB'], [5, b'x'], [7, b'123']])
>>> memory.to_blocks()
[[1, b'AB'], [5, b'x'], [7, b'123']]
>>> memory.to_blocks(2, 9)
[[2, b'B'], [5, b'x'], [7, b'12']]
>>> memory.to_blocks(3, 5)]
[]
abstract to_bytes(start=None, endex=None)[source]#

Exports into bytes.

Exports data within an address range, converting into a standalone bytes object.

Parameters:

  • start (int) – Inclusive start address. If None, start is considered.

  • endex (int) – Exclusive end address. If None, endex is considered.

Returns:

bytes – Exported data bytes.

See also

from_bytes() view()

Examples

>>> from bytesparse import Memory

>>> memory = Memory.from_bytes(b'')
>>> memory.to_bytes()
b''

~~~

0

1

2

3

4

5

6

7

8

[A

B

C

x

y

z]

 
>>> memory = Memory.from_bytes(b'ABCxyz', 2)
>>> memory.to_bytes()
b'ABCxyz'
>>> memory.to_bytes(start=4)
b'Cxyz'
>>> memory.to_bytes(endex=6)
b'ABCx'
>>> memory.to_bytes(4, 6)
b'Cx'
abstract validate()[source]#

Validates internal structure.

It makes sure that all the allocated blocks are sorted by block start address, and that all the blocks are non-overlapping.

Raises:

ValueError – Invalid data detected (see exception message).

abstract values(start=None, endex=None, pattern=None)[source]#

Iterates over values.

Iterates over values, from start to endex. Implemets the interface of dict.

Parameters:

  • start (int) – Inclusive start address. If None, start is considered.

  • endex (int) – Exclusive end address. If None, endex is considered. If Ellipsis, the iterator is infinite.

  • pattern (items) – Pattern of values to fill emptiness.

Yields:

int – Range values.

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

9

A

B

C

D

A

65

66

67

68

65

 
>>> from itertools import islice
>>> memory = Memory()
>>> list(memory.values(endex=8))
[None, None, None, None, None, None, None, None]
>>> list(memory.values(3, 8))
[None, None, None, None, None]
>>> list(islice(memory.values(3, ...), 7))
[None, None, None, None, None, None, None]
>>> list(memory.values(3, 8, b'ABCD'))
[65, 66, 67, 68, 65]

~~~

0

1

2

3

4

5

6

7

8

9

[A

B

C]

<1

2>

[x

y

z]

65

66

67

120

121

122

65

66

67

49

50

120

121

122

 
>>> memory = Memory.from_blocks([[1, b'ABC'], [6, b'xyz']])
>>> list(memory.values())
[65, 66, 67, None, None, 120, 121, 122]
>>> list(memory.values(3, 8))
[67, None, None, 120, 121]
>>> list(islice(memory.values(3, ...), 7))
[67, None, None, 120, 121, 122, None]
>>> list(memory.values(3, 8, b'0123'))
[67, 49, 50, 120, 121]
abstract view(start=None, endex=None)[source]#

Creates a view over a range.

Creates a memory view over the selected address range. Data within the range is required to be contiguous.

Parameters:

  • start (int) – Inclusive start of the viewed range. If None, start is considered.

  • endex (int) – Exclusive end of the viewed range. If None, endex is considered.

Returns:

memoryview – A view of the selected address range.

Raises:

ValueError – Data not contiguous (see contiguous).

Examples

>>> from bytesparse import Memory

0

1

2

3

4

5

6

7

8

9

10

11

[A

B

C

D]

[$]

[x

y

z]

 
>>> memory = Memory.from_blocks([[1, b'ABCD'], [6, b'$'], [8, b'xyz']])
>>> bytes(memory.view(2, 5))
b'BCD'
>>> bytes(memory.view(9, 10))
b'y'
>>> memory.view()
Traceback (most recent call last):
    ...
ValueError: non-contiguous data within range
>>> memory.view(0, 6)
Traceback (most recent call last):
    ...
ValueError: non-contiguous data within range