--------------------------------

bitstring module version history

--------------------------------

-------------------------

April 2024: version 4.2.0

-------------------------

This release contains a fairly large refactor of how different types are managed. This

shouldn't affect the end user, and the main noticeable change should be the new Dtype

class, which is optional to use.

Support for 8-bit and smaller floats has been reworked and expanded. These are still

a 'beta' feature.

Backwardly incompatible changes:

* Dropped support for Python 3.7. Minimum version is now 3.8.

* For tokens that use a non-numeric length, a ':' is now compulsory rather than

recommended. For example use 'uint:foo' instead of 'uintfoo'.

* The previous e4m3float and e5m2float formats have become the slightly modified

p4binary8 and p3binary8 formats.

* Some parameters are now enforced as positional only, such as `auto` in constructors.

Other changes:

* The Array class is no longer 'beta'.

* A new Dtype class can be optionally used to specify types.

* The bitstring.options object is now the preferred method for changing module options.

The `bitstring.lsb0` and `bitstring.bytealigned` variables are now deprecated, use

`bitstring.options.lsb0` and `bitstring.options.bytealigned` instead.

* New fromstring method as another way to create bitstrings from formatted strings.

Instead of relying on the `auto` parameter you can now optionally use `fromstring`.

>>> s1 = BitArray('u24=1000') # This is still fine

>>> s2 = BitArray.fromstring('u24=1000') # This may be clearer and more efficient.

* More types can now be pretty printed. For example integer and float formats can be used.

>>> s.pp('u15, bin')

* Pretty printing is now prettier - optional terminal colours added.

* A range of 8-bit, 6-bit and even 4-bit float formats added (beta):

p3binary8: IEEE 8-bit floating point with 3 bit precision.

p4binary8: IEEE 8-bit floating point with 4 bit precision.

e5m2mxfp: OCP 8-bit floating point with 3 bit precision.

e4m3mxfp: OCP 8-bit floating point with 4 bit precision.

e2m3mxfp: OCP 6-bit floating point with 4 bit precision.

e3m2mxfp: OCP 6-bit floating point with 3 bit precision.

e2m1mxfp: OCP 4-bit floating point with 2 bit precision.

e8m0mxfp: OCP 8-bit unsigned floating point designed to scale the other formats.

mxint: OCP 8-bit floating point that is a scaled integer representation.

* Performance improvements.

----------------------------

November 2023: version 4.1.4

----------------------------

Fixing a regression introduced in 4.1.3

* 'bytes' token can't be used without explicit length. Bug #303.

----------------------------

November 2023: version 4.1.3

----------------------------

A maintenance release, with some changes to the beta features introduced in 4.1.

* Removed a couple of files that accidentally got included in the previous release. Bug #293.

* The 8-bit float formats have been renamed 'e4m3float' and 'e5m2float'.

* Some refactoring and performance optimizations.

--------------------------------------

September 2023: version 4.1.2 released

--------------------------------------

Another maintenance release. Once again some small changes to the 'beta' Array class,

plus new Array functionality.

* Fix for the module command-line usage. Bug #290.

* Fix for when creating bitstrings from memoryview objects.

* Renamed the 'fmt' parameter for Arrays to 'dtype'.

* More Array operator coverage.

* Added operators that act on two Arrays of the same size.

* Added comparison operators for Arrays that return an Array of bools.

* Added Array.equals method as == will now return an Array (see above item).

* Added astype() method for Arrays to easily cast to a new dtype.

-----------------------------------

August 2023: version 4.1.1 released

-----------------------------------

A maintenance release, with some changes to the Array class which is still in 'beta'.

* bitarray dependency now pinned to ">=2.8.0, <3.0.0" rather than a specific version. Bug #283.

* Fix for using numpy integers as integer parameters. Bug #286.

* Removed ability to extend an Array with the '+' operator. Use the 'extend' method instead.

* Improvements when pretty-printing the Array.

* Array.count() can now count 'nan' values for floating point types.

-----------------------------------

August 2023: version 4.1.0 released

-----------------------------------

This has turned into a surprisingly big release, with a major refactor and a brand new

class (the first for 12 years!) There are also a couple of small possibly breaking changes

detailed below, in particular 'auto' initialising bitstrings from integers is now disallowed.

* Speed increased with bitarray dependency.

The major weakness of bitstring has been its poor performance for computationally

intensive tasks relative to lower level alternatives. This was principally due to

relying on pure Python code to achieve things that the base language often didn't have

fast ways of doing.

This release starts to address that problem with a fairly extensive rewrite to replace

much of the pure Python low-level bit operations with methods from the bitarray package.

This is a package that does many of the same things as bitstring, and the two packages

have co-existed for a long time. While bitarray doesn't have all of the options and

facilities of bitstring it has the advantage of being very fast as it is implemented in C.

By replacing the internal datatypes I can speed up bitstring's operations while keeping

the same API.

Huge kudos to Ilan Schnell for all his work on bitarray.

* New Array class for homogeneous data (beta)

If your data is all of the same type you can make use of the new Array class, which

mirrors much of the functionality of the standard array.array type, but doesn't restrict

you to just a dozen formats.

>>> from bitstring import Array

>>> a = Array('uint7', [9, 100, 3, 1])

>>> a.data

BitArray('0x1390181')

>>> b = Array('float16', a.tolist())

>>> b.append(0.25)

>>> b.tobytes()

b'H\x80V@B\x00<\x004\x00'

>>> b.tolist()

[9.0, 100.0, 3.0, 1.0, 0.25]

The data is stored efficiently in a BitArray object, and you can manipulate both the

data and the Array format freely. See the main documentation for more details. Note that

this feature carries the 'beta' flag so may change in future point versions.

Other changes:

* Added two new floating point interpretations: float8_143 and float8_152. These are 8-bit

floating point formats, with very limited range and precision, but useful in some fields,

particularly machine learning. This is an experimental feature - the formats haven't

even been standardised yet.

>>> a = Bits(float8_143=16.5)

>>> a.bin

'01100000'

>>> a.float8_143

16.0

* Auto initialization from ints has been removed and now raises a TypeError. Creating a

bitstring from an int still creates a zeroed bitstring of that length but ints won't

be promoted to bitstrings as that has been a constant source of errors and confusion.

>>> a = BitArray(100) # Fine - create with 100 zeroed bits

>>> a += 0xff # TypeError - previously this would have appended 0xff (=255) zero bits.

>>> a += '0xff' # Probably what was meant - append eight '1' bits.

>>> a += Bits(255) # Fine, append 255 zero bits.

This is a breaking change, but it breaks loudly with an exception, it is easily recoded,

and it removes a confusing wrinkle.

* Explicitly specifying the 'auto' parameter is now disallowed rather than discouraged.

It was always meant to be a positional-only parameter (and will be once I can drop

Python 3.7 support) but for now it's renamed to '__auto'. In the unlikely event

this breaks code, the fix should be just to delete the 'auto=' if it's already the

first parameter.

>>> s = Bits(auto='0xff') # Now raises a CreationError

>>> s = Bits('0xff') # Fine, as always

* Deleting, replacing or inserting into a bitstring resets the bit position to 0 if the

bitstring's length has been changed. Previously the bit position was adjusted but

this was not well defined.

* Only empty bitstring are now considered False in a boolean sense. Previously s was

False is no bits in s were set to 1, but this goes against what it means to be a

container in Python so I consider this to be a bug, even if it was documented. I'm

guessing it's related to __nonzero__ in Python 2 becoming __bool__ in Python 3, and

it's never been fixed before now.

* Casting to bytes now behaves as expected, so that bytes(s) gives the same result as

s.tobytes(). Previously it created a byte per bit.

* Pretty printing with the 'bytes' format now uses characters from the 'Latin Extended-A'

unicode block for non-ASCII and unprintable characters instead of replacing them with '.'

* When using struct-like codes you can now use '=' instead of '@' to signify native-

endianness. They behave identically, but the new '=' is now preferred.

* More fixes for LSB0 mode. There are now no known issues with this feature.

----------------------------------

April 2023: version 4.0.2 released

----------------------------------

A maintenance release.

* Added py.typed file and converted the module to a package to let mypy find type

annotations. Bug 248.

* Fix to shifting operations when using LSB0 mode. Bug 251.

* A few more fixes for LSB0 mode.

* Improved LSB0 documentation.

* Added build-system section to pyproject.toml. Bug 243.

* Rewrote the walkthrough documentation as a jupyter notebook.

* Updated the project's logo.

-------------------------------------

November 2022: version 4.0.1 released

-------------------------------------

This is a major release which drops support for Python 2.7 and has a new minimum

requirement of Python 3.7. Around 95% of downloads satisfy this - users of

older versions can continue to use bitstring 3.1, which will still be supported

with fixes, but no new features.

Other breaking changes are minimal, and there are a few cool features added.

Breaking changes:

* Minimum supported Python version is now Python 3.7.

* Removed ConstBitArray and BitString class aliases. Use Bits and BitStream instead.

* The cut() method will now also yield the final bits of a bitstring, even if they

are shorter than the requested cut size.

* Removed default uint interpretation. This wasn't being applied uniformly - default

is now always to return a bitstring object of the given length and not to interpret

it as a uint. Bug 220.

* If an overwrite goes beyond the end of the bitstring it will now extend the bitstring

rather than raise an exception. Bug 148.

New features and improvements:

* Type hints added throughout the code.

* Underscores are now allowed in strings representing number literals.

* The copy() method now works on Bits as well as BitArray objects.

* The experimental command-line feature is now official. Command-line

parameters are concatenated and a bitstring created from them. If

the final parameter is either an interpretation string or ends with

a '.' followed by an interpretation string then that interpretation

of the bitstring will be used when printing it.

$ python -m bitstring int:16=-400

0xfe70

$ python -m bitstring float:32=0.2 bin

00111110010011001100110011001101

* New pp() method that pretty-prints the bitstring in various formats - useful

especially in interactive sessions. Thanks to Omer Barak for the suggestion

and discussion.

>>> s.pp()

0: 10001000 01110110 10001110 01110110 11111000 01110110 10000111 00101000

64: 01110010 11111001 10000111 10011000 11110111 10011110 10000111 11111101

128: 11111001 10001100 01111111 10111100 10111111 11011011 11101011 11111011

192: 1100

>>> s.pp('bin, hex')

0: 10001000 01110110 10001110 01110110 11111000 01110110 88 76 8e 76 f8 76

48: 10000111 00101000 01110010 11111001 10000111 10011000 87 28 72 f9 87 98

96: 11110111 10011110 10000111 11111101 11111001 10001100 f7 9e 87 fd f9 8c

144: 01111111 10111100 10111111 11011011 11101011 11111011 7f bc bf db eb fb

192: 1100 c

* Shorter and more versatile properties. The bin, oct, hex, float, uint and int

properties can now be shortened to just their first letter. They can also have

a length in bits after them - allowing Rust-like data types. ::

>>> s = BitArray('0x44961000')

>>> s.h

'44961000'

>>> s.f32

1200.5

>>> s.u

1150685184

>>> s.i7 = -60

>>> s.b

'1000100'

>>> t = Bits('u12=160, u12=120, b=100')

* Other types with bit lengths can also be used as properties ::

>>> s.floatle64 = 10.511

* A colon is no longer required in format strings before a bit length. So for

example `Bits('int:15=-101')` could be written as `Bits('int15=-101')`. This is

now the preferred usage in the documentation except where the colon improves

readability.

* Support for IEEE 16 bit floats. Floating point types can now be 16 bits long as well

as 32 and 64 bits. This is using the 'e' format from the struct module.

* Support for bfloats. This is a specialised 16-bit floating point format

mostly used in machine learning. It's essentially a truncated IEEE 32-bit

format that keeps its range but only has a couple of signficant figures of

accuracy.

---------------------------------------

July 20th 2021: version 3.1.9 released

---------------------------------------

(version 3.1.8 was pulled due to serious issues)

Another maintenance release.

* Fixed a couple of outdated results in the readme (Issue 214).

* Some more documentation tidying.

* Turned off some debug code by default.

* Fixed a couple of failing tests in different Python versions.

* Fix for consistent pos initialisation semantics for different types.

* Change to allow wheels to be uploaded to PyPI.

* More work for LSB0 mode, but still not finished or documented (sorry).

---------------------------------------

May 5th 2020: version 3.1.7 released

---------------------------------------

This is a maintenance release with a few bug fixes plus an experimental

feature to allow bits to be indexed in the opposite direction.

* Fixing del not working correctly when stop value negative (Issue 201)

* Removed deprecated direct import of ABC from collections module (Issue 196)

* Tested and added explicit support for Python 3.7 and 3.8. (Issue 193)

* Fixing a few stale links to documentation. (Issue 194)

* Allowing initialisation with an io.BytesIO object. (Issue 189)

Experimental LSB0 mode

----------------------

This feature allows bitstring to use Least Significant Bit Zero

(LSB0) bit numbering; that is the final bit in the bitstring will

be bit 0, and the first bit will be bit (n-1), rather than the

other way around. LSB0 is a more natural numbering

system in many fields, but is the opposite to Most Significant Bit

Zero (MSB0) numbering which is the natural option when thinking of

bitstrings as standard Python containers.

To switch from the default MSB0, use the module level function

>>> bitstring.set_lsb0(True)

Getting and setting bits should work in this release, as will some

other methods. Many other methods are not tested yet and might not

work as expected. This is mostly a release to get feedback before

finalising the interface.

Slicing is still done with the start bit smaller than the end bit.

For example:

>>> s = Bits('0b000000111')

>>> s[0:5]

Bits('0b00111')

>>> s[0]

True

Negative indices work as (hopefully) you'd expect, with the first stored

bit being `s[-1]` and the final stored bit being `s[-n]`.

See https://github.com/scott-griffiths/bitstring/issues/156 for

discussions and to add any further comments.

---------------------------------------

July 9th 2019: version 3.1.6 released

---------------------------------------

A long overdue maintenance release with some fixes.

* Fixed immutability bug. Bug 176.

* Fixed failure of `__contains__` in some circumstances. Bug 180.

* Better handling of open files. Bug 186.

* Better Python 2/3 check.

* Making unit tests easier to run.

* Allowing length of 1 to be specified for bools. (Thanks to LemonPi)

---------------------------------------

May 17th 2016: version 3.1.5 released

---------------------------------------

* Support initialisation from an array.

* Added a separate LICENSE file.

---------------------------------------

March 19th 2016: version 3.1.4 released

---------------------------------------

This is another bug fix release.

* Fix for bitstring types when created directly from other bitstring types.

* Updating contact, website details.

---------------------------------------

March 4th 2014: version 3.1.3 released

---------------------------------------

This is another bug fix release.

* Fix for problem with prepend for bitstrings with byte offsets in their data store.

---------------------------------------

April 18th 2013: version 3.1.2 released

---------------------------------------

This is another bug fix release.

* Fix for problem where unpacking bytes would by eight times too long

---------------------------------------

March 21st 2013: version 3.1.1 released

---------------------------------------

This is a bug fix release.

* Fix for problem where concatenating bitstrings sometimes modified method's arguments

------------------------------------------

February 26th 2013: version 3.1.0 released

------------------------------------------

This is a minor release with a couple of new features and some bug fixes.

New 'pad' token

---------------

This token can be used in reads and when packing/unpacking to indicate that

you don't care about the contents of these bits. Any padding bits will just

be skipped over when reading/unpacking or zero-filled when packing.

>>> a, b = s.readlist('pad:5, uint:3, pad:1, uint:3')

Here only two items are returned in the list - the padding bits are ignored.

New clear and copy convenience methods

--------------------------------------

These methods have been introduced in Python 3.3 for lists and bytearrays,

as more obvious ways of clearing and copying, and we mirror that change here.

t = s.copy() is equivalent to t = s[:], and s.clear() is equivalent to del s[:].

Other changes

-------------

* Some bug fixes.

-----------------------------------------

February 7th 2012: version 3.0.2 released

-----------------------------------------

This is a minor update that fixes a few bugs.

* Fix for subclasses of bitstring classes behaving strangely (Issue 121).

* Fix for excessive memory usage in rare cases (Issue 120).

* Fixes for slicing edge cases.

There has also been a reorganisation of the code to return it to a single

'bitstring.py' file rather than the package that has been used for the past

several releases. This change shouldn't affect users directly.

------------------------------------------

November 21st 2011: version 3.0.1 released

------------------------------------------

This release fixed a small but very visible bug in bitstring printing.

------------------------------------------

November 21st 2011: version 3.0.0 released

------------------------------------------

This is a major release which breaks backward compatibility in a few places.

Backwardly incompatible changes

===============================

Hex, oct and bin properties don't have leading 0x, 0o and 0b

------------------------------------------------------------

If you ask for the hex, octal or binary representations of a bitstring then

they will no longer be prefixed with '0x', 0o' or '0b'. This was done as it

was noticed that the first thing a lot of user code does after getting these

representations was to cut off the first two characters before further

processing.

>>> a = BitArray('0x123')

>>> a.hex, a.oct, a.bin

('123', '0443', '000100100011')

Previously this would have returned ('0x123', '0o0443', '0b000100100011')

This change might require some recoding, but it should all be simplifications.

ConstBitArray renamed to Bits

-----------------------------

Previously Bits was an alias for ConstBitStream (for backward compatibility).

This has now changed so that Bits and BitArray loosely correspond to the

built-in types bytes and bytearray.

If you were using streaming/reading methods on a Bits object then you will

have to change it to a ConstBitStream.

The ConstBitArray name is kept as an alias for Bits.

Stepping in slices has conventional meaning

-------------------------------------------

The step parameter in __getitem__, __setitem__ and __delitem__ used to act

as a multiplier for the start and stop parameters. No one seemed to use it

though and so it has now reverted to the conventional meaning for containers.

If you are using step then recoding is simple: s[a:b:c] becomes s[a*c:b*c].

Some examples of the new usage:

>>> s = BitArray('0x0000')

s[::4] = [1, 1, 1, 1]

>>> s.hex

'8888'

>>> del s[8::2]

>>> s.hex

'880'

New features

============

New readto method

-----------------

This method is a mix between a find and a read - it searches for a bitstring

and then reads up to and including it. For example:

>>> s = ConstBitStream('0x47000102034704050647')

>>> s.readto('0x47', bytealigned=True)

BitStream('0x47')

>>> s.readto('0x47', bytealigned=True)

BitStream('0x0001020347')

>>> s.readto('0x47', bytealigned=True)

BitStream('0x04050647')

pack function accepts an iterable as its format

-----------------------------------------------

Previously only a string was accepted as the format in the pack function.

This was an oversight as it broke the symmetry between pack and unpack.

Now you can use formats like this:

fmt = ['hex:8', 'bin:3']

a = pack(fmt, '47', '001')

a.unpack(fmt)

--------------------------------------

June 18th 2011: version 2.2.0 released

--------------------------------------

This is a minor upgrade with a couple of new features.

New interleaved exponential-Golomb interpretations

--------------------------------------------------

New bit interpretations for interleaved exponential-Golomb (as used in the

Dirac video codec) are supplied via 'uie' and 'sie':

>>> s = BitArray(uie=41)

>>> s.uie

41

>>> s.bin

'0b00010001001'

These are pretty similar to the non-interleaved versions - see the manual

for more details. Credit goes to Paul Sargent for the patch.

New package-level bytealigned variable

--------------------------------------

A number of methods take a 'bytealigned' parameter to indicate that they

should only work on byte boundaries (e.g. find, replace, split). Previously

this parameter defaulted to 'False'. Instead it now defaults to

'bitstring.bytealigned', which itself defaults to 'False', but can be changed

to modify the default behaviour of the methods. For example:

>>> a = BitArray('0x00 ff 0f ff')

>>> a.find('0x0f')

(4,) # found first not on a byte boundary

>>> a.find('0x0f', bytealigned=True)

(16,) # forced looking only on byte boundaries

>>> bitstring.bytealigned = True # Change default behaviour

>>> a.find('0x0f')

(16,)

>>> a.find('0x0f', bytealigned=False)

(4,)

If you're only working with bytes then this can help avoid some errors and

save some typing!

Other changes

-------------

* Fix for Python 3.2, correcting for a change to the binascii module.

* Fix for bool initialisation from 0 or 1.

* Efficiency improvements, including interning strategy.

------------------------------------------

February 23rd 2011: version 2.1.1 released

------------------------------------------

This is a release to fix a couple of bugs that were introduced in 2.1.0.

* Bug fix: Reading using the 'bytes' token had been broken (Issue 102).

* Fixed problem using some methods on ConstBitArrays.

* Better exception handling for tokens missing values.

* Some performance improvements.

-----------------------------------------

January 23rd 2011: version 2.1.0 released

-----------------------------------------

New class hierarchy introduced with simpler classes

---------------------------------------------------

Previously there were just two classes, the immutable Bits which was the base

class for the mutable BitString class. Both of these classes have the concept

of a bit position, from which reads etc. take place so that the bitstring could

be treated as if it were a file or stream.

Two simpler classes have now been added which are purely bit containers and

don't have a bit position. These are called ConstBitArray and BitArray. As you

can guess the former is an immutable version of the latter.

The other classes have also been renamed to better reflect their capabilities.

Instead of BitString you can use BitStream, and instead of Bits you can use

ConstBitStream. The old names are kept as aliases for backward compatibility.

The classes hierarchy is:

ConstBitArray

/ \

/ \

BitArray ConstBitStream (formerly Bits)

\ /

\ /

BitStream (formerly BitString)

Other changes

-------------

A lot of internal reorganisation has taken place since the previous version,

most of which won't be noticed by the end user. Some things you might see are:

* New package structure. Previous versions have been a single file for the

module and another for the unit tests. The module is now split into many

more files so it can't be used just by copying bitstring.py any more.

* To run the unit tests there is now a script called runtests.py in the test

directory.

* File based bitstring are now implemented in terms of an mmap. This should

be just an implementation detail, but unfortunately for 32-bit versions of

Python this creates a limit of 4GB on the files that can be used. The work

around is either to get a 64-bit Python, or just stick with version 2.0.

* The ConstBitArray and ConstBitStream classes no longer copy byte data when

a slice or a read takes place, they just take a reference. This is mostly

a very nice optimisation, but there are occasions where it could have an

adverse effect. For example if a very large bitstring is created, a small

slice taken and the original deleted. The byte data from the large

bitstring would still be retained in memory.

* Optimisations. Once again this version should be faster than the last.

The module is still pure Python but some of the reorganisation was to make

it more feasible to put some of the code into Cython or similar, so

hopefully more speed will be on the way.

--------------------------------------

July 26th 2010: version 2.0.3 released

--------------------------------------

* Bug fix: Using peek and read for a single bit now returns a new bitstring

as was intended, rather than the old behaviour of returning a bool.

* Removed HTML docs from source archive - better to use the online version.

--------------------------------------

July 25th 2010: version 2.0.2 released

--------------------------------------

This is a major release, with a number of backwardly incompatible changes.

The main change is the removal of many methods, all of which have simple

alternatives. Other changes are quite minor but may need some recoding.

There are a few new features, most of which have been made to help the

stream-lining of the API. As always there are performance improvements and

some API changes were made purely with future performance in mind.

The backwardly incompatible changes are:

-----------------------------------------

* Methods removed.

About half of the class methods have been removed from the API. They all have

simple alternatives, so what remains is more powerful and easier to remember.

The removed methods are listed here on the left, with their equivalent

replacements on the right:

s.advancebit() -> s.pos += 1

s.advancebits(bits) -> s.pos += bits

s.advancebyte() -> s.pos += 8

s.advancebytes(bytes) -> s.pos += 8*bytes

s.allunset([a, b]) -> s.all(False, [a, b])

s.anyunset([a, b]) -> s.any(False, [a, b])

s.delete(bits, pos) -> del s[pos:pos+bits]

s.peekbit() -> s.peek(1)

s.peekbitlist(a, b) -> s.peeklist([a, b])

s.peekbits(bits) -> s.peek(bits)

s.peekbyte() -> s.peek(8)

s.peekbytelist(a, b) -> s.peeklist([8*a, 8*b])

s.peekbytes(bytes) -> s.peek(8*bytes)

s.readbit() -> s.read(1)

s.readbitlist(a, b) -> s.readlist([a, b])

s.readbits(bits) -> s.read(bits)

s.readbyte() -> s.read(8)

s.readbytelist(a, b) -> s.readlist([8*a, 8*b])

s.readbytes(bytes) -> s.read(8*bytes)

s.retreatbit() -> s.pos -= 1

s.retreatbits(bits) -> s.pos -= bits

s.retreatbyte() -> s.pos -= 8

s.retreatbytes(bytes) -> s.pos -= 8*bytes

s.reversebytes(start, end) -> s.byteswap(0, start, end)

s.seek(pos) -> s.pos = pos

s.seekbyte(bytepos) -> s.bytepos = bytepos

s.slice(start, end, step) -> s[start:end:step]

s.tell() -> s.pos

s.tellbyte() -> s.bytepos

s.truncateend(bits) -> del s[-bits:]

s.truncatestart(bits) -> del s[:bits]

s.unset([a, b]) -> s.set(False, [a, b])

Many of these methods have been deprecated for the last few releases, but

there are some new removals too. Any recoding needed should be quite

straightforward, so while I apologise for the hassle, I had to take the

opportunity to streamline and rationalise what was becoming a bit of an

overblown API.

* set / unset methods combined.

The set/unset methods have been combined in a single method, which now

takes a boolean as its first argument:

s.set([a, b]) -> s.set(1, [a, b])

s.unset([a, b]) -> s.set(0, [a, b])

s.allset([a, b]) -> s.all(1, [a, b])

s.allunset([a, b]) -> s.all(0, [a, b])

s.anyset([a, b]) -> s.any(1, [a, b])

s.anyunset([a, b]) -> s.any(0, [a, b])

* all / any only accept iterables.

The all and any methods (previously called allset, allunset, anyset and

anyunset) no longer accept a single bit position. The recommended way of

testing a single bit is just to index it, for example instead of:

>>> if s.all(True, i):

just use

>>> if s[i]:

If you really want to you can of course use an iterable with a single

element, such as 's.any(False, [i])', but it's clearer just to write

'not s[i]'.

* Exception raised on reading off end of bitstring.

If a read or peek goes beyond the end of the bitstring then a ReadError

will be raised. The previous behaviour was that the rest of the bitstring

would be returned and no exception raised.

* BitStringError renamed to Error.

The base class for errors in the bitstring module is now just Error, so

it will likely appears in your code as bitstring.Error instead of

the rather repetitive bitstring.BitStringError.

* Single bit slices and reads return a bool.

A single index slice (such as s[5]) will now return a bool (i.e. True or

False) rather than a single bit bitstring. This is partly to reflect the

style of the bytearray type, which returns an integer for single items, but

mostly to avoid common errors like:

>>> if s[0]:

... do_something()

While the intent of this code snippet is quite clear (i.e. do_something if

the first bit of s is set) under the old rules s[0] would be true as long

as s wasn't empty. That's because any one-bit bitstring was true as it was a

non-empty container. Under the new rule s[0] is True if s starts with a '1'

bit and False if s starts with a '0' bit.

The change does not affect reads and peeks, so s.peek(1) will still return

a single bit bitstring, which leads on to the next item...

* Empty bitstrings or bitstrings with only zero bits are considered False.

Previously a bitstring was False if it had no elements, otherwise it was True.

This is standard behaviour for containers, but wasn't very useful for a container

of just 0s and 1s. The new behaviour means that the bitstring is False if it

has no 1 bits. This means that code like this:

>>> if s.peek(1):

... do_something()

should work as you'd expect. It also means that Bits(1000), Bits(0x00) and

Bits('uint:12=0') are all also False. If you need to check for the emptiness of

a bitstring then instead check the len property:

if s -> if s.len

if not s -> if not s.len

* Length and offset disallowed for some initialisers.

Previously you could create bitstring using expressions like:

>>> s = Bits(hex='0xabcde', offset=4, length=13)

This has now been disallowed, and the offset and length parameters may only

be used when initialising with bytes or a file. To replace the old behaviour

you could instead use

>>> s = Bits(hex='0xabcde')[4:17]

* Renamed 'format' parameter 'fmt'.

Methods with a 'format' parameter have had it renamed to 'fmt', to prevent

hiding the built-in 'format'. Affects methods unpack, read, peek, readlist,

peeklist and byteswap and the pack function.

* Iterables instead of *format accepted for some methods.

This means that for the affected methods (unpack, readlist and peeklist) you

will need to use an iterable to specify multiple items. This is easier to

show than to describe, so instead of

>>> a, b, c, d = s.readlist('uint:12', 'hex:4', 'bin:7')

you would instead write

>>> a, b, c, d = s.readlist(['uint:12', 'hex:4', 'bin:7'])

Note that you could still use the single string 'uint:12, hex:4, bin:7' if

you preferred.

* Bool auto-initialisation removed.

You can no longer use True and False to initialise single bit bitstrings.

The reasoning behind this is that as bool is a subclass of int, it really is

bad practice to have Bits(False) be different to Bits(0) and to have Bits(True)

different to Bits(1).

If you have used bool auto-initialisation then you will have to be careful to

replace it as the bools will now be interpreted as ints, so Bits(False) will

be empty (a bitstring of length 0), and Bits(True) will be a single zero bit

(a bitstring of length 1). Sorry for the confusion, but I think this will

prevent bigger problems in the future.

There are a few alternatives for creating a single bit bitstring. My favourite

it to use a list with a single item:

Bits(False) -> Bits([0])

Bits(True) -> Bits([1])

* New creation from file strategy

Previously if you created a bitstring from a file, either by auto-initialising

with a file object or using the filename parameter, the file would not be read

into memory unless you tried to modify it, at which point the whole file would

be read.

The new behaviour depends on whether you create a Bits or a BitString from the

file. If you create a Bits (which is immutable) then the file will never be

read into memory. This allows very large files to be opened for examination

even if they could never fit in memory.

If however you create a BitString, the whole of the referenced file will be read

to store in memory. If the file is very big this could take a long time, or fail,

but the idea is that in saying you want the mutable BitString you are implicitly

saying that you want to make changes and so (for now) we need to load it into

memory.

The new strategy is a bit more predictable in terms of performance than the old.

The main point to remember is that if you want to open a file and don't plan to

alter the bitstring then use the Bits class rather than BitString.

Just to be clear, in neither case will the contents of the file ever be changed -

if you want to output the modified BitString then use the tofile method, for

example.

* find and rfind return a tuple instead of a bool.

If a find is unsuccessful then an empty tuple is returned (which is False in a

boolean sense) otherwise a single item tuple with the bit position is returned

(which is True in a boolean sense). You shouldn't need to recode unless you

explicitly compared the result of a find to True or False, for example this

snippet doesn't need to be altered:

>>> if s.find('0x23'):

... print(s.bitpos)

but you could now instead use

>>> found = s.find('0x23')

>>> if found:

... print(found[0])

The reason for returning the bit position in a tuple is so that finding at

position zero can still be True - it's the tuple (0,) - whereas not found can

be False - the empty tuple ().

The new features in this release are:

-------------------------------------

* New count method.

This method just counts the number of 1 or 0 bits in the bitstring.

>>> s = Bits('0x31fff4')

>>> s.count(1)

16

* read and peek methods accept integers.

The read, readlist, peek and peeklist methods now accept integers as parameters

to mean "read this many bits and return a bitstring". This has allowed a number

of methods to be removed from this release, so for example instead of:

>>> a, b, c = s.readbits(5, 6, 7)

>>> if s.peekbit():

... do_something()

you should write:

>>> a, b, c = s.readlist([5, 6, 7])

>>> if s.peek(1):

... do_something()

* byteswap used to reverse all bytes.

The byteswap method now allows a format specifier of 0 (the default) to signify

that all of the whole bytes should be reversed. This means that calling just

byteswap() is almost equivalent to the now removed bytereverse() method (a small

difference is that byteswap won't raise an exception if the bitstring isn't a

whole number of bytes long).

* Auto initialise with bytearray or (for Python 3 only) bytes.

So rather than writing:

>>> a = Bits(bytes=some_bytearray)

you can just write

>>> a = Bits(some_bytearray)

This also works for the bytes type, but only if you're using Python 3.

For Python 2 it's not possible to distinguish between a bytes object and a

str. For this reason this method should be used with some caution as it will

make you code behave differently with the different major Python versions.

>>> b = Bits(b'abcd\x23\x00') # Only Python 3!

* set, invert, all and any default to whole bitstring.

This means that you can for example write:

>>> a = BitString(100) # 100 zero bits

>>> a.set(1) # set all bits to 1

>>> a.all(1) # are all bits set to 1?

True

>>> a.any(0) # are any set to 0?

False

>>> a.invert() # invert every bit

* New exception types.

As well as renaming BitStringError to just Error

there are also new exceptions which use Error as a base class.

These can be caught in preference to Error if you need finer control.

The new exceptions sometimes also derive from built-in exceptions:

ByteAlignError(Error) - whole byte position or length needed.

ReadError(Error, IndexError) - reading or peeking off the end of

the bitstring.

CreationError(Error, ValueError) - inappropriate argument during

bitstring creation.

InterpretError(Error, ValueError) - inappropriate interpretation of

binary data.

--------------------------------------------------------------

March 18th 2010: version 1.3.0 for Python 2.6 and 3.x released

--------------------------------------------------------------

New features:

* byteswap method for changing endianness.

Changes the endianness in-place according to a format string or

integer(s) giving the byte pattern. See the manual for details.

>>> s = BitString('0x00112233445566')

>>> s.byteswap(2)

3

>>> s

BitString('0x11003322554466')

>>> s.byteswap('h')

3