dulwich.pack : API documentation

Classes for dealing with packed git objects.

A pack is a compact representation of a bunch of objects, stored using deltas where possible.

They have two parts, the pack file, which stores the data, and an index that tells you where the data is.

To find an object you look in all of the index files 'til you find a match for the object name. You then use the pointer got from this as a pointer in to the corresponding packfile.

Function	take_msb_bytes	Read bytes marked with most significant bit.
Class	UnpackedObject	Class encapsulating an object unpacked from a pack file.
Function	read_zlib_chunks	Read zlib data from a buffer.
Function	iter_sha1	Return the hexdigest of the SHA1 over a set of names.
Function	load_pack_index	Load an index file by path.
Function	load_pack_index_file	Load an index file from a file-like object.
Function	bisect_find_sha	Find a SHA in a data blob with sorted SHAs.
Class	PackIndex	An index in to a packfile.
Class	MemoryPackIndex	Pack index that is stored entirely in memory.
Class	FilePackIndex	Pack index that is based on a file.
Class	PackIndex1	Version 1 Pack Index file.
Class	PackIndex2	Version 2 Pack Index file.
Function	read_pack_header	Read the header of a pack file.
Function	chunks_length	Undocumented
Function	unpack_object	Unpack a Git object.
Class	PackStreamReader	Class to read a pack stream.
Class	PackStreamCopier	Class to verify a pack stream as it is being read.
Function	obj_sha	Compute the SHA for a numeric type and object chunks.
Function	compute_file_sha	Hash a portion of a file into a new SHA.
Class	PackData	The data contained in a packfile.
Class	DeltaChainIterator	Abstract iterator over pack data based on delta chains.
Class	PackIndexer	Delta chain iterator that yields index entries.
Class	PackInflater	Delta chain iterator that yields ShaFile objects.
Class	SHA1Reader	Wrapper for file-like object that remembers the SHA1 of its data.
Class	SHA1Writer	Wrapper for file-like object that remembers the SHA1 of its data.
Function	pack_object_header	Create a pack object header for the given object info.
Function	write_pack_object	Write pack object to a file.
Function	write_pack	Write a new pack data file.
Function	write_pack_header	Write a pack header for the given number of objects.
Function	deltify_pack_objects	Generate deltas for pack objects.
Function	pack_objects_to_data	Create pack data from objects
Function	write_pack_objects	Write a new pack data file.
Function	write_pack_data	Write a new pack data file.
Function	write_pack_index_v1	Write a new pack index file.
Function	create_delta	Use python difflib to work out how to transform base_buf to target_buf.
Function	apply_delta	Based on the similar function in git's patch-delta.c.
Function	write_pack_index_v2	Write a new pack index file.
Class	Pack	A Git pack object.
Function	_load_file_contents	Undocumented
Function	_compute_object_size	Compute the size of a unresolved object for use with LRUSizeCache.
Function	_delta_encode_size	Undocumented
Function	_encode_copy_operation	Undocumented

def take_msb_bytes(read, crc32=None):

Read bytes marked with most significant bit.

Parameters read Read function

def read_zlib_chunks(read_some, unpacked, include_comp=False, buffer_size=_ZLIB_BUFSIZE):

Read zlib data from a buffer.

This function requires that the buffer have additional data following the compressed data, which is guaranteed to be the case for git pack files.

Parameters	read_some	Read function that returns at least one byte, but may return less than the requested size.
	unpacked	An UnpackedObject to write result data to. If its crc32 attr is not None, the CRC32 of the compressed bytes will be computed using this starting CRC32. After this function, will have the following attrs set: * comp_chunks (if include_comp is True) * decomp_chunks * decomp_len * crc32
	include_comp	If True, include compressed data in the result.
	buffer_size	Size of the read buffer.
Returns	Leftover unused data from the decompression.
Raises	zlib.error	if a decompression error occurred.

def iter_sha1(iter):

Return the hexdigest of the SHA1 over a set of names.

Parameters	iter	Iterator over string objects
Returns	40-byte hex sha1 digest

def load_pack_index(path):

Load an index file by path.

Parameters	filename	Path to the index file
Returns	A PackIndex loaded from the given path

def _load_file_contents(f, size=None):

Undocumented

def load_pack_index_file(path, f):

Load an index file from a file-like object.

Parameters	path	Path for the index file
	f	File-like object
Returns	A PackIndex loaded from the given file

def bisect_find_sha(start, end, sha, unpack_name):

Find a SHA in a data blob with sorted SHAs.

Parameters	start	Start index of range to search
	end	End index of range to search
	sha	Sha to find
	unpack_name	Callback to retrieve SHA by index
Returns	Index of the SHA, or None if it wasn't found

def read_pack_header(read):

Read the header of a pack file.

Parameters	read	Read function
Returns	Tuple of (pack version, number of objects). If no data is available to read, returns (None, None).

def chunks_length(chunks):

Undocumented

def unpack_object(read_all, read_some=None, compute_crc32=False, include_comp=False, zlib_bufsize=_ZLIB_BUFSIZE):

Unpack a Git object.

Parameters	read_all	Read function that blocks until the number of requested bytes are read.
	read_some	Read function that returns at least one byte, but may not return the number of bytes requested.
	compute_crc32	If True, compute the CRC32 of the compressed data. If False, the returned CRC32 will be None.
	include_comp	If True, include compressed data in the result.
	zlib_bufsize	An optional buffer size for zlib operations.
Returns	A tuple of (unpacked, unused), where unused is the unused data leftover from decompression, and unpacked in an UnpackedObject with the following attrs set: obj_chunks (for non-delta types) pack_type_num delta_base (for delta types) comp_chunks (if include_comp is True) decomp_chunks decomp_len crc32 (if compute_crc32 is True)

def _compute_object_size(value):

Compute the size of a unresolved object for use with LRUSizeCache.

def obj_sha(type, chunks):

Compute the SHA for a numeric type and object chunks.

def compute_file_sha(f, start_ofs=0, end_ofs=0, buffer_size=116):

Hash a portion of a file into a new SHA.

Parameters	f	A file-like object to read from that supports seek().
	start_ofs	The offset in the file to start reading at.
	end_ofs	The offset in the file to end reading at, relative to the end of the file.
	buffer_size	A buffer size for reading.
Returns	A new SHA object updated with data read from the file.

def pack_object_header(type_num, delta_base, size):

Create a pack object header for the given object info.

Parameters	type_num	Numeric type of the object.
	delta_base	Delta base offset or ref, or None for whole objects.
	size	Uncompressed object size.
Returns	A header for a packed object.

def write_pack_object(f, type, object, sha=None):

Write pack object to a file.

Parameters	f	File to write to
	type	Numeric type of the object
	object	Object to write
Returns	Tuple with offset at which the object was written, and crc32

def write_pack(filename, objects, deltify=None, delta_window_size=None):

Write a new pack data file.

Parameters	filename	Path to the new pack file (without .pack extension)
	objects	Iterable of (object, path) tuples to write. Should provide __len__
	window_size	Delta window size
	deltify	Whether to deltify pack objects
Returns	Tuple with checksum of pack file and index file

def write_pack_header(f, num_objects):

Write a pack header for the given number of objects.

def deltify_pack_objects(objects, window_size=None):

Generate deltas for pack objects.

Parameters	objects	An iterable of (object, path) tuples to deltify.
	window_size	Window size; None for default
Returns	Iterator over type_num, object id, delta_base, content delta_base is None for full text entries

def pack_objects_to_data(objects):

Create pack data from objects

Parameters	objects	Pack objects
Returns	Tuples with (type_num, hexdigest, delta base, object chunks)

def write_pack_objects(f, objects, delta_window_size=None, deltify=None):

Write a new pack data file.

Parameters	f	File to write to
	objects	Iterable of (object, path) tuples to write. Should provide __len__
	window_size	Sliding window size for searching for deltas; Set to None for default window size.
	deltify	Whether to deltify objects
Returns	Dict mapping id -> (offset, crc32 checksum), pack checksum

def write_pack_data(f, num_records, records, progress=None):

Write a new pack data file.

Parameters	f	File to write to
	num_records	Number of records
	records	Iterator over type_num, object_id, delta_base, raw
	progress	Function to report progress to
Returns	Dict mapping id -> (offset, crc32 checksum), pack checksum

def write_pack_index_v1(f, entries, pack_checksum):

Write a new pack index file.

Parameters	f	A file-like object to write to
	entries	List of tuples with object name (sha), offset_in_pack, and crc32_checksum.
	pack_checksum	Checksum of the pack file.
Returns	The SHA of the written index file

def _delta_encode_size(size):

Undocumented

def _encode_copy_operation(start, length):

Undocumented

def create_delta(base_buf, target_buf):

Use python difflib to work out how to transform base_buf to target_buf.

Parameters	base_buf	Base buffer
	target_buf	Target buffer

def apply_delta(src_buf, delta):

Based on the similar function in git's patch-delta.c.

Parameters	src_buf	Source buffer
	delta	Delta instructions

def write_pack_index_v2(f, entries, pack_checksum):

Write a new pack index file.

Parameters	f	File-like object to write to
	entries	List of tuples with object name (sha), offset_in_pack, and crc32_checksum.
	pack_checksum	Checksum of the pack file.
Returns	The SHA of the index file written

API Documentation for Dulwich, generated by pydoctor at 2018-11-17 19:05:54.

dulwich.pack module documentation

`dulwich.pack` module documentation