ReactOS 0.4.16-dev-122-g325d74c
Public Attributes | List of all members
ZSTD_cwksp Struct Reference

#include <zstd_cwksp.h>

Collaboration diagram for ZSTD_cwksp:
Collaboration graph
[legend]

Public Attributes

void * workspace
 
void * workspaceEnd
 
void * objectEnd
 
void * tableEnd
 
void * tableValidEnd
 
void * allocStart
 
int allocFailed
 
int workspaceOversizedDuration
 
ZSTD_cwksp_alloc_phase_e phase
 

Detailed Description

Zstd fits all its internal datastructures into a single continuous buffer, so that it only needs to perform a single OS allocation (or so that a buffer can be provided to it and it can perform no allocations at all). This buffer is called the workspace.

Several optimizations complicate that process of allocating memory ranges from this workspace for each internal datastructure:

  • These different internal datastructures have different setup requirements:
    • The static objects need to be cleared once and can then be trivially reused for each compression.
    • Various buffers don't need to be initialized at all–they are always written into before they're read.
    • The matchstate tables have a unique requirement that they don't need their memory to be totally cleared, but they do need the memory to have some bound, i.e., a guarantee that all values in the memory they've been allocated is less than some maximum value (which is the starting value for the indices that they will then use for compression). When this guarantee is provided to them, they can use the memory without any setup work. When it can't, they have to clear the area.
  • These buffers also have different alignment requirements.
  • We would like to reuse the objects in the workspace for multiple compressions without having to perform any expensive reallocation or reinitialization work.
  • We would like to be able to efficiently reuse the workspace across multiple compressions even when the compression parameters change and we need to resize some of the objects (where possible).

To attempt to manage this buffer, given these constraints, the ZSTD_cwksp abstraction was created. It works as follows:

Workspace Layout:

[ ... workspace ... ] [objects][tables ... ->] free space [<- ... aligned][<- ... buffers]

The various objects that live in the workspace are divided into the following categories, and are allocated separately:

  • Static objects: this is optionally the enclosing ZSTD_CCtx or ZSTD_CDict, so that literally everything fits in a single buffer. Note: if present, this must be the first object in the workspace, since ZSTD_free{CCtx, CDict}() rely on a pointer comparison to see whether one or two frees are required.
  • Fixed size objects: these are fixed-size, fixed-count objects that are nonetheless "dynamically" allocated in the workspace so that we can control how they're initialized separately from the broader ZSTD_CCtx. Examples:
  • Tables: these are any of several different datastructures (hash tables, chain tables, binary trees) that all respect a common format: they are uint32_t arrays, all of whose values are between 0 and (nextSrc - base). Their sizes depend on the cparams.
  • Aligned: these buffers are used for various purposes that require 4 byte alignment, but don't require any initialization before they're used.
  • Buffers: these buffers are used for various purposes that don't require any alignment or initialization before they're used. This means they can be moved around at no cost for a new compression.

Allocating Memory:

The various types of objects must be allocated in order, so they can be correctly packed into the workspace buffer. That order is:

  1. Objects
  2. Buffers
  3. Aligned
  4. Tables

Attempts to reserve objects of different types out of order will fail.

Definition at line 131 of file zstd_cwksp.h.

Member Data Documentation

◆ allocFailed

int ZSTD_cwksp::allocFailed

Definition at line 140 of file zstd_cwksp.h.

◆ allocStart

void* ZSTD_cwksp::allocStart

Definition at line 138 of file zstd_cwksp.h.

◆ objectEnd

void* ZSTD_cwksp::objectEnd

Definition at line 135 of file zstd_cwksp.h.

◆ phase

ZSTD_cwksp_alloc_phase_e ZSTD_cwksp::phase

Definition at line 142 of file zstd_cwksp.h.

◆ tableEnd

void* ZSTD_cwksp::tableEnd

Definition at line 136 of file zstd_cwksp.h.

◆ tableValidEnd

void* ZSTD_cwksp::tableValidEnd

Definition at line 137 of file zstd_cwksp.h.

◆ workspace

void* ZSTD_cwksp::workspace

Definition at line 132 of file zstd_cwksp.h.

Referenced by ZSTD_sizeof_CCtx(), and ZSTD_sizeof_CDict().

◆ workspaceEnd

void* ZSTD_cwksp::workspaceEnd

Definition at line 133 of file zstd_cwksp.h.

◆ workspaceOversizedDuration

int ZSTD_cwksp::workspaceOversizedDuration

Definition at line 141 of file zstd_cwksp.h.

The documentation for this struct was generated from the following file: