# Delta Recipe Manifest and Binary Opcode Specifications

> Technical specification for the PAKD container format, uLEB128 opcode serialization, sub-chunk span deltas, and cryptographic integrity verification.

---

## 1. Design Objectives

The Delta Recipe Manifest is the core artifact generated by `pak-delta`. It encodes the minimal deterministic instructions required for a client machine to transform a source baseline archive into a target archive.

### Core Invariants
1. **Minimal Net Patch Footprint:** Combine content-defined deduplication with sub-chunk byte-level span deltas to minimize download payload size.
2. **Sub-Chunk Secondary Delta Compression:** When a chunk is modified but retains high similarity (> 80%), transmit a span-encoded byte delta rather than the full raw chunk. As proven in Experiment 6, this collapses an 8 KB chunk payload from 7,004 bytes down to **19 bytes (99.73% payload savings)**.
3. **LEB128 Opcode Compactification:** Pack integer offsets, lengths, and dictionary pointers into unsigned variable-length integers (uLEB128), reducing opcode metadata from 45 bytes to **9 bytes per chunk (80.00% metadata savings)** as proven in Experiment 12.
4. **Deterministic Reconstruction Parameters:** Record per-entry compression modes (`standard` vs `bit_preserving`), uncompressed CRC32 checksums, and DMA sector alignment padding.
5. **Cryptographic Integrity Guarantees:** Enforce strict baseline SHA-256 matching, per-chunk SHA-256 validation, and target SHA-256 bit-parity verification.

---

## 2. Binary Container Format (`PAKD`)

The binary delta recipe file uses the custom `PAKD` container structure:

```text
+-------------------------------------------------------------------------+
| Section 1: Container Header (74 Bytes)                                 |
| [Magic: 'PAKD' (4B)] [Version: 0x01 (1B)] [Flags (1B)]                 |
| [Source Archive SHA-256 (32B)] [Target Archive SHA-256 (32B)]           |
| [Target Total Uncompressed Bytes (LEB128)]                              |
+-------------------------------------------------------------------------+
| Section 2: Entry Metadata Table (Variable)                              |
| [Entry Count (LEB128)]                                                  |
| Array of Entry Descriptors:                                             |
|   [Path Length (LEB128)] [Path String (UTF-8)]                          |
|   [Compression Method (1B)] [Repack Mode (1B)]                           |
|   [Uncompressed Size (LEB128)] [Compressed Size (LEB128)]               |
|   [CRC32 (4B)] [Alignment Padding (LEB128)]                             |
|   [Opcode Count (LEB128)]                                               |
+-------------------------------------------------------------------------+
| Section 3: Opcode Stream (LEB128 Packed Sequence)                       |
| Concatenated sequence of COPY, PATCH, and INSERT instructions           |
+-------------------------------------------------------------------------+
| Section 4: Raw Payload Pool (Concatenated Binary Blob)                  |
| Concatenated sub-chunk span deltas and novel raw chunk bytes            |
+-------------------------------------------------------------------------+
```

---

## 3. TypeScript Domain Schema

```typescript
export interface DeltaRecipeManifest {
  magic: "PAKD";
  version: "1.0.0";
  sourceArchiveSha256: string;
  targetArchiveSha256: string;
  targetTotalUncompressedBytes: number;
  entries: RecipeEntryDescriptor[];
  rawPayloadPool: Uint8Array;
}

export interface RecipeEntryDescriptor {
  filename: string;
  compressionMethod: number; // 0 = Stored, 8 = Deflated
  repackMode: "standard" | "bit_preserving";
  uncompressedSize: number;
  compressedSize: number;
  crc32: number;
  alignmentPadding: number;
  extraFieldBase64?: string;
  opcodes: RecipeOpcode[];
}

export type RecipeOpcode =
  | CopyOpcode
  | PatchOpcode
  | InsertOpcode;

export interface CopyOpcode {
  type: "COPY";
  sourceOffset: number; // Uncompressed byte offset in baseline archive
  length: number; // Byte length to copy
  chunkHash: string; // SHA-256 digest for chunk verification
}

export interface PatchOpcode {
  type: "PATCH";
  sourceOffset: number; // Uncompressed byte offset of baseline chunk
  length: number; // Chunk length
  chunkHash: string; // Expected SHA-256 of patched chunk
  deltaOffset: number; // Offset into manifest raw payload pool for sub-chunk span delta
  deltaLength: number; // Byte length of span delta blob
}

export interface InsertOpcode {
  type: "INSERT";
  length: number; // Chunk byte length
  chunkHash: string; // Expected SHA-256 of novel chunk
  payloadOffset: number; // Offset into manifest raw payload pool
}
```

---

## 4. Opcode Execution Semantics

### 4.1 `COPY` Opcode
- **Instruction:** Copies `length` bytes directly from the uncompressed baseline archive at `sourceOffset`.
- **Global Deduplication:** The baseline data can reside in any entry across the container, enabling cross-file deduplication when assets are moved, renamed, or duplicated (as proven in Experiment 7 with 99.55% deduplication).
- **Integrity Assertion:** The client hashes the copied chunk and asserts `hash === opcode.chunkHash`.

### 4.2 `PATCH` Opcode (Sub-Chunk Span Delta)
- **Instruction:** Reads `length` baseline bytes from `sourceOffset`, and applies a secondary span delta read from `deltaOffset` (length `deltaLength`) in the manifest payload pool.
- **Span Delta Format:**
  ```text
  [spanCount: uLEB128] ([offset: uLEB128] [length: uLEB128] [replacementBytes])*
  ```
- **Empirical Justification:** In Experiment 6, editing a 12-byte section of an 8 KB chunk generated a span delta of only 19 bytes, compared to transmitting the full 7,004 byte raw chunk.
- **Integrity Assertion:** The patched chunk is hashed and asserted against `opcode.chunkHash`.

### 4.3 `INSERT` Opcode
- **Instruction:** Reads `length` bytes of novel data directly from `payloadOffset` in the manifest payload pool.
- **Integrity Assertion:** The inserted bytes are hashed and asserted against `opcode.chunkHash`.

---

## 5. LEB128 Variable-Length Binary Encoding Specification

Fixed 64-bit integer fields waste substantial space for small numbers (e.g. 4096-byte chunk sizes or small delta offsets). `pak-delta` serializes all integer fields using unsigned Little-Endian Base 128 (uLEB128):

```text
Each byte contains 7 bits of integer payload (Bits 0..6) and 1 continuation bit (Bit 7).
Bit 7 = 1: More bytes follow in the integer.
Bit 7 = 0: Terminal byte of the integer.
```

### uLEB128 TypeScript Implementation
```typescript
export function encodeULEB128(value: number): Uint8Array {
  const bytes: number[] = [];
  let val = Math.floor(value);
  do {
    let byte = val & 0x7F;
    val >>>= 7;
    if (val !== 0) {
      byte |= 0x80;
    }
    bytes.push(byte);
  } while (val !== 0);
  return new Uint8Array(bytes);
}

export function decodeULEB128(buffer: Uint8Array, offset: number): { value: number; bytesRead: number } {
  let result = 0;
  let shift = 0;
  let bytesRead = 0;
  while (offset + bytesRead < buffer.length) {
    const byte = buffer[offset + bytesRead];
    bytesRead++;
    result |= (byte & 0x7F) << shift;
    if ((byte & 0x80) === 0) break;
    shift += 7;
  }
  return { value: result, bytesRead };
}
```

### Opcode Stream Byte Layout
Each opcode is serialized as:
- **Opcode Tag:** 1 byte:
  - `0x00`: `COPY`
  - `0x01`: `PATCH`
  - `0x02`: `INSERT`
- **Fields:** Follow immediately as uLEB128 integers:
  - For `COPY`: `sourceOffset (uLEB128)`, `length (uLEB128)`, `chunkDictionaryIndex (uLEB128)`.
  - For `PATCH`: `sourceOffset (uLEB128)`, `length (uLEB128)`, `chunkDictionaryIndex (uLEB128)`, `payloadOffset (uLEB128)`, `payloadLength (uLEB128)`.
  - For `INSERT`: `payloadOffset (uLEB128)`, `length (uLEB128)`, `chunkDictionaryIndex (uLEB128)`.

**Empirical Result (Experiment 12):** Packaging 500 opcodes with fixed 64-bit binary took 22,500 bytes (45 bytes per opcode). With uLEB128 encoding, the identical instructions consumed only **4,500 bytes (9 bytes per opcode)**, delivering an **80.00% metadata reduction**.

---

## 6. Multi-Tier Integrity Verification Hierarchy

The client assembler executes a 4-tier verification protocol:
1. **Tier 1 (Baseline SHA-256):** Verifies the source archive matches `manifest.sourceArchiveSha256` before performing any decompression or slicing.
2. **Tier 2 (Chunk SHA-256):** Verifies each extracted, patched, or inserted chunk payload against `chunkHash` before emitting into the entry reconstruction stream.
3. **Tier 3 (Entry CRC32):** Verifies the finalized uncompressed entry stream matches `entry.crc32`.
4. **Tier 4 (Target Archive SHA-256):** Verifies the fully reconstituted target container matches `manifest.targetArchiveSha256` down to the exact bit.