Skip to content

WASM zipstream uses unnecessary data descriptors for STORED entries #337

@pflynn-virtru

Description

@pflynn-virtru

Summary

The TDF WASM zipstream writer (opentdf/platform: sdk/experimental/tdf/wasm/zipstream/) sets bit 3 (data descriptor flag) on STORED payload entries and writes zeros for CRC32/sizes in the local file header, even though the actual values are known before the header is written.

This causes Java's ZipInputStream to reject the output with:

java.util.zip.ZipException: only DEFLATED entries can have EXT descriptor

Root Cause

In segment_writer.go, the payload entry unconditionally uses data descriptors:

header := LocalFileHeader{
    GeneralPurposeBitFlag: dataDescriptorBitFlag, // 0x08
    Crc32:                 0,
    CompressedSize:        0,
    UncompressedSize:      0,
}

But encrypt.go computes the full ciphertext and CRC32 before calling WriteSegment:

fullCT, err := hostcrypto.AesGcmEncrypt(dek, plaintext)
crc32Sum := crc32.ChecksumIEEE(fullCT)
header, err := sw.WriteSegment(ctx, 0, uint64(len(fullCT)), crc32Sum)

The sizes and CRC32 are available — the writer just doesn't use them in the local file header. The manifest entry already handles this correctly (no bit 3, sizes written directly in the header).

Impact

  • ZipInputStream (streaming parser) rejects the ZIP entirely for STORED + data descriptor
  • ZipFile (central-directory parser) works fine as a workaround
  • Other ZIP implementations may also reject this combination

Current Workaround

In the java-sdk wasm-host module, we use ZipFile (temp file + central directory) instead of ZipInputStream to parse TDF output.

Proposed Fix

In opentdf/platform, modify segment_writer.go to write the known CRC32 and sizes into the local file header and clear bit 3, matching how the manifest entry is already handled. Remove the data descriptor write for the payload entry.

Upstream

The fix belongs in opentdf/platform sdk/experimental/tdf/wasm/zipstream/zipstream/segment_writer.go.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions