zlib: reject reset while a zstd frame is incomplete

[xia-chao]

Calling reset() on a zstd compressor while a frame is still in progress left
the stream in a state where it produced output that could not be decompressed,
without reporting anything. This makes reset() throw instead.

Fixes #66087.

The problem

reset() drops the state of the frame currently being compressed. Any bytes
that were already written out — by flush(), or by an earlier write() that
filled the output buffer — cannot be taken back, so they stay at the start of
the output stream. The next frame is then appended to that fragment, and the
result decodes as corruption.

stream.write(Buffer.from('hello'));
await new Promise((r) => stream.flush(r));  // emits 14 bytes, frame unfinished
stream.reset();                             // frame state dropped here
stream.end(Buffer.from('world'));           // emits another 14 bytes, new frame
// 28 bytes total: fragment + complete frame -> ZSTD_error_corruption_detected

Without the reset() the same input produces 22 bytes and decodes to
helloworld, because end() continues the existing frame (8 bytes) instead of
starting a new one (14 bytes).

zstd requires buffers to be fully flushed before a new compression job starts —
see ZSTD_compressStream2 in deps/zstd/lib/zstd.h:

Before starting a new compression job, or changing compression parameters, it
is required to fully flush internal buffers.

ZSTD_compressStream2 returns non-zero while a frame is unfinished, which is
how a caller is meant to detect this. The compressor context was not keeping
track of that.

The fix

ZstdCompressContext now records whether the current frame has completed. A
frame is complete once ZSTD_compressStream2 has been called with ZSTD_e_end
and returned 0. ResetStream() refuses with ERR_ZLIB_INCOMPLETE_FRAME
otherwise:

Error [ERR_ZLIB_INCOMPLETE_FRAME]: Cannot reset a zstd stream with an incomplete
frame; end the frame or discard the output produced so far

This follows the existing behaviour for the other invalid reset case, where
reset() during a write already throws.

After the change the repro above reports the error instead of emitting a
corrupt stream.

What is not changed

• reset() before any write still works.
• flush() followed by end() still works and still produces a valid stream.
• reset() on a finished frame is unaffected.

Tests

test/parallel/test-zlib-zstd-reset-incomplete-frame.js covers the reset
mid-frame case, the valid flush() + end() sequence, and the valid
reset() before writing.

Ran against a --debug-node --debug-symbols build on Linux x86_64:

$ ./out/Release/node --test test/parallel/test-zlib-zstd-reset-incomplete-frame.js
✔ ZstdCompress reset throws when a frame is incomplete
✔ ZstdCompress flush followed by end still produces a valid stream
✔ ZstdCompress reset before any write still works
ℹ pass 3
ℹ fail 0

Existing zlib tests:

$ ./out/Release/node --test test/parallel/test-zlib-zstd.js \
    test/parallel/test-zlib-zstd-reset.js \
    test/parallel/test-zlib-zstd-flush.js \
    test/parallel/test-zlib-reject-garbage-after-end.js \
    test/parallel/test-zlib-zstd-pledged-src-size.js \
    test/parallel/test-zlib-zstd-dictionary.js
ℹ pass 19
ℹ fail 0

Notes

gzip and brotli hit the same class of problem for the same call sequence (their
output is undecodable too). I kept this PR to zstd because that is where the
failure is completely silent, but I am happy to look at the other two codecs
separately if that is wanted.

[github-actions]

Welcome to Node.js, and thank you for your first contribution!

Before review, please take a moment to read:

• the guide for first-time contributors
• the contribution and automation policies
• the pull request guide
• the AI use policy
• the Code of Conduct

Please make sure every commit is signed off. For a first pull request, GitHub Actions require collaborator approval and Jenkins CI must be started by a collaborator or triager, so an initial wait is normal.

[xia-chao]

欢迎来到Node.js的世界，感谢您的首次贡献！

在进行审查之前，请花点时间阅读以下内容：

• 新手投稿指南
• 贡献和自动化政策
• 拉取请求指南
• 人工智能使用政策
• 行为准则

请确保每次提交都已签名确认。对于首次提交的拉取请求，GitHub Actions 需要协作者批准，Jenkins CI 也必须由协作者或问题处理人员启动，因此初始等待是正常的。

I'm not a beginner, do you know?

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.28%. Comparing base (513981d) to head (2a6a92c).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #66088      +/-   ##
==========================================
- Coverage   90.28%   90.28%   -0.01%     
==========================================
  Files         790      790              
  Lines      272043   272059      +16     
  Branches    51945    51942       -3     
==========================================
+ Hits       245619   245631      +12     
+ Misses      16931    16930       -1     
- Partials     9493     9498       +5     

Files with missing lines	Coverage Δ
src/node_zlib.cc	80.36% <100.00%> (+0.57%)	⬆️

... and 28 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[xia-chao]

@MikeMcC399
Thank you for pointing out the formatting issue!

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/77589/

[MikeMcC399]

I'm not a beginner, do you know?

Your other PR #65521 is approved, but it seems to have stalled.

PR #65521 has now landed, so congratulations on being upgraded to "Contributor" 🎉 !
You should not see the message for "First-time contributors" on any new PRs you open.

[xia-chao]

@MikeMcC399
Thank you very much for your help. I really want to contribute my efforts to Node
But during the first PR, it hit me a bit and I haven't been merged for so long
Thank you, we will also meet frequently under the node repository in the future

[xia-chao]

@MikeMcC399
The Jenkins task has failed again, and I am unable to view the Jenkins console! Perhaps there was a failed underlying test!
Could you help me? Check which specific test failed.

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/77613/

[MikeMcC399]

The Jenkins task has failed again, and I am unable to view the Jenkins console! Perhaps there was a failed underlying test!
Could you help me? Check which specific test failed.

As described in the Pull requests > Continuous integration testing guide section, Jenkins CI testing located at https://ci.nodejs.org/ is only accessible to team members.

The previous run was started by @inoway46, however I have triggered a re-run of the failing job. If that fails again, it would warrant a closer look, if not, then it was probably a flaky test.

[inoway46]

@xia-chao Sorry for catching this after the full CI had completed.

[inoway46]

@xia-chao The implementation looks good overall. I left a few inline comments for some final cleanup.

Nit: since commit-queue-squash label squashes into the first commit, the final commit would likely be credited to bun-unsafe. If you'd prefer it to be credited to xia-chao, you may want to squash the commits locally and update the author/sign-off identity before landing.

[xia-chao]

Thanks @jasnell and @inoway46.

I applied the latest comments and squashed. I don't plan to make more changes unless a collaborator asks.

[inoway46]

Thanks for addressing the comments. LGTM!

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/77720/