Repository navigation
Conversation
panva
added
flaky-test
nodejs-github-bot
added
c++
Member
Author
|
This locally reproduces the X509 failure that flakes in CI. Eight workers decode an RSA-PSS certificate concurrently with background CA loading. On main I get Source// Flags: --expose-internals
'use strict';
const common = require('../common');
if (!common.hasCrypto)
common.skip('missing crypto');
const { hasOpenSSL, isBoringSSL } = require('../common/crypto');
if (isBoringSSL || !hasOpenSSL(3))
common.skip('OpenSSL 3 providers are required');
const { getOptionValue } = require('internal/options');
if (getOptionValue('[ssl_openssl_cert_store]'))
common.skip('off-thread CA loading is disabled');
const assert = require('node:assert');
const { spawnSync } = require('node:child_process');
const { X509Certificate } = require('node:crypto');
const { once } = require('node:events');
const {
Worker,
parentPort,
workerData,
} = require('node:worker_threads');
const fixtures = require('../common/fixtures');
if (workerData?.barrier) {
parentPort.postMessage('ready');
Atomics.wait(workerData.barrier, 0, 0);
const cert = new X509Certificate(workerData.pem);
assert.strictEqual(cert.publicKey.asymmetricKeyType, 'rsa-pss');
assert(Buffer.isBuffer(cert.toLegacyObject().pubkey));
} else if (process.argv[2] !== 'child') {
// Each process needs cold OpenSSL state. Repeat the concurrent first use to
// exercise different schedules of certificate loading and key decoding.
for (let i = 0; i < 3; i++) {
const child = spawnSync(process.execPath, [
...process.execArgv,
__filename,
'child',
], { encoding: 'utf8' });
assert.strictEqual(child.signal, null, child.stderr || child.stdout);
assert.strictEqual(child.status, 0, child.stderr || child.stdout);
}
} else {
const { internalBinding } = require('internal/test/binding');
const binding = internalBinding('crypto');
const pem = fixtures.readKey('rsa_pss_cert_2048.pem');
const barrier = new Int32Array(new SharedArrayBuffer(4));
async function run() {
const ready = [];
const exited = [];
// Load the workers before starting CA loading so their first key decodes
// can overlap with the background thread's first certificate decodes.
for (let i = 0; i < 8; i++) {
const worker = new Worker(__filename, {
workerData: { pem, barrier },
});
ready.push(once(worker, 'message').then(([message]) => {
assert.strictEqual(message, 'ready');
}));
exited.push(once(worker, 'exit').then(common.mustCall(([code]) => {
assert.strictEqual(code, 0);
})));
}
await Promise.all(ready);
binding.startLoadingCertificatesOffThread();
Atomics.store(barrier, 0, 1);
Atomics.notify(barrier, 0);
await Promise.all(exited);
}
run().then(common.mustCall());
} |
Collaborator
|
Review requested:
|
Collaborator
richardlau
approved these changes
Oct 21, 2026
panva
added
author ready
github-actions
Bot
removed
the
resume-ci
Collaborator
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #66903 +/- ##
=======================================
Coverage 90.57% 90.57%
=======================================
Files 797 797
Lines 278502 278498 -4
Branches 53540 53540
=======================================
Hits 252247 252247
- Misses 16592 16601 +9
+ Partials 9663 9650 -13
🚀 New features to boost your workflow:
|
lpinca
approved these changes
Oct 21, 2026
panva
added
the
commit-queue
Member
Author
|
I'll let CQ let me know when 48 hours are up (it'll fail on the account of the revert commit message), then i'll land this manually. |
nodejs-github-bot
added
commit-queue-failed
Collaborator
Commit Queue failedThe pull request was removed from the Commit Queue and labeled
commit-queue-failed
Full Commit Queue output |
panva
removed
the
commit-queue-failed
Member
Author
|
Landed in 03cca65...6511fb8 |
nodejs-github-bot
pushed a commit
that referenced
this pull request
Oct 30, 2026
OpenSSL 3 builds its table of algorithm names on first use. When 43d3fe9 moved that first use after other threads had started, an RSA-PSS key decoded while the table was being built could fail with `ERR_OSSL_EVP_DECODE_ERROR`, and every later RSA-PSS decode in the process failed too. #66903 reverted that change. The new test fails if it comes back: Workers decode an RSA-PSS certificate while the root certificate loader or a thread pool crypto job uses OpenSSL for the first time. It is based on a reproduction by Filip Skokan in #66903. Refs: #66903 Refs: #66935 Signed-off-by: Christian Aurich Zanettini Martins <christian.aurichzm@gmail.com> PR-URL: #66966 Reviewed-By: Filip Skokan <panva.ip@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This reverts commits 43d3fe9 and 6ecfd65.
I keep finding fallout from this change. It already needed special handling for AIX and OpenSSL provider initialization, and changed behavior with unavailable DRBGs and
--secure-heap. Since landing, I've had to marktest-worker-init-failureflaky on SmartOS because of descriptor-dependent entropy failures. Now I'm seeing intermittenttest-crypto-x509failures across GHA and Jenkins.The X509 failures come from removing eager OpenSSL initialization: background CA loading can race with key decoding while OpenSSL is still populating its algorithm names. Reverting restores initialization before that concurrency starts.
The startup improvement isn't in my opinion worth continuing to chase these failures.