Skip to content

[SER-2815] Expose the remote-URL downloader settings - #167

Open
0mar-K wants to merge 2 commits into
masterfrom
omar/ser-2815-downloader-config
Open

0mar-K wants to merge 2 commits into
masterfrom
omar/ser-2815-downloader-config

Conversation

@0mar-K

@0mar-K 0mar-K commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Companion to https://linear.app/nutrient/issue/SER-2815 and the Document Engine PR.

Summary

Document Engine 1.19.0 makes its remote-URL downloader configurable. The reference architecture points customers at this chart, so without these values the feature is unreachable for anyone deploying that way.

Defaults match what Document Engine used while these were hardcoded, so an existing deployment is unaffected.

On hold until Document Engine 1.19.0 is out. Tracked as a release TODO in the monorepo.

Two things for whoever merges this. The 1.19.0 release lands as its own commit that bumps appVersion and takes the next chart minor, which by the pattern of 1.16.0 to 8.4.0, 1.17.0 to 8.5.0 and 1.18.0 to 8.6.0 will be 8.7.0. So this PR needs renumbering to 8.8.0 once that lands. And the changelog entry deliberately carries no "requires 1.19.0" note, because merging after the appVersion bump makes it true by ordering and the chart has never version-gated an entry.

Fix

  • values.yaml - config.urlFetchPoolSize, config.urlFetchQueueTimeoutSeconds, config.urlFetchQueueMaxSize, next to the existing urlFetchTimeoutSeconds
  • templates/configmap.yaml - maps them, seconds to milliseconds like its neighbours
  • Chart.yaml - 8.6.3 to 8.7.0, backward compatible addition
  • CHANGELOG.md, README.md (helm-docs), values.schema.json (helm schema)

QA

  • helm template with --set config.urlFetchPoolSize=32 renders DOWNLOADER_WORKER_POOL_SIZE: "32", and 5 seconds renders DOWNLOADER_BROKER_CLIENT_QUEUE_TIMEOUT: "5000"
  • --set config.urlFetchPoolSize=0 is rejected by the schema, matching the validation in Document Engine
  • helm lint passes

Document Engine 1.19.0 makes its remote-URL downloader configurable, so
the chart needs the matching values. Without them a customer following the
reference architecture cannot reach the new settings at all.

Adds `config.urlFetchPoolSize`, `config.urlFetchQueueTimeoutSeconds` and
`config.urlFetchQueueMaxSize`, mapped to `DOWNLOADER_WORKER_POOL_SIZE`,
`DOWNLOADER_BROKER_CLIENT_QUEUE_TIMEOUT` and
`DOWNLOADER_BROKER_CLIENT_QUEUE_MAX_SIZE`. Defaults match what Document
Engine used while these were hardcoded, so an existing deployment is
unaffected.

Companion to https://linear.app/nutrient/issue/SER-2815
@0mar-K
0mar-K marked this pull request as ready for review September 16, 2026 10:49
@0mar-K
0mar-K requested review from a team, douglashill and eteubert September 16, 2026 10:49
@0mar-K 0mar-K added the documentation Improvements or additions to documentation label Sep 16, 2026
@zrzka

zrzka commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Note: values.schema.json was hand-edited to mirror the existing workerPoolSize and urlFetchTimeoutSeconds entries, because the helm schema plugin is not installed here. Worth regenerating if you have it.

This is clearly coming from an agent, which is fine, but it also shows that you don't have all the tools required installed. Install them and generate the schema, do not hand edit. Any subsequent generation can produce more changes, make following PRs unnecessary big.

We should also put this PR on hold, add a TODO item here, and merge when DE 1.19 is out (unless we want to have this option available in nightly).

@0mar-K
0mar-K marked this pull request as draft September 17, 2026 07:32
The chart has never gated an entry on a Document Engine version. This PR waits
for the 1.19.0 release commit, which bumps appVersion first, so the requirement
holds by ordering and the entry can read like every other feature entry.
@0mar-K

0mar-K commented Sep 17, 2026

Copy link
Copy Markdown
Contributor Author

Holy! Yea, I didn't know about this at all. First time to touch such changes. Tried to weigh it right with the agent but apparently failied 😅 Many thanks for the correction, @zrzka

All three done. Schema is generated, PR back to draft, TODO is on #58563

Waiting for 1.19 rather than nightly. I set both variables on released 1.18.1 and nothing changed, still 16 and shedding at 5s, so merging early would just hand people a dead knob. Is that the right way?

Also put up #168 for your review. It's for the regeneration command part.

@0mar-K
0mar-K marked this pull request as ready for review September 17, 2026 09:57
@0mar-K
0mar-K requested a review from zrzka September 17, 2026 09:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants