Skip to content

Unified client generation: one client for a module and for a plain Java program - #23

Draft
eunomie wants to merge 28 commits into
dagger:mainfrom
eunomie:unified-client-lead-javasdk-a1705c6e
Draft

Unified client generation: one client for a module and for a plain Java program#23
eunomie wants to merge 28 commits into
dagger:mainfrom
eunomie:unified-client-lead-javasdk-a1705c6e

Conversation

@eunomie

@eunomie eunomie commented Sep 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

A Dagger module written in Java could already call another Dagger module. A plain Java program could not: dagger module client add java outside a module was refused, and the SDK could not open an engine session on its own.

This makes both callers get the same generated bindings for a given module, out of one generator.

What changes

A module that Java code calls is a target. Each target's types are now generated into a package of their own, io.dagger.client.modules.<target>, from that target's own client-facing schema and nothing else — so the files do not depend on what else is generated beside them.

The way into a target moves there too. A field a target contributes to a core type has no class of its own, because Java generates a type once in one package, so it becomes a static method on the target's own root type carrying the receiver it was reached through. The session receiver stays implicit:

import static io.dagger.client.modules.sdkhelpers.SdkHelpers.sdkHelpers;

sdkHelpers().moduleManifest()

One import is the whole of the integration, and a client is something you import rather than something you reach through a global.

Core is one of those packages. It is generated into io.dagger.client.modules.core and reached the same way:

import static io.dagger.client.modules.core.Core.core;

core().container()

dag().container() no longer exists. What makes that possible is that Dagger.dag() stops returning a generated type: Session is hand-written, owns the connection, and is what every entry point takes. Client is gone and AutoCloseableSession replaces AutoCloseableClient. So core(dag()) and sdkHelpers(dag()) are the same shape, and no generated package is privileged over another.

One asymmetry is left, and it is link-time rather than API. The hand-written runtime still imports generated core — Telemetry needs FunctionCall, which is inherent, and the generated Version constant is emitted there, which is merely misfiled. Both are recorded in the design doc.

A client package also carries the descriptor it was generated against — a workspace path or a git reference with the commit it resolved to — and serves its own module before selecting anything on it. A Maven project that is not a Dagger module gets the same bindings under dagger/, plus one marked profile inserted into the project's own pom.xml so its build compiles the generated sources. Connection learns to start dagger session when there is none, so java -jar works with no wrapper command.

The claim, and how it is checked

e-2-e:clients-are-one-artifact-check generates the same target into a Dagger module holding it alone, and into a standalone project holding it alongside another, and asserts the two client packages have the same directory digest. That is the feature stated as a measurement rather than an intention.

e-2-e:client-optional-args-check is the other one worth naming: its fixture is Dagger module source that gets compiled, and it calls clientDefaults() after a single static import. The call shape is verified by a compiler, not by an assertion on a string.

e-2-e:standalone-compiles-check runs mvn package on the generated standalone tree in a container with no engine in reach and no Dagger CLI on PATH, then asserts the class files exist. Every other standalone check reads the generated files rather than building them, so a source naming a type the generator emitted as bare text and never imported would pass all of them.

engine-e-2-e:java-client-runtime-check generates a Dagger module that declares a git client and then calls it with dagger call. It is the only check that runs a generated module, so without it the serve a client package performs on first use is unproven in both directions — every other check stops at generating or compiling.

Which targets load themselves

target in a module standalone
git the client serves it the client serves it
workspace path the engine serves it the client serves it

A module runtime has no filesystem session attachable, so it cannot resolve a workspace path; the engine serves such a target from the module's manifest instead, and the generated package emits no serve at all. A git module is reachable from anywhere, which is why a git target's package comes out byte for byte identical on both sides today, and why that is the target the artifact check compares.

That last cell is the only asymmetry left. It is an engine limitation rather than an SDK choice, and closing it is servesWorkspacePaths: true in one driver function. Nothing here waits on it: a local client inside a module never asks, and a standalone program's currentWorkspace.moduleSource(path) already works on the released engine.

On the engine

No engine change was needed. ModuleSource.clientSchemaIntrospectionJSON (dagger/dagger#13646) already returns core plus exactly one module reached as dag.<moduleName>, which is the same primitive for a module's dependency and for a standalone target, and Module.serve may be called more than once per session — its doc string in dagger/dagger still says otherwise, which is worth a separate fix upstream.

Also worth recording: the released v1.0.0-beta.13 carries the merged module-max interface from dagger/dagger#13992, so the whole end-to-end suite runs against it directly in about ninety seconds. The README no longer claims the SDK needs an engine built from that pull request.

Breaking change

Every Java module that uses this SDK has to change, in two ways.

A target's types move from io.dagger.client.<Type> to io.dagger.client.modules.<target>.<Type>, and so does the way in: dag().target() becomes target() after a static import. The <Target>Arguments holder moves with its method, onto the target's root type.

Core moves the same way. io.dagger.client.Container becomes io.dagger.client.modules.core.Container, and dag().container() becomes core().container(). Client no longer exists.

This is one migration rather than two on purpose. The target move already breaks every module that calls one; doing core in the same release means users edit their imports once. The README says so where a user will meet it.

Design

hack/designs/done/2026-09-13-unified-client-generation.md, added in the first patch and archived in the last. It records what was rejected and why — a flat package, a self-contained package per target with its own core, the abandoned core(dag()) session split, eager serving, and keeping the accessor on core, which this document originally proposed before the goal was restated as a client being autonomous. Its Progress section records what each review round changed.

Residual

ClientPom infers the indentation of the block it inserts from the file around it, which is roughly sixty lines more than the problem needs. It is well covered, and shrinking it was judged a worse trade than leaving it. A follow-up could also fold the generator's remaining single-schema path into the plan format.

@eunomie
eunomie force-pushed the unified-client-lead-javasdk-a1705c6e branch from df66167 to 1e5b5e8 Compare September 14, 2026 19:51
@eunomie
hack/designs: spec unified client generation
fa04c6d 
Generate the same Java client for a module whether the caller is another
module or a plain Maven project, and make a plain Java program able to open
a session and have the engine load the module it calls.

Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
sdk: apply the formatter
93913fa 
Five files drifted from what fmt-maven-plugin produces, so every build
rewrites them and every patch has to be checked for the noise. Apply it once.

Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
codegen: read @sourcemap module attribution
4f1c699 
The engine attributes every type and field a module contributes with a
@sourcemap directive naming that module. Nothing in codegen read it, so a
dependency's types were indistinguishable from core once generated.

Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
codegen: partition a schema into core and one module
b7ea798 
Core is every type no module owns, with every module-contributed field
removed, so it names no client package. A client is every type its module
owns, plus the fields that module contributes to core types, which have no
class of their own and are emitted as entry points on the module's root.

Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
codegen: read a client's entry points off its schema
8876d8b 
A module's root type is the return type of the one Query field it owns, not
its name: deriving it from the name gives E2e where the engine says E2E. A
module that owns no Query field or several, or that is reached as a core
type it does not own, is refused.

Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
codegen: map a module name to a Java package, and refuse a set it can…
a44eafa 
…not separate

A Dagger name can hold characters a Java package segment cannot, and two
distinct names can normalize to the same segment, which would silently make
one module's bindings overwrite another's.

Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
codegen: resolve type references through a registry
2a6301e 
Every generated reference named its target by simple name, which is only
correct while everything lands in one package. Route schema types, the
hand-written runtime and the type being written through a TypeRegistry, and
let CodeWriter take its package from it.

Behaviour is preserved. Generating from a real engine schema
(v1.0.0-beta.13, 114 files) before and after this patch differs in exactly
one way: executeQuery(java.lang.String.class) becomes
executeQuery(String.class), and the same for Boolean and Integer, because a
ClassName lets javapoet elide the implicit java.lang import where the
interpolated name printed it in full.

Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
sdk: make the query transport public API
21238e6 
Generated bindings are about to leave io.dagger.client for one package per
target, io.dagger.client.modules.<target>. Every generated type is built on a
QueryBuilder, chains through it, implements InputValue, merges Arguments and
converts Scalars. All of those were package-private, which was right while the
generated code sat next to them and is impossible once it does not: a class
cannot implement a non-public interface from another package.

So QueryBuilder and its chain/execute methods, its GraphQLClient constructor,
InputValue, Arguments.merge and Scalar.convert become public. QueryBuilder also
gains client(), which exposes the session a chain is attached to so that code
which has to key something on the session identity — the lazy serve registry,
chiefly — can reach it.

This deliberately reverses the decision in
hack/designs/2026-08-17-nullable-object-returns.md to keep QueryBuilder
package-private: a public transport is the price of generating into more than
one package, and the javadoc says it is not a user-facing API.

buildQuery and executeQuery(String) stay package-private: no generated code
calls either, only the tests in this package do.

Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
sdk: serve a target on first use
f268147 
A Java program reaches a target through bindings the generator wrote, but
nothing has asked the engine to load that target. Add the two pieces that do.

ModuleTarget is the descriptor the generator emits: a final module name plus
either a workspace-relative path or a git reference with the commit it resolved
to at generation time. It is sealed over one record per kind, so a descriptor
that is half workspace and half git cannot be constructed; the alternative, one
record with nullable fields and a runtime check, would move the same error from
the compiler to the first serve.

ModuleTargets.serve(session, target) is what every way into a generated client
package calls before it builds anything: until the target is served, the field
the caller is about to select does not exist in the session. It sends

  moduleSource(refString:…, refPin:…)                     for a git target
  currentWorkspace {moduleSource(path:…)}                 for a workspace one

then withName(name:…), asModule and serve. withName pins the name the schema
was generated under, so a change in how the engine derives a target's name
cannot silently produce a root field the bindings do not have.

The descriptor is passed in rather than looked up. A client package owns the
target it was generated against, and holds it as a constant; a client generated
against a target the engine serves on its own carries no descriptor and no call
to this class at all. Whether a target is served is therefore decided when the
bindings are written, not by what happens to be on the class path when they
run.

The memo is per session, not per JVM: two sessions in one process each serve.
It is keyed on the session's GraphQLClient and held weakly, so a closed session
is collectable, and behind a synchronized map because serve can be called
concurrently. Each target has its own holder, whose serve is synchronized and
marks itself done only on success. Concurrent callers for one target therefore
send one query between them, a failed serve is retried rather than remembered,
and neither blocks a serve of another target.

Failure surfaces as an unchecked exception naming the target, with the engine's
refusal as its cause. The generated accessor it runs from returns a lazy object
and declares no checked exception, so there is nowhere to put a checked one;
and swallowing it would replace a message that says why the target could not be
loaded with an "unknown field" from the next query, which says nothing.

The session is a parameter rather than a global, so a client obtained from
Dagger.connect() serves into its own session instead of into whichever one
Dagger.dag() happens to hold.

Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
sdk: open a session when there is none
2fb5902 
Connection.get read DAGGER_SESSION_PORT and DAGGER_SESSION_TOKEN and threw when
they were absent, so a plain `java -jar app.jar` could not reach an engine at
all. It also took a loadWorkspaceModules parameter and did nothing with it.
Both are fixed here.

CLISession starts `dagger session` when the environment carries no session. It
finds the CLI at _EXPERIMENTAL_DAGGER_CLI_BIN or, failing that, as `dagger` on
the PATH, and when neither resolves it says so and names both. It does not
download one: provisioning has its own release, checksum and mirror questions,
and the design names it a non-goal.

The session announces itself as one line of JSON on standard output, read under
a bounded timeout. A timeout, a line this SDK cannot parse, a port outside
1..65535, an empty token and an exit before the announcement are all errors, and
every one of them quotes what the process wrote to standard error, because that
is where the reason is. Capture stops once the handshake succeeds; after that
standard error is forwarded to the SDK logger on a daemon thread, so engine
progress reaches the user without the buffer growing for the life of the
session. Standard output is drained on a thread of its own for the same reason
the CLI needs it drained at all: an unread pipe eventually blocks the writer.

Shutdown closes the CLI's standard input first, which is how the CLI is meant
to be stopped, waits, and only then destroys forcibly. The same shutdown runs
from a JVM hook, so a session cannot outlive the process that opened it, and
close is idempotent so the hook and an explicit close cannot fight.

Connection falls back to CLISession, carries it, and closes it when the
connection closes; it passes --load-workspace-modules through when the caller
asked for it. A generated standalone client does not ask: it serves its own
targets, which is narrower and does not depend on what [modules] happens to
list. The parameter was public API that did nothing, and now it does what it
says.

Dagger.dag() becomes synchronized. It was an unsynchronized lazy singleton, and
now that the first call can start an engine, two threads racing it would start
two.

System.getenv cannot be set from inside a test JVM, so the environment reads sit
in the public entry points and the work sits behind package-private seams the
tests drive directly: Connection.get with the two values as parameters,
CLISession.resolveCLI with the configured binary and the search path, and
CLISession.start with an explicit CLI and handshake timeout. The fake CLI is a
shell script in a temp directory that exits when its standard input closes, like
the real one.

Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
codegen: generate every package a plan names in one pass
3404869 
One invocation now takes a plan directory holding a core schema and one
schema per target, and emits the core package plus one package per target
against a single type registry. A target's own types and the fields it
contributes to core types both go into its package; core keeps neither, and
names no client package.

Each contributed field becomes a static method on the target's root type,
named after the field. A field on Query has no receiver, so it comes in two
forms: one taking the session to reach the target in, one over the session
Dagger.dag() holds. A field on any other core type takes that type as its
receiver and reads the session off it, because serving into a session the
receiver does not belong to lands the target beside the caller's query
rather than in it.

An entry point asks for its target to be served first. Outside a module
nothing has served it and the field the entry point selects does not exist;
inside one the engine served it before the module ran, no descriptor is
registered, and the call returns without a query.

Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
codegen: take core from the targets when a scope has none
981f998 
A standalone scope has no module-facing schema, so core comes from its
targets: every target's client-facing schema carries a full copy of it.
The copies have to agree, and the check is the point — the engine renders
core through each module's declared engine version, so two targets pinned to
different ones hand back two different cores and one of them would compile
against a core it was never generated for. What the targets do not agree
on is each target's own contributions, and those are emitted in that
target's package rather than in core.

A module scope has a core of its own, and the rule there is coverage rather
than equality: the engine hides part of core from module code, so a target's
client-facing core carries types and fields the module-facing one does not.

Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
codegen: add a client-pom goal to register generated clients
ccc90b3 
A standalone scope's pom.xml belongs to the user, so `dagger generate` has
nowhere to put the build configuration the sources under dagger/ need. The
new client-pom goal writes exactly one element into that file: a
dagger-clients profile that activates on the presence of
dagger/src/main/java, adds it as a source root through
build-helper-maven-plugin, and declares the SDK's run-time dependencies at explicit versions, since the
project inherits no dependency management from this repository. It stops
short of a logging implementation, which is the application's choice.

The edit splices text over the original bytes instead of re-serializing a
parsed document, so comments, attribute layout and line endings survive.
A profile the goal wrote before, recognised by a marker comment, is
replaced rather than duplicated; a dagger-clients profile without that
marker is the user's, and the goal refuses to touch it.

Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
prebuilt: rebuild the codegen plugin
ad3585a 
Generation seeds the local Maven repository from prebuilt/m2 whenever it
exists and never compiles the plugin sources in that case, so the driver
patches that follow would run the old generator without this.

Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
java-sdk: generate one package per target
9d4d221 
Code generation moves into a Codegen type of its own, driven by a plan: the
schema core comes from, and one entry per target. A module scope builds that
plan from its own module-facing schema plus each target's client-facing one,
so a target's bindings depend on that target's schema and on nothing else.

A target's types move from io.dagger.client to
io.dagger.client.modules.<target>, and so does the way in: the field a target
was reached through on Query becomes a static method on the target's own root
type. dag().target() becomes target() after a static import, and core gains
nothing when a target is added.

Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
java-sdk: generate standalone client scopes
d76f797 
A Maven project that is not a Dagger module now gets bindings for the modules
it records, instead of being refused. They are the bindings a module gets for
the same target, from the same plan and the same generator, so the package is
identical either way — a check compares the two digests.

What the project needs on top is what a module does not: a descriptor saying
how to reach each target at run time, because the engine has served nothing
for it, and one marked profile in its own pom so its build compiles and
packages the generated sources.

Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
README: document standalone clients
d4ffaaf 
Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
hack/designs: archive the unified-client-generation design
85cf17c 
Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
packager: hold the Maven lock across install and export
1e50190 
The cache mount is LOCKED per exec, not across a chain of them, so with the
install and the export in two execs the lock is released in between. Under one
`dagger check` the unit-test check installs the same plugin concurrently,
without the fixed output timestamp, and can overwrite the jar in the shared
repository before the export copies it. What landed under prebuilt/ then
carried wall-clock entry timestamps and never matched the committed bytes, so
packager:generate failed on a race rather than on anything in the diff.

The race is inferred rather than observed: the check is red and green on the
same tree in CI, a rebuild against a never-used cache volume reproduces the
committed bytes exactly, and a rebuild against one primed by a real module
generation does too. What is left is a concurrent writer, and the unit-test
check is the only one there is.

Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
eunomie force-pushed the unified-client-lead-javasdk-a1705c6e branch from 1e5b5e8 to 1e50190 Compare September 14, 2026 20:53
@eunomie
codegen: name the runtime package apart from the generated core
4e979b0 
The registry resolves two different things through one name: a schema type,
which is about to move, and a hand-written runtime class, which is not. Hold
them apart before either moves, so the move is a change of one constant.

No generated output changes: both packages are io.dagger.client.

Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
java-sdk: generate core as a client package
6802910 
Core was the one generated thing that was privileged: it landed flat in
io.dagger.client beside the hand-written runtime, and it was reached as
dag().container() while every other client is reached through a static method on
its own root type. Two shapes for one idea.

Core now goes to io.dagger.client.modules.core, under the same root as every
other client, and is reached the same way: core(dag()).container(), or core()
over the ambient session.

What makes that possible is that Dagger.dag() stops returning a generated type.
Session is hand-written, owns the connection and exposes the query builder every
generated package chains from, and is what each entry point now takes. Nothing
generated is privileged after this: core(dag()) and myModule(dag()) are the same
shape, and io.dagger.client holds hand-written code alone.

dag().container() no longer exists.

Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
codegen: refuse a module named core
ec08a6e 
Core is generated into io.dagger.client.modules.core, so a module whose name
normalizes to that segment would overwrite it, or be overwritten by it,
depending on emission order. Refuse it where the segment is decided, and say
which package is taken and why, rather than leaving the loser to be found by
whatever fails to compile.

Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
templates: reach core through its own package
4b1f6e0 
A scaffolded module is the first Java anyone writing against this SDK reads, so
it has to show the shape the SDK now has: core imported from its own package and
entered by name, not reached through the session.

Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
e2e: assert core where it now lives
f9dbe1a 
Core is generated under modules/core and entered as core(), so the fixtures that
compile against it and the assertions that read it move with it. Two new
assertions: nothing is emitted flat beside the hand-written runtime any more,
and the entrypoint the annotation processor writes enters core by name.

Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
engine-e2e: pin core's package in the initialized module
8cdc722 
The dev-SDK check is the only one that initializes a module on a real engine and
then calls it, so it is where the layout can be pinned against the engine rather
than against a fixture: core is generated under modules/core, and the module the
template scaffolds runs.

Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
README: document core as a client package
b15f994 
Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
hack/designs: record core as a client package
1bc5e07 
The design rejected making core a target like any other, on the grounds that the
symmetry was not worth rewriting every call site. That call is reversed: the
owner asked for it, and the SDK has already broken compatibility in this series,
so the rewrite is paid for once rather than twice.

The rejected alternative is kept and rewritten rather than deleted. A design
record that quietly loses its own reversals is not a record.

Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
prebuilt: rebuild the codegen plugin for the core move
bf1b2d0 
The second rebuild in this series, and not a duplicate of the first. The
committed jar has to match the generator at the point in the stack where it is
used: the earlier rebuild matches the plan-driven generator, and every patch
between the two generates core flat, correctly, with that jar. This one matches
the generator that emits core into its own package.

Folding the two into one would put a jar that emits modules/core underneath the
patches that still expect core flat, so the intermediate patches would generate
one layout and assert the other.

Generation seeds the local Maven repository from prebuilt/m2 whenever it exists
and never compiles the plugin sources in that case, so without this every scope
would keep generating with the old core layout while the sources say otherwise.

Signed-off-by: Yves Brissaud <yves@dagger.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@eunomie