Commit e361fa6
feat(symbols): key Android uploads by R8's map id, and upload dSYMs from an Xcode build (#769)
* feat(symbols): find the Android mapping and app version in the build
`ldcli symbols upload --type android` needed a --path pointing at a directory
holding mapping.txt and an --app-version matching what the app reports, so
every project had to add build script code to stage the mapping somewhere and
plumb its version into CI.
The Android Gradle Plugin already writes both: R8's mapping at
<module>/build/outputs/mapping/<variant>/mapping.txt, and the version it
packaged in output-metadata.json beside the APK. Read them, and the command
works from an Android project root with no flags and no build script at all.
Several obfuscated variants is an error naming them rather than a guess, since
only one of them is the build being shipped.
An Android mapping is now also stored as mapping.txt however deep it was found.
Symbolication reads it at <lane>/mapping.txt, so a mapping uploaded from a
nested path was previously keyed somewhere nothing looks.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(symbols): read the Android symbols id out of the packaged app
A build that stamps a content-derived symbols id into assets/ld_symbols_id.txt
had to also stage a mapping.txt.symbolsid sidecar next to a copy of the mapping,
purely so the upload could be keyed by the same id the app reports.
The packaged APK/AAB already carries that asset, and it is the app that will
run: what it carries is exactly what will be reported. Reading it there needs
nothing handed over by the build and leaves no way for the two to disagree, so
the staging step goes away and a stamped build uploads on the Symbols Id Lane
with no flags.
An id that does not match the 32-hex-char shape the SDK reports is ignored
rather than keyed on, since it names a lane nothing would ever ask for.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(symbols): gzip what an upload sends
A release R8 mapping is tens of megabytes of text and every build pushes it
again: the e2e app's is 61.3 MB, which gzips to 5.0 MB in under half a second.
React Native source maps and Apple symbol maps compress on the same order.
Each object is marked Content-Encoding: gzip, so it stays self-describing —
storage returns the header on read, an HTTP client inflates it in transit, and
the backend inflates whatever still arrives compressed.
Artifacts held in memory are compressed before an upload URL is asked for rather
than at the point of sending, because the digest that proves an object is
already stored is compared against the ETag of the stored bytes; hashing the
artifact instead would never match and every source bundle would be re-sent.
Files are compressed as they are read, streamed through the compressor so a
large mapping is never held in memory whole.
Anything that comes out of gzip no smaller is sent as it is, so a .srcbundle,
which already gzips its own entries, is not wrapped a second time.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(symbols): build the R8 index in the CLI instead of uploading the mapping
An Android upload sent R8's mapping.txt as it was, tens of megabytes of text, and
the backend parsed it into a random-access index on the first crash of every build
that ever crashed. The mapping is here, on the machine that just produced it, so
the index is built here too and the text is never stored: `upload` and `generate`
now write mapping.v1.index to the Symbols Id and Version lanes.
The mapping is streamed through r8index.EncodeFrom rather than parsed into memory,
so a build machine does not have to find hundreds of megabytes to produce a few.
A mapping that yields no index is an error: symbolication reads only the index, so
storing anything else would leave a build looking like it has symbols while every
crash arrives obfuscated.
internal/symbols/r8index is a verbatim copy of the backend's package, the way
dsymmap and srcbundle already are, and srcbundle.Builder now compresses on add so
that a per-class bundle costs its compressed size rather than its raw one.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(symbols): pin the R8 index format with a golden fixture
The r8index package exists twice, here and in the backend that reads what this
writes, and an index is only interchangeable if the two copies agree byte for byte.
Two copies cannot import each other's tests, so what they share is a fixture: a
readable mapping and the exact index bytes it encodes to, identical in both repos.
Either side drifting now fails here rather than in production, where the symptom
would be an unreadable index or a silently wrong frame.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(symbols): cross-check a CLI-built index against its mapping
The golden fixture proves the encoder here still produces the bytes both repos
agreed on; this proves the artifact a build machine wrote answers like the mapping
it came from, which is the guarantee symbolication actually rests on. Opt-in via
R8_MAPPING and R8_INDEX, since it needs a real release build.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(symbols): name the real mapping to check against with a flag
Keeps the r8index copy in step with the backend's, where reading these paths from
R8_MAPPING and R8_INDEX fails a CI check that bans os.Getenv anywhere in the
backend. A test flag is what the package already reaches for anyway (-update, for
the golden fixture).
end-of-file-fixer runs here too, through pre-commit, and it appended a newline to
the backend's copy of the golden index — a byte that moves the footer the reader
locates the index by, and breaks the byte-for-byte agreement the fixture exists to
prove. It cannot tell a binary file from a text one, so *.index is excluded.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(symbols): accept --source-path on `symbols generate`
Generating an Android source bundle reads sourcePathFlag, which generate never
registered: --source-path was rejected outright, so sources could only be scanned
from the working directory. The read answered anyway, because both commands bind
the same viper keys and upload's flag was left to supply the default — a value the
caller of generate could neither see nor change.
Also stop describing --include-sources as Apple-only, since an Android mapping
carries sources now too.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(symbols): key an Android upload by the id R8 recorded for the mapping
An Android build only reached the Symbols Id Lane if something had stamped an
id into the app for it, which meant a Gradle task in every project that wanted
one. R8 has been recording an id for its own mapping all along — "# pg_map_id:"
in the header — and from AGP 8.12 it stamps that same id into each class, so
the shipped app already reports it on every frame of every crash.
Read it from the header when no id was given or found in the packaged app, and
key the upload by it. A project that adds nothing to its build now uploads to
the lane its crashes arrive on; an id the app stamped still wins, since that is
a build saying what it will report, and the only answer for one too old for R8
to stamp anything itself.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(symbols): upload the dSYMs an Xcode build just produced
A dSYM is best uploaded by the build that made it, which means a Run Script
phase — and every project writing one wrote the same two pieces of shell first:
a --path pointing at DWARF_DSYM_FOLDER_PATH, and a guard skipping the
configurations that make no dSYM, without which a Debug build fails on "no
.dSYM bundles found". Both are debugged inside a build phase, where the way you
find out is a failed build.
Read the folder from the build environment when no --path was given, and treat
finding no dSYM there as nothing to do rather than as an error. What a phase has
to say is now the project it uploads to. An explicit --path still wins, so
uploading a dSYM from an archive or from CI is unchanged.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(symbols): one index per mapping when a class name repeats
Mirrors the observability copy of r8index, which has to stay byte-identical apart
from the srcbundle import path.
srcbundle.Builder.Add ignores a key it already holds, so streaming keeps the first
block for a repeated obfuscated name, while Parse's map assignment kept the last.
One mapping then encoded to two different indexes depending on which encoder read
it, which is not something an artifact addressed by its mapping's id can afford.
Parse keeps the first too: streaming has no other option, since a class is gzipped
and handed off before a duplicate arrives.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>1 parent 404c4cc commit e361fa6
10 files changed
Lines changed: 338 additions & 13 deletions
File tree
- cmd/symbols
- internal/symbols/r8index
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
126 | 126 | | |
127 | 127 | | |
128 | 128 | | |
129 | | - | |
130 | | - | |
131 | | - | |
132 | | - | |
133 | | - | |
134 | 129 | | |
135 | 130 | | |
136 | 131 | | |
137 | 132 | | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
138 | 148 | | |
139 | 149 | | |
140 | 150 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
54 | 54 | | |
55 | 55 | | |
56 | 56 | | |
57 | | - | |
58 | | - | |
| 57 | + | |
| 58 | + | |
59 | 59 | | |
60 | 60 | | |
61 | 61 | | |
62 | 62 | | |
63 | | - | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
64 | 72 | | |
65 | 73 | | |
66 | 74 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
224 | 224 | | |
225 | 225 | | |
226 | 226 | | |
227 | | - | |
228 | | - | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
229 | 232 | | |
230 | 233 | | |
231 | 234 | | |
| |||
700 | 703 | | |
701 | 704 | | |
702 | 705 | | |
703 | | - | |
| 706 | + | |
704 | 707 | | |
705 | 708 | | |
706 | | - | |
| 709 | + | |
707 | 710 | | |
708 | 711 | | |
709 | 712 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
55 | 55 | | |
56 | 56 | | |
57 | 57 | | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
58 | 61 | | |
59 | 62 | | |
60 | 63 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
72 | 72 | | |
73 | 73 | | |
74 | 74 | | |
75 | | - | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
76 | 89 | | |
77 | 90 | | |
78 | 91 | | |
| |||
0 commit comments