feat(render-pdf): keep an inline icon's text in the PDF text layer - #698
Merged
Merged
Conversation
An inline SVG icon is drawn from paths, so the page has no character where it sits: a line with 🚀 in it, copied out of a PDF into a messenger, loses the rocket, and search and extraction miss it too. Since the previous change every emoji icon knows its text (SvgIcon.text()). ActualText cannot carry it: it replaces the text of the glyphs it covers, and a drawing covers none; PDFBox, poppler, pdf.js and MuPDF extract nothing from a Span with ActualText around paths. PdfTextLayer writes the text as one invisible glyph over the icon, on the line's baseline, font size the icon's height and Tz scaled so its advance is the icon's width. The glyph comes from a Type 3 font of empty glyph procedures whose ToUnicode maps each one-byte code to its text, a whole ZWJ sequence included, shown in rendering mode 3 so nothing is painted. q/Q isolates it, since rendering mode, Tz and the font outlive ET, and Tc, Tw and Ts are zeroed inside because a letter-spaced word before the icon leaves its tracking in Tc. One font serves a document, one code per distinct text, a new font after 255; the dictionaries grow as codes are handed out, and /ToUnicode is attached after construction because PDFont's constructor reads it. Text over 256 UTF-16 units is not written. Handlers reach it through PdfRenderEnvironment.writeTextLayer (@beta); the multi-section path shares one PdfTextLayer across sections. InlineSvgToken and ParagraphSvgSpan carry the text through layout, and PdfParagraphFragmentRenderHandler.renderSvg writes it after painting. The capability matrix and the rich-text recipe say what copies and where; the committed emoji-shortcodes.pdf preview is re-rendered, the only committed preview whose code path writes icon text. PdfTextLayerTest (8): the emoji line extracts codepoint for codepoint; pages with and without the text are identical pixel for pixel; an icon given withText copies its text and one without adds no font; 300 texts fill one font and start a second, ToUnicode in blocks of 100/100/55; one font across sections; the 256-unit cap; a 2:1 icon after Tc-tracked text gets a 20 pt glyph at zero rise; deterministic output stays byte-identical. Each was run against a broken implementation (no q/Q, no Tc reset, no Tz, one CMap block, no code cache, a font per section, no cap, no handler call) and went red. poppler, pdf.js and MuPDF extract the example line identically. Verified: full CI reactor gate (clean verify, 13 modules) BUILD SUCCESS, 3254 tests, 0 failures; examples module 87 tests green, including CommittedAssetDriftTest.
check-stability-doc.mjs requires every @beta member to be named in docs/api-stability.md; writeTextLayer was not, which failed the JDK 17 CI leg. It joins letterSpacedFont in the list of @beta members of the PDF backend, now seven. Verified: all four knowledge-pack checks (extract-api --check, check-stability-doc, check-claims --check, check-routes) pass locally; javadoc:javadoc -pl :graph-compose-core BUILD SUCCESS, 0 warnings.
…o-left lines PdfTextLayer.write wrote q, BT, the glyph, ET and Q in a straight line, so a failure after the q (setFont, showText, the stream itself) left the rest of the page inside the glyph's text object and saved state. ET and Q now sit in finally blocks. A call inside a caller's open text object already failed clean: PDFBox refuses the q before writing it, so nothing reaches the stream; that is now pinned, and the writeTextLayer Javadoc says both. Right-to-left lines, measured with Hebrew and Arabic: the emoji's glyph lies between the words it was written between and states its whole text. Reading such a line back, PDFBox's handleDirection reverses the run one UTF-16 unit at a time (surrogates swapped) and poppler reverses the code points of a ZWJ sequence or a U+FE0F pair; pdf.js and MuPDF keep the emoji whole. The capability matrix records it. PdfTextLayerTest +5 (13): a call inside an open text object throws IllegalStateException and the stream reads BT ET only; a Mockito spy failing showText still leaves the stream ending ET Q with balanced q/Q and BT/ET; Hebrew and Arabic RTL lines place the glyph between their words and state the rocket whole; a ZWJ sequence and a heart in an RTL line stay on one glyph each; a left-to-right sentence with a Hebrew word reads back in written order. Without the finally blocks, and with the glyph shifted off the icon, the matching tests fail. Verified: full CI reactor gate BUILD SUCCESS, 3259 tests, 0 failures; the four knowledge-pack checks and core javadoc pass; examples 87 tests green with the committed preview unchanged.
DemchaAV
force-pushed
the
feat/emoji-copy-pdf
branch
from
September 18, 2026 11:35
6ca96e5 to
8b64fb9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
An inline SVG icon is drawn from paths, so the page has no character where it sits: select a line with
:rocket:in a PDF, copy it into a messenger, and the rocket is gone; search and text extraction miss it the same way. Since #697 every emoji icon knows its text (SvgIcon.text()); the PDF still never wrote it.ActualTextis the obvious tool and does not work here: it replaces the text of the glyphs it covers, and a drawing covers none. ASpanwithActualTextaround the paths extracts nothing in PDFBox, poppler, pdf.js or MuPDF.What changed
PdfTextLayer(render-pdf, package-private) writes text into the page's text layer without painting: one glyph over the icon from a Type 3 font whose glyph procedures are empty (1000 0 d0) and whoseToUnicodemaps each one-byte code to its text — a whole ZWJ sequence when that is what the icon depicts. Shown in rendering mode 3, so even a viewer that cannot read the font paints nothing — the invisible-text approach of OCR'd scans.Tzscaled so its advance equals the icon's width: the text lands inside its line and a selection runs across the icon.q/Qaround it, because rendering mode,Tzand the font outliveET— without it every word after the icon would be invisible. Inside,Tc,TwandTsare set to zero: a letter-spaced word before the icon leaves its tracking inTc, which would otherwise stretch the glyph.qbefore writing it, so the call fails withIllegalStateExceptionand nothing written. Past theq,ETandQsit infinallyblocks, so a failure half-way cannot leave the rest of the page inside the glyph's text object or saved state.CharProcs,Differences,Widths,LastCharandToUnicodegrow as codes are handed out, so the file is complete whenever it is saved./ToUnicodeis attached after construction becausePDFont's constructor reads it, and it is empty until the first code. A text over 256 UTF-16 units (beyond a CMap destination's 512 bytes) is not written.PdfRenderEnvironment.writeTextLayer(...)(@Beta,@since 2.5.0) is how a handler reaches it — handlers live in a sub-package. The multi-section path shares onePdfTextLayeracross sections, like the letter-spaced font registry.InlineSvgToken→ParagraphSvgSpancarrytext(layout.payloadsis@Internal);PdfParagraphFragmentRenderHandler.renderSvgwrites it after painting.withText, emoji copy). The committedemoji-shortcodes.pdfpreview is re-rendered — the only committed preview whose code path writes icon text; its emoji now copy out.Verification
./mvnw -f examples/pom.xml test→ 87 tests green,CommittedAssetDriftTestincluded. Knowledge pack regenerated (backends: +writeTextLayer) and--checkcurrent.PdfTextLayerTest(13):Launch 🚀 by 👩💻 with ❤️ done.codepoint for codepoint;withText("✓")copies as✓; an icon without text adds no font;ToUnicodein blocks of 100 / 100 / 55;Tcgets a glyph exactly 20 pt wide at zero rise;deterministic(true)output stays byte-identical;IllegalStateException, content stream readsBT ETonly;showTextfailing half-way (a Mockito spy): the stream still endsET Q,q/QandBT/ETbalanced;🚀whole;❤️in a right-to-left line stay on one glyph each;Deploy שלום 🚀 done(left-to-right with a Hebrew word) reads back in written order.q/Q, without theTcreset, withoutTz, with one 255-entry CMap block, without the code cache, with a font per section, without the length cap, with no handler call, without thefinallyblocks, with the glyph shifted off the icon — and each turns the matching tests red.pdftotext), pdf.js 4.10 and MuPDF.Notes
PDFTextStripper.handleDirectionreverses a right-to-left run one UTF-16 unit at a time (word.charAt(end)), so an emoji inside one comes back with its surrogates swapped; poppler reverses the code points of a multi-code-point glyph, so🚀survives but👩💻becomes💻👩and❤️loses its order. pdf.js and MuPDF keep every emoji whole. That is a reversal inside the reader, of a glyph the file states correctly; left-to-right lines, Hebrew word or not, read back exactly. Bidi marks (LRM) around the glyph's text might steer those two readers, but they would add invisible characters to every copy; not tried here.emoji-shortcodes.pdfpastes into a messenger with its emoji.Lane: shared-engine (render-pdf handler +
PdfRenderEnvironment; a pass-through field inlayout).Merge order: after #697, which is merged into
2.5-dev; this PR now sits directly on2.5-dev.