Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/templates/tinytex-issue-body.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
## TinyTeX Pattern Update: {{DATE}}

The daily TinyTeX regex patterns have changed and need review.
The daily TinyTeX patterns (`regex.json`) or hyphenation map (`hyphen.json`) have changed and need review.

### Pattern Diff

Expand All @@ -22,6 +22,7 @@ See [dev-docs/tinytex-pattern-maintenance.md](./dev-docs/tinytex-pattern-mainten
- [ ] Review diff for significant changes
- [ ] Determine if patterns need adaptation
- [ ] Update `parse-error.ts` if needed
- [ ] If `hyphen.json` changed, update `kHyphenationPackageNames` in `parse-error.ts` and the copy in the unit test
- [ ] Add/update filter functions
- [ ] Run tests: `unit\latexmk\parse-error.test.ts`
- [ ] Add test cases for new patterns if needed
Expand Down
106 changes: 72 additions & 34 deletions .github/workflows/verify-tinytex-patterns.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v6

- name: Download and extract regex.json
- name: Download and extract regex.json and hyphen.json
env:
GH_TOKEN: ${{ github.token }}
run: |
Expand All @@ -28,33 +28,55 @@ jobs:
exit 0
fi
tar -xzf regex.tar.gz
echo "✓ Downloaded and extracted regex.json"
# Files compared by the later steps
PATTERN_FILES="regex.json hyphen.json"
if [ ! -f hyphen.json ]; then
echo "::warning::hyphen.json not found in regex.tar.gz"
PATTERN_FILES="regex.json"
fi
echo "PATTERN_FILES=$PATTERN_FILES" >> "$GITHUB_ENV"
echo "✓ Downloaded and extracted: $PATTERN_FILES"

- name: Restore cached regex.json
- name: Restore cached patterns
id: cache-restore
uses: actions/cache/restore@v5
with:
path: .cache/regex.json
key: tinytex-regex-latest
path: .cache
key: tinytex-patterns

- name: Compare versions
id: compare
run: |
if [ -f .cache/regex.json ]; then
if git diff --no-index --quiet .cache/regex.json regex.json; then
echo "changed=false" >> $GITHUB_OUTPUT
echo "first_run=false" >> $GITHUB_OUTPUT
echo "✓ No changes detected"
else
echo "changed=true" >> $GITHUB_OUTPUT
echo "first_run=false" >> $GITHUB_OUTPUT
echo "✗ Changes detected"
git diff --no-index .cache/regex.json regex.json > pattern-diff.txt || true
fi
else
# The download step sets PATTERN_FILES only when it extracted the files
if [ -z "${PATTERN_FILES:-}" ]; then
echo "changed=false" >> $GITHUB_OUTPUT
echo "first_run=false" >> $GITHUB_OUTPUT
echo "skipped=true" >> $GITHUB_OUTPUT
echo "::warning::No pattern files downloaded - comparison skipped"
exit 0
fi

if [ ! -f .cache/regex.json ]; then
echo "changed=false" >> $GITHUB_OUTPUT
echo "first_run=true" >> $GITHUB_OUTPUT
echo "⚠ No cached version (first run)"
exit 0
fi

changed=false
for f in $PATTERN_FILES; do
if [ ! -f ".cache/$f" ]; then
echo "✗ $f has no cached version"
changed=true
elif ! git diff --no-index --quiet ".cache/$f" "$f"; then
echo "✗ Changes detected in $f"
changed=true
fi
done
echo "changed=$changed" >> $GITHUB_OUTPUT
echo "first_run=false" >> $GITHUB_OUTPUT
if [ "$changed" = "false" ]; then
echo "✓ No changes detected"
fi

- name: Handle first run
Expand All @@ -69,6 +91,10 @@ jobs:
# Count patterns and categories
PATTERN_COUNT=$(jq '[.[] | length] | add' regex.json)
CATEGORY_COUNT=$(jq 'keys | length' regex.json)
# A checked assignment: invalid JSON must fail the step before it is cached
if [ -f hyphen.json ]; then
HYPHEN_COUNT=$(jq 'length' hyphen.json)
fi

# Write GitHub Actions summary
echo "## TinyTeX Pattern Baseline Established" >> "$GITHUB_STEP_SUMMARY"
Expand All @@ -77,34 +103,42 @@ jobs:
echo "- **TinyTeX commit:** [\`$TINYTEX_SHORT\`](https://github.com/rstudio/tinytex/commit/$TINYTEX_COMMIT)" >> "$GITHUB_STEP_SUMMARY"
echo "- **Pattern source:** [R/latex.R](https://github.com/rstudio/tinytex/blob/$TINYTEX_COMMIT/R/latex.R)" >> "$GITHUB_STEP_SUMMARY"
echo "- **Baseline:** $PATTERN_COUNT patterns across $CATEGORY_COUNT categories" >> "$GITHUB_STEP_SUMMARY"
echo "- **Cache key:** tinytex-regex-latest" >> "$GITHUB_STEP_SUMMARY"
if [ -f hyphen.json ]; then
echo "- **Hyphenation map:** $HYPHEN_COUNT names" >> "$GITHUB_STEP_SUMMARY"
fi
echo "- **Cache key:** tinytex-patterns" >> "$GITHUB_STEP_SUMMARY"
echo "" >> "$GITHUB_STEP_SUMMARY"
echo "No issue created (first run - baseline established)." >> "$GITHUB_STEP_SUMMARY"

# Prepare cache directory
mkdir -p .cache
cp regex.json .cache/regex.json
for f in $PATTERN_FILES; do
cp "$f" ".cache/$f"
done

echo "✓ Baseline established - cache will be saved"

- name: Exit if unchanged
if: steps.compare.outputs.changed == 'false' && steps.compare.outputs.first_run == 'false'
if: steps.compare.outputs.changed == 'false' && steps.compare.outputs.first_run == 'false' && steps.compare.outputs.skipped != 'true'
run: |
echo "No pattern changes detected. Cache hit - exiting."
exit 0

- name: Prepare readable diff
if: steps.compare.outputs.changed == 'true'
run: |
# Pretty-print both JSON files for readable diff
if [ -f .cache/regex.json ]; then
jq --sort-keys . .cache/regex.json > old-formatted.json
jq --sort-keys . regex.json > new-formatted.json
git diff --no-index old-formatted.json new-formatted.json > readable-diff.txt || true
else
jq --sort-keys . regex.json > new-formatted.json
echo "First run - no previous version to compare" > readable-diff.txt
fi
# Pretty-print both versions of each file for a readable diff
: > readable-diff.txt
for f in $PATTERN_FILES; do
if [ -f ".cache/$f" ]; then
jq --sort-keys . ".cache/$f" > "old-formatted-$f"
else
old="/dev/null"
fi
jq --sort-keys . "$f" > "new-formatted-$f"
git diff --no-index "${old:-old-formatted-$f}" "new-formatted-$f" >> readable-diff.txt || true
unset old
done

- name: Create or update issue
if: steps.compare.outputs.changed == 'true'
Expand Down Expand Up @@ -147,7 +181,9 @@ jobs:
if: steps.compare.outputs.changed == 'true'
run: |
mkdir -p .cache
cp regex.json .cache/regex.json
for f in $PATTERN_FILES; do
cp "$f" ".cache/$f"
done
echo "✓ Cache updated with new patterns"

- name: Delete old cache
Expand All @@ -156,19 +192,21 @@ jobs:
env:
GH_TOKEN: ${{ github.token }}
run: |
gh cache delete tinytex-regex-latest || echo "No existing cache to delete"
gh cache delete tinytex-patterns || echo "No existing cache to delete"

- name: Save new cache
if: steps.compare.outputs.changed == 'true' || steps.compare.outputs.first_run == 'true'
uses: actions/cache/save@v5
with:
path: .cache/regex.json
key: tinytex-regex-latest
path: .cache
key: tinytex-patterns

- name: Summary
if: always()
run: |
if [ "${{ steps.compare.outputs.first_run }}" == "true" ]; then
if [ "${{ steps.compare.outputs.skipped }}" == "true" ]; then
echo "⚠ Download failed - comparison skipped"
elif [ "${{ steps.compare.outputs.first_run }}" == "true" ]; then
echo "✓ Baseline established - cache created"
elif [ "${{ steps.compare.outputs.changed }}" == "true" ]; then
echo "✗ Pattern changes detected - issue created/updated"
Expand Down
15 changes: 14 additions & 1 deletion dev-docs/tinytex-pattern-maintenance.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ The R package **tinytex** maintains a comprehensive database of LaTeX error patt
- Every problem will be fixed in the R package first
- Low update frequency (~4 changes/year) makes manual adaptation practical

`regex.tar.gz` also contains `hyphen.json`, tinytex's map from language names to the package with their hyphenation patterns (only the exceptions to `hyphen-<name>`). It is generated from the `execute AddHyphen` lines of `texlive.tlpdb` (`R/hyphen.R`).

**Our process:**

- Daily automated check detects when TinyTeX patterns change
Expand Down Expand Up @@ -52,6 +54,16 @@ tinytex R package and Quarto LaTeX engine use patterns differently:
}
```

## Hyphenation packages

`kHyphenationPackageNames` in [parse-error.ts](../src/command/render/latexmk/parse-error.ts) is a copy of `hyphen.json`, and the unit tests keep a second copy. Detection of missing hyphenation patterns follows tinytex's `hyphen_pkgs()`, `check_babel()`, `detect_hyphen()` and `check_polyglossia()` in `R/latex.R`.

Quarto-only differences:

- the `chinese` and `chinese-hans` skip list ([#10291](https://github.com/quarto-dev/quarto-cli/issues/10291))
- the `Hyphen rules for '...' set to \l@nil` fallback
- unquoted language names in the babel "You haven't defined the language" error regex

## Manual Adaptation Process

When the automated workflow detects TinyTeX pattern changes, it creates/updates a GitHub issue with:
Expand Down Expand Up @@ -90,7 +102,7 @@ When the automated workflow detects TinyTeX pattern changes, it creates/updates
The automated workflow runs daily:

1. Downloads `regex.tar.gz` from [TinyTeX releases](https://github.com/rstudio/tinytex-releases)
2. Extracts and compares `regex.json` with cached version
2. Extracts and compares `regex.json` and `hyphen.json` with the cached versions
3. If changed: generates diff and creates/updates issue
4. If unchanged: exits early (no notification)

Expand All @@ -115,4 +127,5 @@ The automated workflow runs daily:
- [parse-error.ts](../src/command/render/latexmk/parse-error.ts) - Pattern implementation
- [parse-error.test.ts](../tests/unit/latexmk/parse-error.test.ts) - Unit tests
- [TinyTeX R source](https://github.com/rstudio/tinytex/blob/main/R/latex.R) - How patterns are used in R
- [TinyTeX hyphenation map](https://github.com/rstudio/tinytex/blob/main/R/hyphen.R)
- [TinyTeX releases](https://github.com/rstudio/tinytex-releases) - Source of regex.json
1 change: 1 addition & 0 deletions news/changelog-1.11.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ All changes included in 1.11:

- ([#14982](https://github.com/quarto-dev/quarto-cli/issues/14982)): Fix a footnote in document metadata, such as `title` or `author`, making PDF and Typst output shift heading levels, which turned the first level-1 heading into the document title.
- ([#15027](https://github.com/quarto-dev/quarto-cli/issues/15027)): Fix PDF rendering with regional German variants such as `lang: de-CH` failing when German hyphenation patterns are not installed, instead of installing them automatically.
- ([#15035](https://github.com/quarto-dev/quarto-cli/issues/15035)): Fix automatic installation of missing hyphenation patterns for the pinned German patterns (`ngerman-x-latest`), for several languages at once, and for languages loaded with `polyglossia`.

### `revealjs`

Expand Down
Loading
Loading