Repository navigation
Conversation
JSON Schema defines `pattern` and `patternProperties` as ECMA-262 regular expressions. Julia compiles PCRE with UCP and without DOLLAR_ENDONLY, so `$` matched before a trailing newline and `\d`, `\w` matched Unicode digits and letters. - Compile with DOLLAR_ENDONLY and without UCP: `$` matches only at the end, and `\d`, `\w`, `\b` are ASCII. - Translate `\s` and `\S` to ECMA-262 white space: tab, vertical tab, form feed, every Zs, U+FEFF, and the line terminators. - Map Unicode's extra General_Category aliases (digit, cntrl, punct, Combining_Mark). - Run optional/ecmascript-regex.json from the official suite for each draft, and add unit tests that run by default.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
JSON Schema defines
patternandpatternPropertiesas ECMA-262 regular expressions._ecma_regexcompiles them with Julia's default PCRE options. Those includeUCPandlack
DOLLAR_ENDONLY, so several ECMA-262 rules did not hold:^abc$"abc\n"$before a trailing newline)^\d$"߀"(NKO digit zero)UCP)^\w$"é"UCP)^\s$^\p{digit}+$"߀"The official suite's
optional/ecmascript-regex.jsoncovers these cases, but it did notrun.
Changes (
src/schema_engine/dialects.jl)DOLLAR_ENDONLYand withoutUCP.$matches only at the end, and\d,\wand\bare ASCII, as in ECMA-262.\p{…}still matches Unicode properties,because it needs only UTF mode.
\sand\Sto the ECMA-262 white-space set: tab, vertical tab, form feed,every
Zs, U+FEFF, and the line terminators LF, CR, U+2028 and U+2029. Julia compileswith
ALT_BSUX, so the code points use the\uhhhhform.ECMA_GENERAL_CATEGORIES:digit,cntrl,punctandCombining_Mark.Limit: inside a character class,
[\S]keeps PCRE's ASCII meaning, because PCREcannot remove a set from inside a class.
\sinside a class is translated.Tests
test/schema_engine/official.jlalso runsoptional/ecmascript-regex.jsonfor eachdraft (2020-12, 2019-09, 7, 6, 4) when
OPENAPI_SCHEMA_SUITE=all. Before this change,the 2020-12 file stops with a compile error at
\p{digit}. The other cases also fail:\d,\D,\w,\W,\s,\S. After it, all 74 tests pass for each draft.test/schema_engine/compiled.jlhas unit tests that run by default: the trailingnewline, ASCII
\dand\w, Unicode\s(also inside a class),\p{digit}and\p{Letter}.OPENAPI_SCHEMA_SUITE=all julia --project=. -e 'using Pkg; Pkg.test()': "OpenAPI testspassed". 12061 passes, 0 failures, 0 errors.
Official JSON Schema suite: 10624 of 10624.Context
This PR is independent of #135 (
date-time-local). Both come from the Sienna SDK(Sienna-Platform/SiennaSchemas#109), which relies on
patternfor RFC 3339 timestamps.Without
DOLLAR_ENDONLY,"2030-01-01T00:00:00\n"passes a^…$timestamp pattern.