diff --git a/apps/guide/README.md b/apps/guide/README.md index f536d4e86..3cd4cd240 100644 --- a/apps/guide/README.md +++ b/apps/guide/README.md @@ -9,32 +9,34 @@ contract only. Lesson copy belongs to the guide author. ## Layout -| Path | Purpose | -| --------------------------------- | ------------------------------------------------ | -| `app/lessons/` | Lesson MDX files and metadata | -| `app/GuideLessonContent.res` | Build-time lesson discovery and validation | -| `app/GuideHome.res` | Guide workspace UI | -| `app/GuideCompilerBridgeHook.res` | Browser compiler and execution integration | -| `styles/main.css` | Guide-only visual system and responsive behavior | -| `public/` | Guide static assets, including `robots.txt` | -| `__tests__/` | Browser and unit coverage for the guide | +| Path | Purpose | +| --------------------------------- | -------------------------------------------------- | +| `app/lessons/` | Lesson MDX files and metadata | +| `app/GuideLessonContent.res` | Build-time lesson discovery and validation | +| `app/GuideHome.res` | Guide workspace UI | +| `app/GuideCompilerBridgeHook.res` | Browser compiler and execution integration | +| `app/GuideLessonValidation.res` | Build-time compilation and reference output checks | +| `styles/main.css` | Guide-only visual system and responsive behavior | +| `public/` | Guide static assets, including `robots.txt` | +| `__tests__/` | Browser and unit coverage for the guide | ## Lesson Contract Every `.mdx` file under `app/lessons/` is discovered recursively and ordered by its integer `position`. The build requires these frontmatter fields: -| Field | Required | Meaning | -| ------------------------- | -------- | ------------------------------------------------------ | -| `position` | Yes | Integer lesson order | -| `id` | Yes | Stable URL-safe hash and lesson identifier | -| `missionLabel` | Yes | Short lesson label | -| `title` | Yes | Lesson heading | -| `description` | Yes | Lesson summary used by the lesson model | -| `exercise.id` | Yes | Stable identifier for saved code and completion | -| `exercise.title` | Yes | Exercise label | -| `exercise.initialCode` | Yes | Initial editor contents | -| `exercise.expectedOutput` | No | Matching runtime log line that unlocks the next lesson | +| Field | Required | Meaning | +| ------------------------- | -------- | -------------------------------------------------------- | +| `position` | Yes | Integer lesson order | +| `id` | Yes | Stable URL-safe hash and lesson identifier | +| `missionLabel` | Yes | Short lesson label | +| `title` | Yes | Lesson heading | +| `description` | Yes | Lesson summary used by the lesson model | +| `exercise.id` | Yes | Stable identifier for saved code and completion | +| `exercise.title` | Yes | Exercise label | +| `exercise.initialCode` | Yes | Initial editor contents | +| `exercise.expectedOutput` | Yes | Matching runtime log line that unlocks the next lesson | +| `exercise.solutionCode` | No | Build-only reference solution; defaults to `initialCode` | Lesson IDs determine durable hash-based deep links. Renaming a published lesson ID requires a redirect or an explicit decision to let existing deep links fall @@ -47,9 +49,17 @@ decision to discard existing learner progress. Lesson IDs must be URL-safe slugs made from lowercase ASCII letters, digits, and hyphens. Exercise IDs may use the established slash-delimited form. -An exercise without `expectedOutput` is parsed as a manual check, but manual -checks are not currently completable. Published lessons therefore need a -deterministic `expectedOutput` until another completion mechanism exists. +Every published exercise needs a deterministic `expectedOutput` string. The +build compiles `initialCode` and `solutionCode`, runs the reference solution, +and requires a matching runtime log line. Matching preserves whitespace and +uses the same argument serialization as the browser checkpoint. + +When the learner must edit the starter to reach the checkpoint, supply +`exercise.solutionCode`. Otherwise it defaults to `initialCode`. Reference +solutions are only read during validation and are excluded from published +lesson data. Solutions must produce synchronous output and finish within one +second; browser-only APIs and asynchronous exercises need a separate validation +strategy before they can be authored. ## Local Development @@ -72,13 +82,24 @@ yarn build:guide yarn workspace @rescript-lang/guide ci:test ``` -`yarn build:guide` compiles ReScript and pre-renders `/` into -`apps/guide/build/client/`, the assets directory configured in `wrangler.toml`. -Vite serves and copies the guide's `public/` directory, including `robots.txt`. -The root route imports the shared docs favicons so Vite emits them with the -guide's other assets. The guide fetches available compiler versions from -`https://cdn.rescript-lang.org` during pre-rendering; the -interactive compiler and runtime then run in the learner's browser. +`yarn build:guide` compiles ReScript, validates the lessons during pre-rendering, +and pre-renders `/` into `apps/guide/build/client/`, the assets directory configured +in `wrangler.toml`. Vite serves and copies the guide's `public/` directory, +including `robots.txt`. The root route imports the shared docs favicons so Vite +emits them with the guide's other assets. + +Validation uses the actual browser compiler and its libraries from `https://cdn.rescript-lang.org`. +A failed download, compilation, execution, or output check stops the build. +`PLAYGROUND_BUNDLE_ENDPOINT` can override the CDN base URL. + +The guide pins ReScript **v12.3.1** in `app/GuideCompilerSettings.res`. Both +validation and browser execution use that version and its runtime files; new +CDN releases cannot silently change lessons. To upgrade, change the pin and run +`yarn workspace @rescript-lang/guide test:compiler` and `yarn build:guide`. +The guide CI command runs the compiler checks before the browser tests. + +In the browser, the output panel reports a failed compiler download or a +15-second loading timeout with instructions to reload. ## Current Capability @@ -99,6 +120,7 @@ show a stop screen rather than a usable guide. Track outstanding guide launch work in this checklist: - [ ] Author, review, and run every lesson using its real expected output. +- [x] Compile starters and check reference solution output against a pinned compiler. - [ ] Add a content-validation test for the complete lesson set, including unique `position`, lesson IDs, and exercise IDs. - [ ] Decide and implement the completion model for exercises that cannot use diff --git a/apps/guide/__tests__/GuideCompilerLoading_.test.res b/apps/guide/__tests__/GuideCompilerLoading_.test.res new file mode 100644 index 000000000..ff56f9fd6 --- /dev/null +++ b/apps/guide/__tests__/GuideCompilerLoading_.test.res @@ -0,0 +1,46 @@ +open Vitest + +@module("./compilerLoadingFixture.js") +external interceptCompilerScripts: bool => unit = "interceptCompilerScripts" + +@module("./compilerLoadingFixture.js") +external expireCompilerLoading: unit => unit = "expireCompilerLoading" + +let renderWithCompiler = () => + render( + + + , + ) + +test("shows a compiler loading failure in the guide output", async () => { + await viewport(1440, 900) + interceptCompilerScripts(true) + let screen = await renderWithCompiler() + await (await screen->getByText("Compiler setup failed"))->element->toBeVisible + await ( + await screen->getByText( + "Could not load compiler from url https://guide-compiler.test/v12.3.1/compiler.js", + ) + ) + ->element + ->toBeVisible +}) + +test("shows a useful message when the compiler CDN never responds", async () => { + await viewport(1440, 900) + interceptCompilerScripts(false) + expireCompilerLoading() + let screen = await renderWithCompiler() + await (await screen->getByText("Compiler setup failed"))->element->toBeVisible + await ( + await screen->getByText( + "ReScript v12.3.1 did not load within 15 seconds. Reload the page to try again.", + ) + ) + ->element + ->toBeVisible +}) diff --git a/apps/guide/__tests__/GuideCompilerSettings_.test.res b/apps/guide/__tests__/GuideCompilerSettings_.test.res index 853643400..a902d271d 100644 --- a/apps/guide/__tests__/GuideCompilerSettings_.test.res +++ b/apps/guide/__tests__/GuideCompilerSettings_.test.res @@ -1,18 +1,8 @@ open Vitest -test("selects the latest stable ReScript compiler with ESM output", async () => { - let version = - GuideCompilerSettings.latestStableVersion([ - "v12.1.0-alpha.1", - "v11.1.4", - "v12.0.0", - "v12.2.0-beta.1", - "v12.1.3", - ]) - ->Option.getOrThrow - ->Semver.toString - - expect(version)->toBe("v12.1.3") +test("pins the guide to its validated v12 compiler with ESM output", async () => { + expect(GuideCompilerSettings.version)->toBe("v12.3.1") + expect(GuideCompilerSettings.parsedVersion->Semver.toString)->toBe(GuideCompilerSettings.version) expect(GuideCompilerSettings.moduleSystem)->toBe("esmodule") expect(GuideCompilerSettings.warnFlags->String.includes("-109"))->toBe(true) }) diff --git a/apps/guide/__tests__/compiler/GuideLessons_.test.res b/apps/guide/__tests__/compiler/GuideLessons_.test.res new file mode 100644 index 000000000..7fef0b29f --- /dev/null +++ b/apps/guide/__tests__/compiler/GuideLessons_.test.res @@ -0,0 +1,188 @@ +type assertion<'a> +@module("vitest") external test: (string, unit => promise) => unit = "test" +@module("vitest") @scope("test") +external for_: array<'a> => (string, 'a => promise) => unit = "for" +@module("vitest") external beforeAll: (unit => promise, int) => unit = "beforeAll" +@module("vitest") external afterEach: (unit => unit) => unit = "afterEach" +@module("vitest") external expect: 'a => assertion<'a> = "expect" +@send external toBe: (assertion<'a>, 'a) => unit = "toBe" +@send @scope("not") external notToContain: (assertion, string) => unit = "toContain" +@send external toThrow: (assertion 'a>, string) => unit = "toThrow" +@send @scope("rejects") +external rejectsWith: (assertion>, string) => promise = "toThrow" +@send @scope("resolves") +external resolvesToUnit: assertion> => promise = "toBeUndefined" +@module("vitest") @scope("vi") +external stubFetch: (@as("fetch") _, string => promise) => unit = + "stubGlobal" +@module("vitest") @scope("vi") external unstubGlobals: unit => unit = "unstubAllGlobals" + +let bundleBaseUrl = "https://cdn.rescript-lang.org" +let compiler = ref(None) + +beforeAll(async () => { + compiler.contents = Some( + await GuideLessonValidation.loadCompiler( + ~bundleBaseUrl, + ~version=GuideCompilerSettings.version, + ~moduleSystem=GuideCompilerSettings.moduleSystem, + ~warnFlags=GuideCompilerSettings.warnFlags, + ), + ) +}, 60000) +afterEach(unstubGlobals) + +let exercise = ( + ~initialCode=`let greeting = "hello"`, + ~solutionCode=`let greeting = "hello"`, + ~expectedOutput="hello", + (), +): GuideLessonValidation.exercise => { + sourcePath: "test-lesson.mdx", + initialCode, + solutionCode, + expectedOutput, +} + +let validateExercise = exercise => + GuideLessonValidation.validateExercise( + ~compiler=compiler.contents->Option.getOrThrow, + ~bundleBaseUrl, + exercise, + ) + +let loadCompiler = bundleBaseUrl => + GuideLessonValidation.loadCompiler( + ~bundleBaseUrl, + ~version=GuideCompilerSettings.version, + ~moduleSystem=GuideCompilerSettings.moduleSystem, + ~warnFlags=GuideCompilerSettings.warnFlags, + ) + +test("the pinned browser compiler validates every real lesson", async () => { + expect(compiler.contents->Option.getOrThrow->RescriptCompilerApi.Compiler.version)->toBe( + GuideCompilerSettings.version->String.slice(~start=1), + ) + await expect(GuideLessonValidation.validate(~bundleBaseUrl))->resolvesToUnit + let published = GuideLessonContent.load()->JSON.stringifyAny->Option.getOrThrow + expect(published)->notToContain("solutionCode") + expect(published)->notToContain(`greet(\\"Spock\\")`) +}) + +test("invalid starter code fails even when its reference solution is valid", async () => { + await expect(validateExercise(exercise(~initialCode="let x =", ())))->rejectsWith( + "test-lesson.mdx exercise.initialCode does not compile", + ) +}) + +test("invalid reference solutions identify the file and field", async () => { + await expect(validateExercise(exercise(~solutionCode="let x: int = false", ())))->rejectsWith( + "test-lesson.mdx exercise.solutionCode does not compile", + ) +}) + +test("output mismatches fail with the expected and actual strings", async () => { + await expect( + validateExercise(exercise(~expectedOutput="hello ", ())), + )->rejectsWith(`exercise.expectedOutput "hello " did not match solution output "hello"`) +}) + +test("runtime failures identify the reference solution", async () => { + await expect( + validateExercise(exercise(~solutionCode=`JsError.throwWithMessage("boom")`, ())), + )->rejectsWith("test-lesson.mdx exercise.solutionCode failed at runtime: boom") +}) + +test("solutions execute against the pinned compiler's published runtime imports", async () => { + await expect( + validateExercise( + exercise( + ~solutionCode="let values = [1, 2]->Array.map(value => value + 1)", + ~expectedOutput="[2,3]", + (), + ), + ), + )->resolvesToUnit +}) + +test("final expressions use the browser's source instrumentation", async () => { + await expect( + validateExercise( + exercise( + ~solutionCode=`let greeting = "starter"\nInt.toString(42)`, + ~expectedOutput="42", + (), + ), + ), + )->resolvesToUnit +}) + +test("multiple console arguments match the browser checkpoint serialization", async () => { + await expect( + validateExercise( + exercise(~solutionCode=`Console.log2("answer", 42)`, ~expectedOutput="answer 42", ()), + ), + )->resolvesToUnit +}) + +test("runaway solutions stop with a timeout", async () => { + await expect( + GuideLessonValidation.runProgram({code: "while (true) {}", imports: Dict.make()}), + )->rejectsWith("timed out") +}) + +test("missing compiler assets fail with the URL and HTTP status", async () => { + stubFetch(async _url => WebAPI.Response.fromString("missing", ~init={status: 404})) + await expect(loadCompiler("https://missing-guide.test"))->rejectsWith( + `https://missing-guide.test/${GuideCompilerSettings.version}/compiler.js: HTTP 404`, + ) +}) + +test("a bundle claiming a different compiler version is rejected", async () => { + stubFetch(async url => { + let code = + url->String.endsWith("/compiler.js") + ? `globalThis.rescript_compiler = {make: () => ({version: "13.0.0"})};` + : "" + WebAPI.Response.fromString(code) + }) + await expect(loadCompiler("https://wrong-guide.test"))->rejectsWith( + `Expected guide compiler ${GuideCompilerSettings.version}, received 13.0.0`, + ) +}) + +for_([ + None, + Some(JSON.Number(42.0)), + Some(JSON.Boolean(false)), +])( + "reference validation rejects missing or non-string expected output: %s", + async expectedOutput => { + let field = switch expectedOutput { + | None => "" + | Some(value) => ` expectedOutput: ${value->JSON.stringify}\n` + } + let raw = `---\nexercise:\n initialCode: let x = 1\n${field}---\n` + expect(() => GuideLessonValidation.fromRaw(~raw, ~sourcePath="broken.mdx"))->toThrow( + "broken.mdx exercise.expectedOutput must be a string", + ) + }, +) + +test("reference code defaults to the starter and preserves an empty expected output", async () => { + let parsed = GuideLessonValidation.fromRaw( + ~raw=`---\nexercise:\n initialCode: let x = ""\n expectedOutput: ""\n---\n`, + ~sourcePath="empty.mdx", + ) + expect(parsed.solutionCode)->toBe(parsed.initialCode) + expect(parsed.expectedOutput)->toBe("") +}) + +test("empty reference code is an authoring error", async () => { + expect(() => + GuideLessonValidation.fromRaw( + ~raw=`---\nexercise:\n initialCode: let x = 1\n expectedOutput: "1"\n solutionCode: ""\n---\n`, + ~sourcePath="empty.mdx", + ) + )->toThrow("empty.mdx exercise.solutionCode must be a non-empty string") +}) diff --git a/apps/guide/__tests__/compilerLoadingFixture.js b/apps/guide/__tests__/compilerLoadingFixture.js new file mode 100644 index 000000000..bbab7ce3c --- /dev/null +++ b/apps/guide/__tests__/compilerLoadingFixture.js @@ -0,0 +1,29 @@ +import { afterEach, vi } from "vitest"; + +afterEach(() => { + vi.restoreAllMocks(); +}); + +// Intercept script insertion rather than loading a remote compiler in UI failure tests. +export function interceptCompilerScripts(fail) { + const appendChild = document.body.appendChild.bind(document.body); + vi.spyOn(document.body, "appendChild").mockImplementation((node) => { + if ( + node instanceof HTMLScriptElement && + node.src.includes("/compiler.js") + ) { + if (fail) queueMicrotask(() => node.dispatchEvent(new Event("error"))); + return node; + } + return appendChild(node); + }); +} + +export function expireCompilerLoading() { + // Advance only the guide's loading deadline; leave editor and test-library timers real. + const setTimeout = window.setTimeout.bind(window); + vi.spyOn(window, "setTimeout").mockImplementation( + (callback, delay, ...args) => + setTimeout(callback, delay === 15000 ? 0 : delay, ...args), + ); +} diff --git a/apps/guide/app/GuideCompilerBridge.res b/apps/guide/app/GuideCompilerBridge.res index 18072bd76..fe0698018 100644 --- a/apps/guide/app/GuideCompilerBridge.res +++ b/apps/guide/app/GuideCompilerBridge.res @@ -1,18 +1,11 @@ @react.component let make = ( ~bundleBaseUrl, - ~versions: array, ~code, ~editorRef: React.ref>, ~setOutput, ) => { - GuideCompilerBridgeHook.useCompilerBridge( - ~bundleBaseUrl, - ~versions, - ~code, - ~editorRef, - ~setOutput, - ) + GuideCompilerBridgeHook.useCompilerBridge(~bundleBaseUrl, ~code, ~editorRef, ~setOutput)
diff --git a/apps/guide/app/GuideCompilerBridgeHook.res b/apps/guide/app/GuideCompilerBridgeHook.res index 2b19d1a1b..6efa9c34e 100644 --- a/apps/guide/app/GuideCompilerBridgeHook.res +++ b/apps/guide/app/GuideCompilerBridgeHook.res @@ -1,22 +1,14 @@ let useCompilerBridge = ( ~bundleBaseUrl, - ~versions, ~code, ~editorRef: React.ref>, ~setOutput, ) => { - let compilerVersions = React.useMemo( - () => GuideCompilerSettings.supportedVersions(versions), - [versions], - ) - let initialVersion = React.useMemo( - () => compilerVersions->GuideCompilerSettings.latestStableParsedVersion, - [compilerVersions], - ) + let compilerVersions = React.useMemo(() => [GuideCompilerSettings.parsedVersion], []) let (compilerState, compilerDispatch) = CompilerManagerHook.useCompilerManager( ~bundleBaseUrl, - ~initialVersion?, + ~initialVersion=GuideCompilerSettings.parsedVersion, ~initialModuleSystem=GuideCompilerSettings.moduleSystem, ~initialWarnFlags=GuideCompilerSettings.warnFlags, ~syncUrl=false, @@ -27,6 +19,27 @@ let useCompilerBridge = ( let lastExecutedJsCode = React.useRef("") let isWaitingForRuntimeOutput = React.useRef(false) + React.useEffect(() => { + switch compilerState { + | Init => + let timer = setTimeout( + ~handler=() => + setOutput( + _ => + GuideCompilerFeedback.Output.make( + ~status="Compiler setup failed", + ~diagnostics=[ + `ReScript ${GuideCompilerSettings.version} did not load within 15 seconds. Reload the page to try again.`, + ], + ), + ), + ~timeout=GuideCompilerSettings.loadingTimeoutMs, + ) + Some(() => clearTimeout(timer)) + | _ => None + } + }, (compilerState, setOutput)) + React.useEffect(() => { switch compilerState { | Ready({targetLang}) if code !== lastCompiledCode.current => @@ -83,18 +96,11 @@ let useCompilerBridge = ( | Ready({selected, result: Comp(Success({jsCode, typeHints}))}) if jsCode !== lastExecutedJsCode.current => lastExecutedJsCode.current = jsCode - let runtimeJsCode = switch GuideRuntimeSource.instrument(~code, ~typeHints) { - | Some(runtimeCode) => - // The instrumented source may fail on compiler internals; fall back to user JS in that case. - switch selected.instance->RescriptCompilerApi.Compiler.resCompile(runtimeCode) { - | Success({jsCode}) => jsCode - | Fail(_) | UnexpectedError(_) | Unknown(_, _) => jsCode - } - | None => jsCode - } - - switch runtimeJsCode->GuideRuntimeTransform.transform( - ~resultBindingName=GuideRuntimeSource.resultBindingName, + switch GuideRuntimeProgram.fromCompilation( + ~compiler=selected.instance, + ~code, + ~jsCode, + ~typeHints, ) { | Some({code: runtimeCode, imports}) => isWaitingForRuntimeOutput.current = true diff --git a/apps/guide/app/GuideCompilerData.res b/apps/guide/app/GuideCompilerData.res index d4a32be91..0bfb0d297 100644 --- a/apps/guide/app/GuideCompilerData.res +++ b/apps/guide/app/GuideCompilerData.res @@ -1,43 +1,3 @@ -type t = { - bundleBaseUrl: string, - versions: array, -} +type t = {bundleBaseUrl: string} -module Env = { - @scope(("process", "env")) external nodeEnv: string = "NODE_ENV" - @scope(("process", "env")) - external playgroundBundleEndpoint: option = "PLAYGROUND_BUNDLE_ENDPOINT" -} - -let fetchVersions = async versionsBaseUrl => { - let response = await fetch(versionsBaseUrl ++ "/playground-bundles/versions.json") - let json = await WebAPI.Response.json(response) - json - ->JSON.Decode.array - ->Option.getOrThrow - ->Array.map(json => json->JSON.Decode.string->Option.getOrThrow) -} - -let load = async () => { - let (bundleBaseUrl, versionsBaseUrl) = switch (Env.playgroundBundleEndpoint, Env.nodeEnv) { - | (Some(baseUrl), _) => (baseUrl, baseUrl) - | (None, "development") => - let baseUrl = "https://cdn.rescript-lang.org" - (baseUrl, baseUrl) - | (None, _) => - let baseUrl = "https://cdn.rescript-lang.org" - (baseUrl, baseUrl) - } - - try { - let versions = await fetchVersions(versionsBaseUrl) - Some({ - bundleBaseUrl, - versions, - }) - } catch { - | JsExn(error) => - Console.error2("error while fetching guide compiler versions", error) - None - } -} +let load = () => {bundleBaseUrl: CompilerEnvironment.remoteBundleBaseUrl()} diff --git a/apps/guide/app/GuideCompilerSettings.res b/apps/guide/app/GuideCompilerSettings.res index 843d36835..52bdebc67 100644 --- a/apps/guide/app/GuideCompilerSettings.res +++ b/apps/guide/app/GuideCompilerSettings.res @@ -1,47 +1,6 @@ +// Upgrade deliberately, together with the lesson validation run. +let version = "v12.3.1" +let parsedVersion = version->Semver.parse->Option.getOrThrow let moduleSystem = "esmodule" let warnFlags = "+a-4-9-20-40-41-42-50-61-102-109" - -let isSupportedVersion = (version: Semver.t) => - switch version.major { - | 8 | 9 => false - | 10 => version.minor >= 1 - | 11 => version.minor >= 1 && version.preRelease->Option.isNone - | 12 => - switch version.preRelease { - | None => true - | Some(_) => version.minor > 1 - } - | _ => true - } - -let compareIntDescending = (a, b) => { - if a > b { - -1.0 - } else if a < b { - 1.0 - } else { - 0.0 - } -} - -let compareVersionDescending = (a: Semver.t, b: Semver.t) => { - switch compareIntDescending(a.major, b.major) { - | 0.0 => - switch compareIntDescending(a.minor, b.minor) { - | 0.0 => compareIntDescending(a.patch, b.patch) - | result => result - } - | result => result - } -} - -let supportedVersions = versions => - versions - ->Array.filterMap(Semver.parse) - ->Array.filter(isSupportedVersion) - ->Array.toSorted(compareVersionDescending) - -let latestStableParsedVersion = (versions: array) => - versions->Array.find(version => version.preRelease->Option.isNone) - -let latestStableVersion = versions => versions->supportedVersions->latestStableParsedVersion +let loadingTimeoutMs = 15000 diff --git a/apps/guide/app/GuideHome.res b/apps/guide/app/GuideHome.res index 1462ecde6..6aebc26dc 100644 --- a/apps/guide/app/GuideHome.res +++ b/apps/guide/app/GuideHome.res @@ -128,13 +128,9 @@ let make = (
{switch compilerData { - | Some({bundleBaseUrl, versions}) => + | Some({bundleBaseUrl}) => | None => React.null }} diff --git a/apps/guide/app/GuideHomeRoute.res b/apps/guide/app/GuideHomeRoute.res index 1f36b5179..10d29ed11 100644 --- a/apps/guide/app/GuideHomeRoute.res +++ b/apps/guide/app/GuideHomeRoute.res @@ -1,22 +1,24 @@ type loaderData = { - compilerData: option, + compilerData: GuideCompilerData.t, lessons: array, } exception LessonLoadError(string) let loader: ReactRouter.Loader.t = async _ => { - let compilerData = await GuideCompilerData.load() + let compilerData = GuideCompilerData.load() let lessons = switch GuideLessonContent.load() { | Ok(lessons) => lessons | Error(message) => throw(LessonLoadError(message)) } + await GuideLessonValidation.validate(~bundleBaseUrl=compilerData.bundleBaseUrl) + {compilerData, lessons} } @react.component let default = () => { let {compilerData, lessons}: loaderData = ReactRouter.useLoaderData() - + } diff --git a/apps/guide/app/GuideHomeRoute.resi b/apps/guide/app/GuideHomeRoute.resi index 380c80ddc..321b35251 100644 --- a/apps/guide/app/GuideHomeRoute.resi +++ b/apps/guide/app/GuideHomeRoute.resi @@ -1,5 +1,5 @@ type loaderData = { - compilerData: option, + compilerData: GuideCompilerData.t, lessons: array, } diff --git a/apps/guide/app/GuideLessonValidation.res b/apps/guide/app/GuideLessonValidation.res new file mode 100644 index 000000000..5860fc56a --- /dev/null +++ b/apps/guide/app/GuideLessonValidation.res @@ -0,0 +1,131 @@ +// Build-only authoring data: reference solutions never enter the route's loader data. +type exercise = { + sourcePath: string, + initialCode: string, + solutionCode: string, + expectedOutput: string, +} + +@module("../scripts/lesson-compiler.mjs") +external loadCompiler: ( + ~bundleBaseUrl: string, + ~version: string, + ~moduleSystem: string, + ~warnFlags: string, +) => promise = "loadCompiler" + +@module("../scripts/lesson-compiler.mjs") +external runProgram: GuideRuntimeTransform.t => promise< + array, +> = "runProgram" + +let fail = message => JsError.throwWithMessage(message) + +let readString = (~dict, ~sourcePath, ~key, ~allowEmpty=false) => + switch dict->Dict.get(key) { + | Some(JSON.String(value)) if allowEmpty || value->String.trim !== "" => value + | _ => + fail( + `Guide lesson ${sourcePath} exercise.${key} must be ${allowEmpty + ? "a string" + : "a non-empty string"}.`, + ) + } + +let fromRaw = (~raw, ~sourcePath) => { + let {frontmatter}: MarkdownParser.result = MarkdownParser.parseSync(raw) + let dict = + frontmatter + ->JSON.Decode.object + ->Option.flatMap(dict => dict->Dict.get("exercise")) + ->Option.flatMap(JSON.Decode.object) + switch dict { + | None => fail(`Guide lesson ${sourcePath} must have exercise frontmatter.`) + | Some(dict) => + let initialCode = readString(~dict, ~sourcePath, ~key="initialCode") + let solutionCode = switch dict->Dict.get("solutionCode") { + | None => initialCode + | Some(_) => readString(~dict, ~sourcePath, ~key="solutionCode") + } + { + sourcePath, + initialCode, + solutionCode, + expectedOutput: readString(~dict, ~sourcePath, ~key="expectedOutput", ~allowEmpty=true), + } + } +} + +let compile = (~compiler, ~sourcePath, ~field, code) => + switch compiler->RescriptCompilerApi.Compiler.resCompile(code) { + | Success(result) => result + | Fail(failure) => + let details = failure->GuideCompilerFeedback.compileFailToOutputLines->Array.join("\n") + fail( + `Guide lesson ${sourcePath} exercise.${field} does not compile with ${GuideCompilerSettings.version}:\n${details}`, + ) + | UnexpectedError(message) | Unknown(message, _) => + fail(`Guide lesson ${sourcePath} exercise.${field}: ${message}`) + } + +let validateExercise = async (~compiler, ~bundleBaseUrl, exercise) => { + let {sourcePath, initialCode, solutionCode, expectedOutput} = exercise + compile(~compiler, ~sourcePath, ~field="initialCode", initialCode)->ignore + let {jsCode, typeHints} = compile(~compiler, ~sourcePath, ~field="solutionCode", solutionCode) + let program = switch GuideRuntimeProgram.fromCompilation( + ~compiler, + ~code=solutionCode, + ~jsCode, + ~typeHints, + ) { + | Some(program) => { + ...program, + imports: program.imports->Dict.mapValues(path => + GuideRuntimeImport.url( + ~bundleBaseUrl, + ~compilerVersion=GuideCompilerSettings.parsedVersion, + path, + ) + ), + } + | None => fail(`Guide lesson ${sourcePath} exercise.solutionCode has no executable output.`) + } + + let logs = try { + await runProgram(program) + } catch { + | JsExn(error) => + fail( + `Guide lesson ${sourcePath} exercise.solutionCode failed at runtime: ${error + ->JsExn.message + ->Option.getOr("Unknown runtime error")}`, + ) + } + if !(logs->Array.some(log => GuideLesson.runtimeLogText(log) === expectedOutput)) { + let actual = logs->Array.map(GuideLesson.runtimeLogText)->Array.join("\n") + fail( + `Guide lesson ${sourcePath} exercise.expectedOutput ${expectedOutput + ->JSON.Encode.string + ->JSON.stringify} did not match solution output ${actual + ->JSON.Encode.string + ->JSON.stringify}.`, + ) + } +} + +let validate = async (~bundleBaseUrl, ~dir=GuideLessonContent.lessonsDir()) => { + let exercises = GuideLessonContent.scanDir(dir)->Array.map(sourcePath => { + let raw = Node.Fs.readFileSync2(sourcePath, "utf8") + fromRaw(~raw, ~sourcePath) + }) + let compiler = await loadCompiler( + ~bundleBaseUrl, + ~version=GuideCompilerSettings.version, + ~moduleSystem=GuideCompilerSettings.moduleSystem, + ~warnFlags=GuideCompilerSettings.warnFlags, + ) + // The browser compiler is stateful; compile lessons sequentially. + for index in 0 to exercises->Array.length - 1 { + await validateExercise(~compiler, ~bundleBaseUrl, exercises->Array.getUnsafe(index)) + } +} diff --git a/apps/guide/app/GuideRuntimeProgram.res b/apps/guide/app/GuideRuntimeProgram.res new file mode 100644 index 000000000..c6a6b4a6e --- /dev/null +++ b/apps/guide/app/GuideRuntimeProgram.res @@ -0,0 +1,15 @@ +// Keep lesson validation and browser execution on the same transformation path. +let fromCompilation = (~compiler, ~code, ~jsCode, ~typeHints) => { + let runtimeJsCode = switch GuideRuntimeSource.instrument(~code, ~typeHints) { + | Some(runtimeCode) => + switch compiler->RescriptCompilerApi.Compiler.resCompile(runtimeCode) { + | Success({jsCode}) => jsCode + | Fail(_) | UnexpectedError(_) | Unknown(_, _) => jsCode + } + | None => jsCode + } + + runtimeJsCode->GuideRuntimeTransform.transform( + ~resultBindingName=GuideRuntimeSource.resultBindingName, + ) +} diff --git a/apps/guide/app/lessons/02-functions.mdx b/apps/guide/app/lessons/02-functions.mdx index 46504a93a..cedf3cd12 100644 --- a/apps/guide/app/lessons/02-functions.mdx +++ b/apps/guide/app/lessons/02-functions.mdx @@ -12,6 +12,10 @@ exercise: let greeting = greet("ReScript") expectedOutput: Hello, Spock! + solutionCode: | + let greet = name => "Hello, " ++ name ++ "!" + + let greeting = greet("Spock") --- Functions take values as input and return a new value. diff --git a/apps/guide/package.json b/apps/guide/package.json index 78163a861..5f09e1eda 100644 --- a/apps/guide/package.json +++ b/apps/guide/package.json @@ -8,9 +8,10 @@ "build:res": "node ../docs/generate-route-types.mjs && rescript build --warn-error +3+8+11+12+26+27+31+32+33+34+35+39+44+45+110", "build:vite": "react-router build", "build": "yarn build:res && yarn build:vite", - "ci:test": "yarn build:res && yarn vitest --run --browser.headless", + "ci:test": "yarn build:res && yarn test:compiler && yarn vitest --run --browser.headless", "dev:vite": "yarn build:res && vite --host", "vitest": "vitest", + "test:compiler": "vitest run --config vitest.compiler.config.mjs", "test:e2e": "yarn build && playwright test" }, "dependencies": { @@ -40,6 +41,7 @@ "@types/react": "^19.2.14", "@vitejs/plugin-react": "^6.0.1", "@vitest/browser-playwright": "^5.0.0", + "esbuild": "0.28.1", "lightningcss": "^1.32.0", "playwright": "^1.63.0", "vite": "^8.3.0", diff --git a/apps/guide/scripts/lesson-compiler.mjs b/apps/guide/scripts/lesson-compiler.mjs new file mode 100644 index 000000000..1727d0cfa --- /dev/null +++ b/apps/guide/scripts/lesson-compiler.mjs @@ -0,0 +1,113 @@ +import vm from "node:vm"; +import { build } from "esbuild"; +import { serializeArg } from "../../../packages/shared/src/runtimeConsole.js"; + +const sources = new Map(); + +async function fetchSource(url) { + if (!sources.has(url)) { + sources.set( + url, + (async () => { + const response = await fetch(url, { + signal: AbortSignal.timeout(15000), + }); + if (!response.ok) + throw new Error( + `Could not load guide compiler asset ${url}: HTTP ${response.status}`, + ); + return response.text(); + })(), + ); + } + try { + return await sources.get(url); + } catch (cause) { + sources.delete(url); + throw new Error( + `Could not load guide compiler asset ${url}: ${cause.message}`, + { cause }, + ); + } +} + +export async function loadCompiler( + bundleBaseUrl, + version, + moduleSystem, + warnFlags, +) { + const context = vm.createContext({ console }); + for (const path of [ + "compiler.js", + "@rescript/react/cmij.js", + "compiler-builtins/cmij.js", + ]) { + const url = `${bundleBaseUrl}/${version}/${path}`; + vm.runInContext(await fetchSource(url), context, { + filename: url, + timeout: 10000, + }); + } + const compiler = context.rescript_compiler.make(); + if (`v${compiler.version}` !== version) + throw new Error( + `Expected guide compiler ${version}, received ${compiler.version}`, + ); + compiler.setModuleSystem(moduleSystem === "esmodule" ? "es6" : "nodejs"); + compiler.setWarnFlags(warnFlags); + compiler.setExperimentalFeatures([]); + compiler.setJsxPreserveMode(false); + return compiler; +} + +export async function runProgram({ code, imports }) { + const importCode = Object.entries(imports) + .map(([name, url]) => `import * as ${name} from ${JSON.stringify(url)};`) + .join("\n"); + const result = await build({ + // Bundle the runtime dependencies only; preserve the guide's executable JS verbatim. + stdin: { + contents: `${importCode}\nglobalThis.__guideRuntimeImports = {${Object.keys(imports).join(",")}};`, + sourcefile: "guide-runtime.js", + }, + bundle: true, + format: "iife", + write: false, + logLevel: "silent", + plugins: [ + { + name: "guide-cdn-runtime", + setup(build) { + build.onResolve({ filter: /^https?:\/\// }, (args) => ({ + path: args.path, + namespace: "guide-cdn", + })); + build.onResolve({ filter: /.*/, namespace: "guide-cdn" }, (args) => ({ + path: new URL(args.path, args.importer).href, + namespace: "guide-cdn", + })); + build.onLoad( + { filter: /.*/, namespace: "guide-cdn" }, + async (args) => ({ + contents: await fetchSource(args.path), + loader: "js", + }), + ); + }, + }, + ], + }); + const logs = []; + const console = Object.fromEntries( + ["log", "warn", "error"].map((level) => [ + level, + (...args) => logs.push({ level, content: args.map(serializeArg) }), + ]), + ); + const context = vm.createContext({ console }); + vm.runInContext(result.outputFiles[0].text, context, { timeout: 1000 }); + Object.assign(context, context.__guideRuntimeImports); + vm.runInContext(code, context, { timeout: 1000 }); + return logs; +} diff --git a/apps/guide/vitest.compiler.config.mjs b/apps/guide/vitest.compiler.config.mjs new file mode 100644 index 000000000..7e0029d33 --- /dev/null +++ b/apps/guide/vitest.compiler.config.mjs @@ -0,0 +1,10 @@ +import { defineConfig } from "vitest/config"; + +export default defineConfig({ + test: { + environment: "node", + include: ["__tests__/compiler/*_.test.jsx"], + testTimeout: 60000, + expect: { requireAssertions: true }, + }, +}); diff --git a/packages/shared/src/CompilerEnvironment.res b/packages/shared/src/CompilerEnvironment.res new file mode 100644 index 000000000..a2e25a269 --- /dev/null +++ b/packages/shared/src/CompilerEnvironment.res @@ -0,0 +1,6 @@ +@scope(("process", "env")) external nodeEnv: string = "NODE_ENV" +@scope(("process", "env")) +external playgroundBundleEndpoint: option = "PLAYGROUND_BUNDLE_ENDPOINT" + +let cdnBaseUrl = "https://cdn.rescript-lang.org" +let remoteBundleBaseUrl = () => playgroundBundleEndpoint->Option.getOr(cdnBaseUrl) diff --git a/packages/shared/src/runtimeConsole.js b/packages/shared/src/runtimeConsole.js new file mode 100644 index 000000000..be097cb8b --- /dev/null +++ b/packages/shared/src/runtimeConsole.js @@ -0,0 +1,25 @@ +export function serializeArg(arg) { + if (arg === undefined) return "undefined"; + if (arg === null) return "null"; + if (typeof arg === "object") + return JSON.stringify(arg, Object.getOwnPropertyNames(arg)); + if (typeof arg === "function") return arg.toString(); + return arg; +} + +function installConsole(serializeArg) { + const sendLog = + (level) => + (...args) => { + parent.window.postMessage( + { type: level, args: args.map(serializeArg) }, + "*", + ); + }; + console.log = sendLog("log"); + console.warn = sendLog("warn"); + console.error = sendLog("error"); +} + +// Both functions are self-contained so the iframe can install them without module loading. +export const bridgeScript = `(${installConsole.toString()})(${serializeArg.toString()});`; diff --git a/yarn.lock b/yarn.lock index c3261161e..81437406a 100644 --- a/yarn.lock +++ b/yarn.lock @@ -2047,6 +2047,7 @@ __metadata: "@types/react": "npm:^19.2.14" "@vitejs/plugin-react": "npm:^6.0.1" "@vitest/browser-playwright": "npm:^5.0.0" + esbuild: "npm:0.28.1" isbot: "npm:^5" lightningcss: "npm:^1.32.0" playwright: "npm:^1.63.0"