Skip to content

fix: keep crawlers out of event counts, keep preview cards, and file approved planner decisions - #83

Merged
jm-rivera merged 2 commits into
devfrom
fix/release-review-findings
Oct 29, 2026
Merged

jm-rivera merged 2 commits into
devfrom
fix/release-review-findings

Conversation

@jm-rivera

Copy link
Copy Markdown
Contributor

Fixes from a review of the website release (dev into main, #81). None of these breaks a page on its own, but three need to ship with that release: crawlers would be counted as share and embed views, a shared gallery search would lose its preview card, and once dfo-ai's planner decisions ship an approved plan would file them under "Counted as".

  • Crawlers no longer record events. track (src/lib/events/track.ts) returns before the log line, the database row and the PostHog forward when the request comes from a bot (isBotRequest in the new src/lib/bot.ts, on the isbot package the repo pins). The release opens /s/ and /embed/ to crawlers, and the event table keeps no user agent, so those rows could not be told apart later. requestLog.ts uses the same check.
  • Every page's openGraph comes from pageOpenGraph (src/lib/site.ts), which keeps the site name and the root card image unless a page has its own card, so /data?share=<gallery id> previews with the image again.
  • /data?spec= and a /data?share= that is not a gallery dataset now send X-Robots-Tag: noindex, nofollow (noindexHeaders in src/lib/crawl.ts), so their noindex no longer depends on when the streamed metadata reaches the head. A test matches each URL with Next's own getPathMatch and matchHas and checks the header agrees with the page metadata.
  • An approved plan files planner decisions under their own heading by choice_id (approvedView in src/lib/studio/plan.ts), which dfo-ai sends on plan.approved from ai#106. A plan stored in a chat before that has no ids and reads as before.
  • A chip the harness refuses (422 scope_choice_stale) no longer spends a guest prompt (withRefusedChip marks it refunded, Studio.tsx subtracts it).
  • The merged entity menu numbers groups that share a wording ("Congo (1)", "Congo (2)") in entity_choice order (entityChoiceLabels).
  • A stale switch refused without a current card ends read-only with no Approve bar (refusedSwitch), and canSwitch and isStale are the single tests the plan code uses.

Testing: lint, CSS lint, typecheck and build pass; the full test suite passes (2,078, including the database suite) and the later search, studio and chat runs pass (1,609). Each change has a test that fails on dev. Under next start, /data?share=zzzz and /data?spec=x send the header and a gallery share does not.

…approved planner decisions

track drops bot requests, page openGraph keeps the site name and root card, /data share and spec URLs send X-Robots-Tag, approved choices file planner decisions by choice_id, a refused chip refunds the guest prompt, shared wordings are numbered in the entity menu, and a stale switch without a current card ends read-only.
…nction

stream refunds a metered turn's prompt on any refusal, as meterTurn does. src/proxy.ts sets /data's X-Robots-Tag from dataPageRobots, the function the page metadata uses. Preview cards carry their size and type, and a gallery dataset previews with its own card.
@jm-rivera
jm-rivera merged commit 350e539 into dev Oct 29, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant