Repository navigation
fix: issue triage quick wins (chunk cap, map, path search) - #154
PatrickSys wants to merge 1 commit into
Conversation
Clarify searchable chunk limits, exclude vendored bundles from bounded map, and boost keyword hits for file-like query tokens. Adds regression tests.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 28e71e31a2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| /(?:^|\/)[^/]*\.(?:generated|gen|min)\.[^/]+$/i, | ||
| /(?:^|\/)raw(?:\/|$)/i, | ||
| /(?:^|\/)live-from-ai(?:\/|$)/i, | ||
| /[^/]+-[A-Za-z0-9]{6,}\.(?:js|css|mjs|cjs)$/i, |
There was a problem hiding this comment.
In the default bounded map, this pattern also excludes handwritten files whose final hyphen-separated word has six or more letters. For example, the existing scripts/sync-grammars.mjs matches it, as would request-handler.js. Because isMapEligiblePath filters graph nodes, edges, and chunks, these legitimate source files disappear from architecture sections rather than just generated bundles being removed. Narrow the bundle detection so ordinary descriptive filenames remain eligible.
Useful? React with 👍 / 👎.
| fuseResults = [ | ||
| ...pathRescue, | ||
| ...rescue, | ||
| ...fuseResults.filter(({ item }) => !seen.has(item.id)) |
There was a problem hiding this comment.
When a filename-bearing query triggers rescue and a chunk matches both its path token and the lexical rescue criteria, that chunk appears in both pathRescue and rescue. The seen set only filters the original Fuse results, so the duplicate survives selectCurrentCandidates, consumes two candidate slots, and causes collectHybridMatches to append two keyword ranks for the same query variant. fuseAndScoreResults then sums both contributions, artificially inflating relevance and confidence. Deduplicate the rescue lists against each other before candidate selection.
Useful? React with 👍 / 👎.
| searchableChunks: stats.searchableChunks, | ||
| chunkLimitApplied: stats.chunkLimitApplied, |
There was a problem hiding this comment.
After indexing a capped repository, restarting without source changes takes the incremental short-circuit, whose persisted-stats reader restores only indexedFiles, totalChunks, and totalFiles. Although these new fields are written here, neither is restored, so formatIndexingCompleteLine falls back to reporting all analyzed chunks as indexed—for example, 88424 chunks instead of 5000/88424 chunks searchable. Restore searchableChunks and chunkLimitApplied in the no-change path so the new reporting remains accurate on subsequent starts.
Useful? React with 👍 / 👎.
 [Medium impact] The changes reach indexing reports, project search results, and bounded codebase summaries. Fix the map exclusions and search ranking before merging, and satisfy the documentation requirements. Findings
SummaryThis PR adds chunk-cap reporting, filters bounded maps, and boosts filename-like search terms.
Reviews (1) · Last reviewed commit: "fix: triage quick wins for chunk cap, ma..." · Reviewed by Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| /(?:^|\/)[^/]*\.(?:generated|gen|min)\.[^/]+$/i, | ||
| /(?:^|\/)raw(?:\/|$)/i, | ||
| /(?:^|\/)live-from-ai(?:\/|$)/i, | ||
| /[^/]+-[A-Za-z0-9]{6,}\.(?:js|css|mjs|cjs)$/i, |
There was a problem hiding this comment.
The bundle pattern also matches maintained source such as query-builder.js, error-handler.js, and scripts/check-agent-instructions.mjs. The default bounded map removes these files and their graph edges, hiding useful source from the map. Restrict the exclusion to paths with evidence that they are generated bundles, and test an ordinary hyphenated filename.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
| fuseResults = [ | ||
| ...pathRescue, | ||
| ...rescue, | ||
| ...fuseResults.filter(({ item }) => !seen.has(item.id)) | ||
| ].sort((a, b) => (a.score ?? 0) - (b.score ?? 0)); |
There was a problem hiding this comment.
pathRescue and rescue can contain the same chunk, but seen only removes duplicates from the trailing Fuse results. A natural-language script query can match both the script's path and multiple query words. The repeated chunk consumes candidate slots and gets two contributions to its final search score. Deduplicate the combined lists by item.id before selecting candidates.
| const pathRescue: FuseResult<CodeChunk>[] = this.matchChunksByPathTokens(query, filters).map( | ||
| (chunk, refIndex) => ({ | ||
| item: chunk, | ||
| refIndex, | ||
| score: 0 | ||
| }) |
There was a problem hiding this comment.
Every path match gets score: 0, including loose substring matches. A query naming NOW.md also matches archive/now.md.bak through includes(token), giving both the maximum keyword score. Matches keep index order, and the candidate limit applies before the later boosts. Enough loose matches can exclude NOW.md entirely. Rank exact filenames above stem and substring matches before applying the limit.
| searchableChunks: stats.searchableChunks, | ||
| chunkLimitApplied: stats.chunkLimitApplied, |
There was a problem hiding this comment.
The new cap fields are saved but not restored when indexing finds no changed files. That branch restores only indexedFiles, totalChunks, and totalFiles, then returns. After a capped build, an unchanged run loses the searchable count and prints the analyzed total without the cap warning. Restore both fields from the saved stats and test a capped build followed by an unchanged run.
| - `projects[].analyzerHints.analyzer`: prefers a registered analyzer by name for that project and falls back safely when the name is missing or invalid | ||
| - `projects[].analyzerHints.extensions`: adds project-local source extensions for indexing and auto-refresh watching without changing defaults for other projects | ||
| - `projects[].parsing.maxChunks`: sets the maximum number of searchable chunks indexed for that project (requires 2.4.0, not yet published) | ||
| - `projects[].parsing.maxChunks`: sets the maximum number of searchable chunks indexed for that project (default 5000; configurable via `~/.codebase-context/config.json`) |
There was a problem hiding this comment.
Removing the unreleased label presents projects[].parsing.maxChunks as available to npm users. The guide says unmarked features apply to the published package, but the v2.3.0 config reader does not read parsing; this option was added later. Keep an unreleased or built-checkout qualifier here and in README.md. The repository requires public docs not to claim capabilities that are not shipped, so this must be corrected before merging.
Context Used: AGENTS.md (source)
| for (const token of extractQueryPathTokens(query)) { | ||
| const base = path.basename(relativePathLower); | ||
| const stem = base.replace(/\.[^.]+$/, ''); | ||
| if ( | ||
| base === token || | ||
| relativePathLower.endsWith(`/${token}`) || | ||
| stem === token.replace(/\.[^.]+$/, '') | ||
| ) { | ||
| score = Math.min(1.0, score + 0.45); | ||
| } else if (stem.includes(token) || relativePathLower.includes(token)) { | ||
| score = Math.min(1.0, score + 0.25); |
There was a problem hiding this comment.
The new ranking adds 0.45 for exact path matches and 0.25 for partial matches without explaining the weights or how repeated tokens should affect the score. The repository requires every ranking heuristic to include research citations or a principled justification. Before merging, document the reason for these boosts, including their interaction with the score cap and the full-name/stem tokens.
Context Used: AGENTS.md (source)
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Summary
Open-issue triage for
PatrickSys/codebase-context(7 open). Skipped implementation overlap with open PRs #145, #146, and #147 and closed PR #142.Triage (by category)
OPENAI_BASE_URL— PR #145(GitHub token could not add issue comments/labels; suggested labels: #68 bug+help wanted, #70/#71 enhancement+good first issue, #149/#151 bug, #150/#152 bug+help wanted.)
Changes
searchableChunks/chunkLimitAppliedon indexing stats; clearer cap warning with~/.codebase-context/config.jsonhint;Complete: 5000/88424 chunks searchableline; README/capabilities no longer claimmaxChunksneeds unpublished 2.4.0./raw/,live-from-ai/, and Vite-style hashed bundles.extractQueryPathTokens+ keyword rescue/boost forNOW.md,AGENTS.md, hyphenated script stems.Tests
tests/indexer-chunk-limit-reporting.test.tstests/search-path-token-boost.test.tstests/codebase-map.test.ts(vendored bundle exclusion)Fixes #149
Fixes #151
Fixes #150
Fixes #152