Add projectionHint to SG Parquet mode by SungJin1212 · Pull Request #7206 · cortexproject/cortex

SungJin1212 · 2026-01-13T12:57:45Z

This PR adds -blocks-storage.bucket-store.honor-projection-hints CLI flag. If enabled, Store Gateway in Parquet mode will honor projection hints and only materialize requested labels.

The benchmark results

goos: darwin
goarch: arm64
pkg: github.com/cortexproject/cortex/pkg/storegateway
cpu: Apple M4 Max
BenchmarkParquetBucketStore_ProjectionHints
BenchmarkParquetBucketStore_ProjectionHints/series_100
BenchmarkParquetBucketStore_ProjectionHints/series_100/without_projection_hints
BenchmarkParquetBucketStore_ProjectionHints/series_100/without_projection_hints-14         	    1671	    776046 ns/op	  480315 B/op	    4336 allocs/op
BenchmarkParquetBucketStore_ProjectionHints/series_100/with_projection_hints
BenchmarkParquetBucketStore_ProjectionHints/series_100/with_projection_hints-14            	    1904	    597504 ns/op	  395957 B/op	    3969 allocs/op
BenchmarkParquetBucketStore_ProjectionHints/series_1000
BenchmarkParquetBucketStore_ProjectionHints/series_1000/without_projection_hints
BenchmarkParquetBucketStore_ProjectionHints/series_1000/without_projection_hints-14        	     585	   1745712 ns/op	 3887004 B/op	   29645 allocs/op
BenchmarkParquetBucketStore_ProjectionHints/series_1000/with_projection_hints
BenchmarkParquetBucketStore_ProjectionHints/series_1000/with_projection_hints-14           	     759	   1648167 ns/op	 2916027 B/op	   27443 allocs/op
BenchmarkParquetBucketStore_ProjectionHints/series_10000
BenchmarkParquetBucketStore_ProjectionHints/series_10000/without_projection_hints
BenchmarkParquetBucketStore_ProjectionHints/series_10000/without_projection_hints-14       	     100	  10303832 ns/op	37756795 B/op	  282501 allocs/op
BenchmarkParquetBucketStore_ProjectionHints/series_10000/with_projection_hints
BenchmarkParquetBucketStore_ProjectionHints/series_10000/with_projection_hints-14          	      97	  11581134 ns/op	28394294 B/op	  262156 allocs/op
PASS

Process finished with the exit code 0

Which issue(s) this PR fixes:
Fixes #

Checklist

Tests updated
Documentation added
CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

yeya24 · 2026-01-13T19:00:28Z

 	f.Float64Var(&cfg.TokenBucketBytesLimiter.TouchedChunksTokenFactor, "blocks-storage.bucket-store.token-bucket-bytes-limiter.touched-chunks-token-factor", 1, "Multiplication factor used for touched chunks token")
 	f.IntVar(&cfg.MatchersCacheMaxItems, "blocks-storage.bucket-store.matchers-cache-max-items", 0, "Maximum number of entries in the regex matchers cache. 0 to disable.")
 	cfg.ParquetShardCache.RegisterFlagsWithPrefix("blocks-storage.bucket-store.", f)
+	f.BoolVar(&cfg.HonorProjectionHints, "blocks-storage.bucket-store.honor-projection-hints", false, "[Experimental] If enabled, Store Gateway will honor projection hints and only materialize requested labels. It is only effect when `-blocks-storage.bucket-store.bucket-store-type` is a parquet.")


is parquet Remove a.

Just to double check, if I want to actually use parquet store gateway projection, I need to also honor projection hints in Querier (but not enable parquet querier)?

yeah, even if the Parquet Querier is disabled (using the standard Querier), the Parquet Store Gateway relies on these hints to perform projection pushdown (Querier send the projection hints).

Seems that even if we don't enable honor projection hints in querier, it still propagates projection hints to store gateway. I don't see you enable that in the integration test.

yeya24 · 2026-01-13T19:03:20Z

+		storageHints.ProjectionLabels = queryHints.ProjectionLabels
+
+		// Reset projection hints if not all parquet shard have the hash column.
+		if !allParquetBlocksHaveHashColumn(shards) {


We should only project if it is projection include. Add that check first can avoid checking all blocks of hash column when unnecessary

yeya24 · 2026-01-14T16:56:18Z

+					vec := aggResult.(model.Vector)
+					require.Len(t, vec, 1)
+					require.Equal(t, model.LabelValue("series_1"), vec[0].Metric["series_1"])
+					require.Equal(t, expectedVector1[0].Value, vec[0].Value)


I want to avoid adding more test cases to this test.
I think overall the test is very flaky and it keeps failing for different reasons. I would prefer separate tests for parquet store gateway mode tbh.

yeya24 · 2026-01-14T21:41:03Z

+				storageHints.ProjectionInclude = false
+				storageHints.ProjectionLabels = nil
+			}
+		}


I would also reset the projection hints if it is not projection include. Same as what we do in querier.

cortex/pkg/querier/querier.go

Line 455 in 28ad1ac

if !sp.ProjectionInclude || q.distributor.UseQueryable(q.now, mint, maxt) {

If we don't do this then it can do wrong below as we only add series hash if it is projection include. In general it is hard to estimate the cost of projection not include and it can be more expensive

SungJin1212 · 2026-01-15T07:08:46Z

@yeya24
I have applied the fix and added dedicated test code.

sandy2008

Thanks. The earlier review points look addressed. I checked that the new TestQuerier_ProjectionHints* and TestParquetBucketStore_* cases fail without the querier and store-gateway changes, and that the generated config reference matches. There's one problem with how the V1 fallback works across multiple store-gateways (inline). The test (arm64) failure is an unrelated port collision in pkg/util/tls/test.

sandy2008 · 2026-10-28T12:19:18Z

+		lbls := make([]string, 0, len(queryHints.ProjectionLabels)+1)
+		storageHints.ProjectionLabels = append(lbls, queryHints.ProjectionLabels...)
+
+		if !allParquetBlocksHaveHashColumn(shards) {


This fallback only sees the blocks this gateway was asked for. The querier sends the same projection hints to every store-gateway, so if one gateway holds a V1 block and another holds only V2 blocks, the first returns full label sets and the second returns the projected labels plus s_series_hash.

blocksStoreQuerier merges the responses with storage.NewMergeSeriesSet(…, storage.ChainedSeriesMerge), which only merges identical label sets. The same series therefore comes back twice, and overlapping samples (e.g. a source block and its compacted block both still in the bucket) are no longer deduplicated. I reproduced the split with buildSelectHints plus that merge: V1-only shards → no projection, V2-only → [job s_series_hash], 2 series after the merge.

The direct parquet queryable avoids this by deciding once for all selected blocks before fan-out (allParquetBlocksHaveHashColumn over the bucket-index blocks in parquet_queryable.go). queryWithConsistencyCheck already has those blocks (knownBlocks, with Parquet.Version), so the hints could be reset there when any selected block is V1.

It makes sense, I applied it to the latest commit. Thanks.

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>

Signed-off-by: Sungjin1212 <tjdwls1201@gmail.com>

pull-request-size Bot added the size/L label Jan 13, 2026

dosubot Bot added component/store-gateway type/feature labels Jan 13, 2026

yeya24 reviewed Jan 13, 2026

View reviewed changes

yeya24 reviewed Jan 14, 2026

View reviewed changes

SungJin1212 force-pushed the Add-ProjectionHints-to-SG branch from 903d00e to d9638cc Compare January 15, 2026 07:01

pull-request-size Bot added size/XL and removed size/L labels Jan 15, 2026

SungJin1212 requested a review from yeya24 January 22, 2026 01:47

SungJin1212 force-pushed the Add-ProjectionHints-to-SG branch 2 times, most recently from 6e47284 to 1c9539b Compare February 16, 2026 04:37

SungJin1212 force-pushed the Add-ProjectionHints-to-SG branch from 9f5b844 to d860aff Compare March 3, 2026 04:36

SungJin1212 force-pushed the Add-ProjectionHints-to-SG branch 3 times, most recently from 8b10151 to f033993 Compare April 27, 2026 08:24

SungJin1212 force-pushed the Add-ProjectionHints-to-SG branch 5 times, most recently from b20e2e2 to a2b1d29 Compare June 19, 2026 05:47

SungJin1212 force-pushed the Add-ProjectionHints-to-SG branch from b45fb3f to 30437f2 Compare October 6, 2026 11:34

SungJin1212 requested a review from a team as a code owner October 6, 2026 11:34

pull-request-size Bot added size/XXL and removed size/XL labels Oct 6, 2026

SungJin1212 force-pushed the Add-ProjectionHints-to-SG branch from 6792a7b to bac13d7 Compare October 12, 2026 02:14

sandy2008 reviewed Oct 28, 2026

View reviewed changes

SungJin1212 added 2 commits October 30, 2026 20:27

add support for honoring projection hints in Parquet mode bucket store

96a8ac6

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>

update doc

34347ff

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>

SungJin1212 added 2 commits October 30, 2026 20:27

fix lint

992671d

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>

reset resetting projection hints based on Parquet block versions

52d1f96

Signed-off-by: Sungjin1212 <tjdwls1201@gmail.com>

SungJin1212 force-pushed the Add-ProjectionHints-to-SG branch from bac13d7 to 52d1f96 Compare October 30, 2026 12:01

rename functions and update tests for clarity on hash column checks

6ec078d

Signed-off-by: Sungjin1212 <tjdwls1201@gmail.com>

SungJin1212 force-pushed the Add-ProjectionHints-to-SG branch from 2b82175 to 6ec078d Compare October 30, 2026 12:29

Navigation Menu

Repository navigation

Add projectionHint to SG Parquet mode - #7206

Add projectionHint to SG Parquet mode#7206
SungJin1212 wants to merge 5 commits into
cortexproject:masterfrom
SungJin1212:Add-ProjectionHints-to-SG

SungJin1212 commented Jan 13, 2026 •

edited

Loading

Uh oh!

yeya24 Jan 13, 2026

Uh oh!

yeya24 Jan 13, 2026

Uh oh!

SungJin1212 Jan 14, 2026

Uh oh!

yeya24 Jan 14, 2026

Uh oh!

yeya24 Jan 13, 2026

Uh oh!

Uh oh!

Uh oh!

yeya24 Jan 14, 2026

Uh oh!

yeya24 Jan 14, 2026

Uh oh!

SungJin1212 commented Jan 15, 2026

Uh oh!

sandy2008 left a comment

Uh oh!

sandy2008 Oct 28, 2026

Uh oh!

SungJin1212 Oct 30, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

SungJin1212 commented Jan 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

The benchmark results

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SungJin1212 commented Jan 15, 2026

Uh oh!

sandy2008 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

SungJin1212 commented Jan 13, 2026 •

edited

Loading