Skip to content

pd: enable log redaction by default - #24548

Open
JmPotato wants to merge 1 commit into
pingcap:masterfrom
JmPotato:docs/pd-redact-info-log-default
Open

JmPotato wants to merge 1 commit into
pingcap:masterfrom
JmPotato:docs/pd-redact-info-log-default

Conversation

@JmPotato

@JmPotato JmPotato commented Oct 26, 2026 •

Copy link
Copy Markdown
Member

What is changed, added, or deleted?

Update the documentation for security.redact-info-log to reflect that PD enables log redaction by default:

  • Change the documented default value from false to true in pd-configuration-file.md, tso-configuration-file.md, and scheduling-configuration-file.md.
  • Update log-redaction.md to explain that redaction is enabled when the configuration item is not explicitly set.
  • Document that redaction covers only logs and outputs that support redaction (PD logs, operator additional info, unsafe recovery plans, and some client-facing error messages).
  • Document that audit logs are not affected by this setting and that a small number of other logs might still contain raw keys (tracked in Clarify audit-log redaction policy and cover remaining raw-key log paths tikv/pd#11406), that this item cannot be changed online, and that a restart is required.
  • Note that TSO, Scheduling, and other PD microservices use the same setting, while TiKV and TiDB log redaction still default to disabled.

Which TiDB version(s) do your changes apply to?

  • master (the latest development version)
  • v9.0 (TiDB 9.0 versions)
  • v8.5 (TiDB 8.5 versions)
  • v8.1 (TiDB 8.1 versions)
  • v7.5 (TiDB 7.5 versions)
  • v7.1 (TiDB 7.1 versions)
  • v6.5 (TiDB 6.5 versions)

The target version in this documentation change is v9.0.0. The actual target version is subject to the release that includes tikv/pd#11402; please confirm the version number during review. If the PD change is backported to a release branch, this documentation change should be cherry-picked to the corresponding branch.

What is the related PR or file link(s)?

AI agent involvement

  • The changes in this PR were primarily made by an AI agent on behalf of the PR author.

Do your changes match any of the following descriptions?

  • Delete files
  • Change aliases
  • Need modification after applied to another branch
  • Might cause conflicts after applied to another branch

Summary by CodeRabbit

  • Documentation
    • Clarified that log redaction is enabled by default starting with v9.0.0; earlier versions default to disabled.
    • Documented which logs and outputs support redaction, along with limitations such as audit logs retaining original request content.
    • Clarified that changing the setting requires restarting the relevant service, and that TiKV and TiDB redaction settings are configured separately.

Signed-off-by: JmPotato <github@ipotato.me>
@ti-chi-bot

ti-chi-bot Bot commented Oct 26, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign breezewish for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added missing-translation-status This PR does not have translation status info. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Oct 26, 2026
@coderabbitai

coderabbitai Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Walkthrough

The documentation now states that redact-info-log defaults to true from v9.0.0 and false in earlier versions. It describes supported redaction coverage, exceptions, and restart requirements for PD, Scheduling, and TSO.

Changes: Redaction configuration documentation

Layer / File(s) Summary
Document redaction defaults and behavior
log-redaction.md, pd-configuration-file.md, scheduling-configuration-file.md, tso-configuration-file.md
Documents the version-dependent default, supported logs and outputs, exceptions, and restart requirements for the redaction setting. It also notes that TiKV and TiDB redaction remain disabled by default.

Priority: Low

Estimated code review effort: 1 (Trivial) | ~5 minutes

Change: Other

Suggested reviewers: qiancai, hfxsd

Merge Risk: Moderate · up to 7d0ee

Confirm the release that enables redaction by default before publishing that version claim. Clarify marker-mode output and list the supported settings so operators can configure redaction accurately.

Pre-merge checks: Passed 5
Check name Status Explanation
Title check Passed The title clearly identifies the primary change: enabling PD log redaction by default.
Description check Passed The description covers the required change, affected files and behavior, target version, related code and documentation links, AI involvement, and branch considerations. The version checkbox lists onl…
Docstring Coverage Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Description check

Explanation

The description covers the required change, affected files and behavior, target version, related code and documentation links, AI involvement, and branch considerations. The version checkbox lists only master while the text identifies v9.0.0, so the affected branch selection may need confirmation.

Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (4 skipped: 4 unsupported.)

Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🔇 Additional comments: 1
log-redaction.md-72-72: 1

72-72: Minor · Maintainability & Code Quality · Quick win

⚠️ Unverified finding
Verification ran but could not confirm this finding. It is shown for review, not as a verified issue.

Name all services that share this setting.

The current wording hides Resource Manager and Router behind “other PD microservices”. List the services explicitly so operators can identify the applicable configuration.

Proposed replacement
-This configuration item cannot be changed online. Restart PD after changing it. TSO, Scheduling, and other PD microservices use the same setting, and each service takes the default value of `true` when the item is not explicitly set in its configuration. TiKV and TiDB log redaction still default to disabled; when troubleshooting, disable redaction for each component and restart it as needed.
+This configuration item cannot be changed online. Restart PD after changing it. TSO, Scheduling, Resource Manager, and Router use the same setting, and each service takes the default value of `true` when the item is not explicitly set in its configuration. TiKV and TiDB log redaction still default to disabled; when troubleshooting, disable redaction for each component and restart it as needed.

Review info
Run configuration
  • Source mode: Auto
  • Repository: pingcap/docs
  • Branch: master
  • File: .coderabbit.yaml
  • Commit: d6df199
  • Layers: Repository YAML
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 48a4577b-2052-49a1-bbcc-ae66aca70185
Commits

Reviewing files that changed from the base of the PR and between d6df199 and 7d0ee5c.

Files selected for processing: 4
  • log-redaction.md
  • pd-configuration-file.md
  • scheduling-configuration-file.md
  • tso-configuration-file.md

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.

Comment thread log-redaction.md
## Log redaction in PD side

To enable log redaction in the PD side, set the value of [`security.redact-info-log`](/pd-configuration-file.md#redact-info-log-new-in-v50) to `true` or `"marker"`. This configuration value defaults to `false`, which means that log redaction is disabled.
To enable log redaction in the PD side, set the value of [`security.redact-info-log`](/pd-configuration-file.md#redact-info-log-new-in-v50) to `true` or `"marker"`. Starting from v9.0.0, if this configuration item is not explicitly set in the configuration file, it defaults to `true`, which means that log redaction is enabled. Before v9.0.0, the default value is `false`.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Functional Correctness · Quick win

🔎 Supported by static analysis

Script executed:

set -o pipefail
printf '%s\n' '--- changed paths ---'
git diff --name-status d6df19920d3937716405b3f70773ff1762f71d41 7d0ee5c5ea495e202c3f009ad208430a7393ea03 -- '*log-redaction.md' '*configuration*.md' '*pd*.md' | sed -n '1,120p'
printf '%s\n' '--- relevant diff ---'
git diff --unified=12 d6df19920d3937716405b3f009ad208430a7393ea03 7d0ee5c5ea495e202c3f009ad208430a7393ea03 -- '*log-redaction.md' '*configuration*.md' '*pd*.md' | sed -n '1,260p'
printf '%s\n' '--- occurrences in reviewed head ---'
git grep -n -F -- 'Starting from v9.0.0' 7d0ee5c5ea495e202c3f009ad208430a7393ea03 -- '*.md' || :
git grep -n -F -- 'security.redact-info-log' 7d0ee5c5ea495e202c3f009ad208430a7393ea03 -- '*.md' | sed -n '1,160p'

Repository: pingcap/docs

Length of output: 13387


Web query:

What is the merge status and release inclusion of tikv/pd pull request 11402, and which PD release first includes its change to make security.redact-info-log default to true?

Result:

<source_evidence>
<source kind="open_page">
<title>*: enable log redaction by default by JmPotato · Pull Request #11402 · tikv/pd · GitHub</title>
<location>GitHub pull request 11402 in tikv/pd (link omitted to avoid creating a cross-reference)</location>
<excerpt>citeturn1view0 [wordlim: 200] Crawled: today; Content type: text/html; Source: open({&quot;ref_id&quot;:&quot;https://github.com/tikv/pd/pull/11402&quot;,&quot;lineno&quot;:null}); Total lines: 482
L0: 【0†Skip to content】
L1: ## Navigation Menu
L2: 
L3: 【1†Sign in】
L4: 
L5: Appearance settings
L6: 
L7:   * Platform
L8: 
L9:     * AI CODE CREATION
L10:       * 【2†GitHub CopilotWrite better code with AI】
L11:       * 【3†GitHub Copilot appDirect agents from issue to merge】
L12:       * 【4†MCP RegistryIntegrate external tools】
L13: 
L14:     * DEVELOPER WORKFLOWS
L15:       * 【5†ActionsAutomate any workflow】
L16:       * 【6†CodespacesInstant dev environments】
L17:       * 【7†IssuesPlan and track work】
L18:       * 【8†Code ReviewManage code changes】
L19:       * 【9†Code QualityEnforce quality at merge】
L20:     * APPLICATION SECURITY
L21:       * 【10†GitHub Advanced SecurityFind and fix vulnerabilities】
L22:       * 【11†Code securitySecure your code as you build】
L23:       * 【12†Secret protectionStop leaks before they start】
L24: 
L25:     * EXPLORE
L26:       * 【13†Why GitHub】
L27:       * 【14†Documentation†docs.github.com】
L28:       * 【15†Blog†github.blog】
L29:       * 【16†Changelog†github.blog】
L30:       * 【17†Marketplace】
L31: 
L32: 【18†View all features】
L33: 
L34:   * Solutions
L35:     * BY COMPANY SIZE
L36:       * 【19†Enterprises】
L37:       * 【20†Small and medium teams】
L38:       * 【21†Startups】
L39:       * 【22†Nonprofits】
L40: 
L41:     * BY USE CASE
L42:       * 【23†App Modernization】
L43:       * 【24†DevSecOps】
L44:       * 【25†DevOps】
L45:       * 【26†CI/CD】
L46:       * 【27†View all use cases】
L47: 
L48:     * BY INDUSTRY
L49:       * 【28†Healthcare】
L50:       * 【29†Financial services】
L51:       * 【30†Manufacturing】
L52:       * 【31†Government】
L53:       * 【32†See all industries】
L54: 
L55: 【33†View all solutions】
L56: 
L57:   * Resources
L58:     * EXPLORE BY TOPIC
L59:       * 【34†AI】
L60:       * 【35†Software Development】
L61:       * 【36†DevOps】
L62:       * 【37†Security】
L63:       * 【38†View all topics】
L64: 
L65:     * EXPLORE BY TYPE
L66:       * 【39†Customer stories】
L67:       * 【40†Events &amp; webinars】
L68:       * 【41†Ebooks &amp; reports】
L69:       * 【42†Business insights】
L70:       * 【43†GitHub Skills†skills.github.com】
L71:     * SUPPORT &amp; SERVICES
L72:       * 【14†Documentation†docs.github.com】
L73:       * 【44†Customer support†support.github.com】
L74:       * 【45†Community forum】
L75:       * 【46†Trust center】
L76:       * 【47†Partners】
L77: 
L78: 【48†View all resources】
L79: 
L80:   * Open Source
L81: 
L82:     * COMMUNITY
L83:       * 【49†GitHub SponsorsFund open source developers】
L84:     * PROGRAMS
L85:       * 【50†Security Lab†securitylab.github.com】
L86:       * 【51†Maintainer Community†maintainers.github.com】
L87:       * 【52†GitHub Stars†stars.github.com】
L88:       * 【53†Archive Program†archiveprogram.github.com】
L89: 
L90:     * REPOSITORIES
L91:       * 【54†Topics】
L92:       * 【55†Trending】
L93:       * 【56†Collections】
L94: 
L95:   * Enterprise
L96: 
L97:     * ENTERPRISE SOLUTIONS
L98:       * 【19†Enterprise platformAI-powered developer platform】
L99:     * AVAILABLE ADD-ONS
L100:       * 【10†GitHub Advanced SecurityEnterprise-grade security features】
L101:       * 【57†Copilot for BusinessEnterprise-grade AI features】
L102:       * 【58†Premium SupportEnterprise-grade 24/7 support】
L103: 
L104:   * 【59†Pricing】
L105: 
L106: Search`/`
L107: 
L108: 【1†Sign in】
L109: 
L110: 【60†Sign up】
L111: 
L112: Appearance settings
L113: You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
L114: 
L115:   1. 【61†tikv】
L116:   2. 【62†pd】
L117: ## Repository navigation
L118: 
L119:   *   * 【62†Code】
L120:   * 【63†Issues686 (686)】
L121:   * 【64†Pull requests344 (344)】
L122:   * 【65†Actions】
…[truncated]</excerpt>
</source>
<source kind="search">
<title>pd/conf/config.toml at master · tikv/pd · GitHub</title>
<location>https://github.com/tikv/pd/blob/master/conf/config.toml</location>
<excerpt>citeturn0search0 [wordlim: 200] Crawled: last month; ## - true: enable redact log, which will replace the sensitive information with &quot;?&quot;. ... # redact-info-log = false ... ## Currently we use prometheus as metric storage, we may use PD/TiKV as metric storage later.

# config.toml

Copy path

[Input: Go to file]

Blame More file actions

Blame More file actions

## Latest commit

 

## History

History

master

## Breadcrumbs

  1. pd
  2. /conf

/

# config.toml

Copy path

Top

## File metadata and controls

  * Code

  * Blame

217 lines (192 loc) · 8.67 KB

Raw

Copy raw file

Download raw file

Open symbols panel

Edit and raw actions

1

# PD Configuration.

## Human-readable name for this pd member.

# name = &quot;&quot;

## path to the data directory, default: &quot;default.${name}&quot;.

# data-dir = &quot;&quot;

# client-urls = &quot;http://127.0.0.1:2379&quot;

## if not set, use ${client-urls}

# advertise-client-urls = &quot;&quot;

# peer-urls = &quot;http://127.0.0.1:2380&quot;

## if not set, use ${peer-urls}

# advertise-peer-urls = &quot;&quot;

## The initial cluster configuration for bootstrapping.

# initial-cluster = &quot;&quot;

# initial-cluster-state = &quot;new&quot;

## Set different tokens to prevent communication between PDs in different clusters.

# initial-cluster-token = &quot;pd-cluster&quot;

## starts a new cluster even if previously started; unsafe.

# force-new-cluster = false

## Join to an existing cluster. The value should be cluster&#39;s ${advertise-client-urls}

# join = &quot;&quot;

[security]

## Path of file that contains list of trusted SSL CAs. if set, following four settings shouldn&#39;t be empty

# cacert-path = &quot;&quot;

## Path of file that contains X509 certificate in PEM format.

# cert-path = &quot;&quot;

## Path of file that contains X509 key in PEM format.

# key-path = &quot;&quot;

## A CN which must be provided by a client

# cert-allowed-cn = [&quot;example.com&quot;]

## Whether to enable the log redaction. It can be the following values:

## - false: disable redact log.

## - true: enable redact log, which will replace the sensitive information with &quot;?&quot;.

## - &quot;MARKER&quot;: enable redact log, which will use single guillemets ‹› to enclose the sensitive information.

# redact-info-log = false

[security.encryption]

## Encryption method to use for PD data. One of &quot;plaintext&quot;, &quot;aes128-ctr&quot;, &quot;aes192-ctr&quot; and &quot;aes256-ctr&quot;.

## Defaults to &quot;plaintext&quot; if not set.

# data-encryption-method = &quot;plaintext&quot;

## Specifies how often PD rotates data encryption key. Default is 7 days.

# data-key-rotation-period = &quot;168h&quot;

## Specifies master key if encryption is enabled. There are three types of master key:

##

## * &quot;plaintext&quot;:

##

## Plaintext as a master key means no master key is given and only applicable when

## encryption is not enabled, i.e. data-encryption-method = &quot;plaintext&quot;. This type doesn&#39;t

## have sub-config items. Example:

##

## [security.encryption.master-key]

## type = &quot;plaintext&quot;

##

## * &quot;kms&quot;:

##

## Use a KMS service to supply a master key. Currently, only AWS KMS is supported. This type of

## master key is recommended for production use. Example:

##

## [security.encryption.master-key]

## type = &quot;kms&quot;

## ## KMS CMK key id. Must be a valid KMS CMK where the TiKV process has access to.

## ## In production is recommended to grant access of the CMK to TiKV using IAM.

## key-id = &quot;1234abcd-12ab-34cd-56ef-1234567890ab&quot;

## ## AWS region of the KMS CMK.

## region = &quot;us-west-2&quot;

## ## (Optional) AWS KMS service endpoint. Only required when non-default KMS endpoint is

## ## desired.

## endpoint = &quot;https://kms.us-west-2.amazonaws.com&quot;

##

## * &quot;file&quot;:

##

## Supply a custom encryption key stored in a file. It is recommended NOT to use i…[truncated]</excerpt>
</source>
<source kind="search">
<title>tikv/CHANGELOG.md at master · tikv/tikv · GitHub</title>
<location>https://github.com/tikv/tikv/blob/master/CHANGELOG.md</location>
<excerpt>citeturn0search1 [wordlim: 200] Crawled: 3 weeks ago;     * Fix a bug that some files are missed to be imported during the process of importing snapshot files when upgrading TiKV from v3.x to v4.x or v5.x #10902 ... Starting from v4.0.16, TiDB processes the conversion according to the legal UTF-8 prefix in the string #11466 ...     * Fix the issue that TiKV frequently reconnects the PD client #9818
## [5.0.0] - 2021-04-07

  * Compatibility Changes
    * Replace the `rocksdb.auto-tuned` configuration item with `rocksdb.rate-limiter-auto-tuned`
    * Delete the `raftstore.sync-log` configuration item. By default, written data is forcibly spilled to the disk. Before v5.0, you can explicitly disable `raftstore.sync-log`. Since v5.0, the configuration value is forcibly set to `true`
    * Change the default value of the `gc.enable-compaction-filter` configuration item from `false` to `true`
    * Change the default value of the `rate-limiter-auto-tuned` configuration item from `false` to `true`
  * New features
    * Support log redaction to desensitize the output log information. The configuration item `security.redact-info-log`. Its default value is `false`, which means that desensitization is disabled. To enable desensitization for tikv-server logs, set the variable value to `true`
    * Support transaction async commit
    * Support Raft joint consensus
  * Improvements
    * Enable the system to automatically adjust the data compaction speed by default to balance the contention for I/O resources between background tasks and foreground reads and writes

## [4.0.10] - 2021-01-15

  * Bug Fixes
    * Fix the wrong mapping between ready and peer #9409
    * Fix the issue that some logs are not redacted when `security.redact-info-log` is set to `true` #9314

## [4.0.9] - 2020-12-18

  * Improvements
    * Add the tag to trace the source of the `split` command #8936
    * Support dynamically changing the `pessimistic-txn.pipelined` configuration #9100
    * Reduce the impact on performance when running Backup &amp; Restore and TiDB Lightning #9098
    * Add monitoring metrics for the ingesting SST errors #9096
    * Prevent hibernation when some peers are still catching up with logs #9093
    * Increase the success rate of the pipelined pessimistic locking #9086
    * Change the default value of `apply-max-batch-size` and `store-max-batch-size` to `1024` #9020
    * Add the `max-background-flushes` configuration item #8947
    * Enable the unified read pool for the storage module by default #8887
## [4.0.8] - 2020-10-30

  * Improvements
    * Add the Fast-Tune panel page to assist performance diagnostics #8804
    * Add the `security.redact-info-log` configuration item, which redacts user data from logs #8746
    * Reformat the metafile of error codes #8877
    * Enable dynamically changing the `pessimistic-txn.pipelined` configuration #8853
    * Enable the memory profiling features by default #8801
  * Bug Fixes
    * Fix the bug that the mutex conflict in encryption causes pd-worker to process heartbeats slowly #8869
    * Fix the issue that the memory profile is mistakenly generated #8790
    * Fix the failure to back up databases on GCS when the storage class is specified #8763
    * Fix the bug that a learner cannot find a leader when the Region is restarted or newly split #8864

## [4.0.7] - 2020-09-29

  * Improvements
    * Support the JSON log format #8382
  * Bug Fixes
    * Fix the issue of unavailable Status API when TLS handshake fails #8649
</excerpt>
</source>
<source kind="search">
<title>TiKV Configuration File | TiDB Docs</title>
<location>https://website-docs-preview.vercel.app/tidb/stable/tikv-configuration-file/</location>
<excerpt>citeturn0search2 [wordlim: 200] Crawled: 4 days ago; ### `redact-info-log` New in v4.0.8 ... Value options: `true`, `false`, `&quot;on&quot;`, `&quot;off&quot;`, and `&quot;marker&quot;`. ...   * For details on how to use it, see Log redaction in TiKV side.
  * Range: `[1, 16]`
  * Default value: `1`


Configuration items related to security.

### `ca-path`

  * The path of the CA file
  * Default value: `&quot;&quot;`

### `cert-path`

  * The path of the Privacy Enhanced Mail (PEM) file that contains the X.509 certificate
  * Default value: `&quot;&quot;`

### `key-path`

  * The path of the PEM file that contains the X.509 key
  * Default value: `&quot;&quot;`

### `cert-allowed-cn`

  * A list of acceptable X.509 Common Names in certificates presented by clients. Requests are permitted only when the presented Common Name is an exact match with one of the entries in the list.
  * Default value: `[]`. This means that the client certificate CN check is disabled by default.

### `redact-info-log` New in v4.0.8

  * This configuration item enables or disables log redaction. Value options: `true`, `false`, `&quot;on&quot;`, `&quot;off&quot;`, and `&quot;marker&quot;`. The `&quot;on&quot;`, `&quot;off&quot;`, and `&quot;marker&quot;` options are introduced in v8.3.0.
  * If the configuration item is set to `false` or `&quot;off&quot;`, log redaction is disabled.
  * If the configuration item is set to `true` or `&quot;on&quot;`, all user data in the log is replaced by `?`.
  * If the configuration item is set to `&quot;marker&quot;`, all user data in the log is wrapped in `‹ ›`. If user data contains `‹` or `›`, `‹` is escaped as `‹‹`, and `›` is escaped as `››`. Based on the marked logs, you can decide whether to desensitize the marked information when the logs are displayed.
  * Default value: `false`
  * For details on how to use it, see Log redaction in TiKV side.

</excerpt>
</source>
<source kind="search">
<title>Log Redaction | TiDB Docs</title>
<location>https://docs.pingcap.com/tidb/stable/log-redaction/</location>
<excerpt>citeturn0search3 [wordlim: 200] Crawled: last month;     [2024/07/02 11:35:01.426 +08:00] [INFO] [conn.go:1146] [&quot;command dispatched failed&quot;] [conn=1482686470] [session_alias=] [connInfo=&quot;id:1482686470, addr:127.0.0.1:52258 status:10, collation:utf8mb4_0900_ai_ci, user:root&quot;] [command=Query] [status=&quot;inTxn:0, autocommit:1&quot;] [sql=&quot;insert into `t` values ( ‹1› ) , ( ‹1› )&quot;] [txn_mode=PESSIMISTIC] [timestamp=450859185309483010] [err=&quot;[kv:1062]Duplicate entry &#39;‹1›&#39; for key &#39;t.a&#39;&quot;] ... To enable log redaction in the TiKV side, set the value of `security.redact-info-log` to `true` or `&quot;marker&quot;`. ... To enable log redaction in the PD side, set the value of `security.redact-info-log` to `true` or `&quot;marker&quot;`.
    
    `
    
    create table t (a int, unique key (a));
    Query OK, 0 rows affected (0.07 sec)
    
    insert into t values (1),(1);
    ERROR 1062 (23000): Duplicate entry &#39;‹1›&#39; for key &#39;t.a&#39;
    
    
    `

The error log is as follows:
    
    `
    
    [2024/07/02 11:35:01.426 +08:00] [INFO] [conn.go:1146] [&quot;command dispatched failed&quot;] [conn=1482686470] [session_alias=] [connInfo=&quot;id:1482686470, addr:127.0.0.1:52258 status:10, collation:utf8mb4_0900_ai_ci, user:root&quot;] [command=Query] [status=&quot;inTxn:0, autocommit:1&quot;] [sql=&quot;insert into `t` values ( ‹1› ) , ( ‹1› )&quot;] [txn_mode=PESSIMISTIC] [timestamp=450859185309483010] [err=&quot;[kv:1062]Duplicate entry &#39;‹1›&#39; for key &#39;t.a&#39;&quot;]
    
    
    `

As you can see from the preceding error log, after you set `tidb_redact_log` to `MARKER`, TiDB marks sensitive information using `‹ ›` in the log. You can customize redaction rules to handle sensitive information in the log as needed.

## Log redaction in TiKV side

To enable log redaction in the TiKV side, set the value of `security.redact-info-log` to `true` or `&quot;marker&quot;`. This configuration value defaults to `false`, which means that log redaction is disabled.

## Log redaction in PD side

To enable log redaction in the PD side, set the value of `security.redact-info-log` to `true` or `&quot;marker&quot;`. This configuration value defaults to `false`, which means that log redaction is disabled.

## Log redaction in TiFlash side

To enable log redaction in the TiFlash side, set both the `security.redact_info_log` value in tiflash-server and the `security.redact-info-log` value in tiflash-learner to `true` or `&quot;marker&quot;`. Both configuration values default to `false`, which means that log redaction is disabled.</excerpt>
</source>
<omitted_sources count="25">Additional source records were omitted from this bounded result.</omitted_sources>
</source_evidence>

Citations:

- 1: GitHub pull request 11402 in tikv/pd (link omitted to avoid creating a cross-reference)
- 2: GitHub pull request 11402 in tikv/pd (link omitted to avoid creating a cross-reference)

Script executed:

set -o pipefail
printf '%s\n' '--- local revision availability ---'
for rev in d6df19920d3937716405b3f70773ff1762f71d41 7d0ee5c5ea495e202c3f009ad208430a7393ea03; do
  printf '%s: ' "$rev"
  git cat-file -t "$rev" 2>&1 || true
done
printf '%s\n' '--- exact documentation diff ---'
git diff --unified=8 d6df19920d3937716405b3f70773ff1762f71d41 7d0ee5c5ea495e202c3f009ad208430a7393ea03 -- log-redaction.md pd-configuration-file.md scheduling-configuration-file.md tso-configuration-file.md
printf '%s\n' '--- PR metadata ---'
curl -L --fail --silent --show-error https://api.github.com/repos/tikv/pd/pulls/11402 | python3 -c 'import json,sys; d=json.load(sys.stdin); print(json.dumps({k:d.get(k) for k in ("state","merged","merged_at","merge_commit_sha","title","base","head","html_url")}, indent=2))'
printf '%s\n' '--- release tags ---'
curl -L --fail --silent --show-error https://api.github.com/repos/tikv/pd/git/ref/tags/v9.0.0 | python3 -c 'import json,sys; d=json.load(sys.stdin); print(json.dumps(d, indent=2))'
printf '%s\n' '--- tag comparison against PR merge commit ---'
merge_sha=$(curl -L --fail --silent --show-error https://api.github.com/repos/tikv/pd/pulls/11402 | python3 -c 'import json,sys; print(json.load(sys.stdin).get("merge_commit_sha") or "")')
if test -n "$merge_sha"; then
  curl -L --fail --silent --show-error "https://api.github.com/repos/tikv/pd/compare/v9.0.0...$merge_sha" | python3 -c 'import json,sys; d=json.load(sys.stdin); print(json.dumps({k:d.get(k) for k in ("status","ahead_by","behind_by","total_commits")}, indent=2))'
else
  echo 'No merge commit SHA available'
fi

Repository: pingcap/docs

Length of output: 22114


Do not publish the v9.0.0 default yet.

PD PR #11402 is still open and is not merged into master. Update the v9.0.0 wording in log-redaction.md, pd-configuration-file.md, scheduling-configuration-file.md, and tso-configuration-file.md to the first release that contains the change.

Comment thread log-redaction.md
To enable log redaction in the PD side, set the value of [`security.redact-info-log`](/pd-configuration-file.md#redact-info-log-new-in-v50) to `true` or `"marker"`. This configuration value defaults to `false`, which means that log redaction is disabled.
To enable log redaction in the PD side, set the value of [`security.redact-info-log`](/pd-configuration-file.md#redact-info-log-new-in-v50) to `true` or `"marker"`. Starting from v9.0.0, if this configuration item is not explicitly set in the configuration file, it defaults to `true`, which means that log redaction is enabled. Before v9.0.0, the default value is `false`.

Redaction applies only to logs and outputs that support redaction, including PD logs, operator additional info, [Online Unsafe Recovery](/online-unsafe-recovery.md) plans, and some error messages returned to clients. Keys in operator additional info are displayed as a redaction placeholder rather than the raw key. Audit logs are not affected by this configuration item and still record the original request content. A small number of other logs might still contain raw keys. See [#11406](https://github.com/tikv/pd/issues/11406).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Security & Privacy · Quick win

Document the mode-specific operator-key output.

This sentence applies the placeholder claim to both supported modes. The reviewed implementation displays 3F in true mode, while the "marker" path can emit hex-encoded marker and key content. Align this text with the PD configuration reference so users do not treat "marker" as the same placeholder mode. (github.com)

Proposed replacement
--- "a/log-redaction.md"
+++ "b/log-redaction.md"
@@ -67,7 +67,7 @@
 
 To enable log redaction in the PD side, set the value of [`security.redact-info-log`](/pd-configuration-file.md#redact-info-log-new-in-v50) to `true` or `"marker"`. Starting from v9.0.0, if this configuration item is not explicitly set in the configuration file, it defaults to `true`, which means that log redaction is enabled. Before v9.0.0, the default value is `false`.
 
-Redaction applies only to logs and outputs that support redaction, including PD logs, operator additional info, [Online Unsafe Recovery](/online-unsafe-recovery.md) plans, and some error messages returned to clients. Keys in operator additional info are displayed as a redaction placeholder rather than the raw key. Audit logs are not affected by this configuration item and still record the original request content. A small number of other logs might still contain raw keys. See [#11406](https://github.com/tikv/pd/issues/11406).
+Redaction applies only to logs and outputs that support redaction, including PD logs, operator additional info, [Online Unsafe Recovery](/online-unsafe-recovery.md) plans, and some error messages returned to clients. With `true`, keys in operator additional info are displayed as `3F` rather than raw keys. With `"marker"`, operator-key formatting uses a different representation; see [#11407](https://github.com/tikv/pd/issues/11407). Audit logs are not affected by this configuration item and still record the original request content. A small number of other logs might still contain raw keys. See [#11406](https://github.com/tikv/pd/issues/11406).
 
 This configuration item cannot be changed online. Restart PD after changing it. TSO, Scheduling, and other PD microservices use the same setting, and each service takes the default value of `true` when the item is not explicitly set in its configuration. TiKV and TiDB log redaction still default to disabled; when troubleshooting, disable redaction for each component and restart it as needed.
 
Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
Redaction applies only to logs and outputs that support redaction, including PD logs, operator additional info, [Online Unsafe Recovery](/online-unsafe-recovery.md) plans, and some error messages returned to clients. Keys in operator additional info are displayed as a redaction placeholder rather than the raw key. Audit logs are not affected by this configuration item and still record the original request content. A small number of other logs might still contain raw keys. See [#11406](https://github.com/tikv/pd/issues/11406).
Redaction applies only to logs and outputs that support redaction, including PD logs, operator additional info, [Online Unsafe Recovery](/online-unsafe-recovery.md) plans, and some error messages returned to clients. With `true`, keys in operator additional info are displayed as `3F` rather than raw keys. With `"marker"`, operator-key formatting uses a different representation; see [#11407](https://github.com/tikv/pd/issues/11407). Audit logs are not affected by this configuration item and still record the original request content. A small number of other logs might still contain raw keys. See [#11406](https://github.com/tikv/pd/issues/11406).
🧰 Tools
🪛 LanguageTool

[style] ~70-~70: Specify a number, remove phrase, use “a few”, or use “some”
Context: ...ll record the original request content. A small number of other logs might still contain raw keys...

(SMALL_NUMBER_OF)

Comment thread pd-configuration-file.md
+ Optional value: `false`, `true`, `"marker"`
+ Default value: `false`
+ Default value: starting from v9.0.0, the default value is `true`. Before v9.0.0, the default value is `false`.
+ Redaction applies only to logs and outputs that support redaction, including PD logs, operator additional info, [Online Unsafe Recovery](/online-unsafe-recovery.md) plans, and some error messages returned to clients. Keys in operator additional info are displayed as a redaction placeholder rather than the raw key.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Security & Privacy · Quick win

Keep the configuration reference accurate for "marker".

This entry also says that operator additional info always uses a placeholder. In true mode the reviewed implementation displays 3F; "marker" uses a different representation and can encode key content. Keep this entry aligned with log-redaction.md. (github.com)

Proposed replacement
-Redaction applies only to logs and outputs that support redaction, including PD logs, operator additional info, [Online Unsafe Recovery](/online-unsafe-recovery.md) plans, and some error messages returned to clients. Keys in operator additional info are displayed as a redaction placeholder rather than the raw key.
+Redaction applies only to logs and outputs that support redaction, including PD logs, operator additional info, [Online Unsafe Recovery](/online-unsafe-recovery.md) plans, and some error messages returned to clients. With `true`, keys in operator additional info are displayed as `3F` rather than raw keys. With `"marker"`, operator-key formatting uses a different representation; see [#11407](https://github.com/tikv/pd/issues/11407).

- Controls whether to enable log redaction in Scheduling node logs.
- When you set the configuration value to `true`, user data is redacted in Scheduling node logs.
- Default value: `false`
- Default value: starting from v9.0.0, the default value is `true`. Before v9.0.0, the default value is `false`.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Functional Correctness · Quick win

List the supported values for the shared setting.

The shared PD security setting accepts false, true, and "marker". This reference currently documents only the true behavior. Add the optional-value entry here and in tso-configuration-file.md so users can discover all supported modes. (github.com)

Proposed insertion
 - When you set the configuration value to `true`, user data is redacted in Scheduling node logs.
+- Optional value: `false`, `true`, `"marker"`
 - Default value: starting from v9.0.0, the default value is `true`. Before v9.0.0, the default value is `false`.
Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- Default value: starting from v9.0.0, the default value is `true`. Before v9.0.0, the default value is `false`.
- Optional value: `false`, `true`, `"marker"`
- Default value: starting from v9.0.0, the default value is `true`. Before v9.0.0, the default value is `false`.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

missing-translation-status This PR does not have translation status info. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant