[wyatt-holden]

Hi there - my name's Wyatt Holden and I'm a Software Engineer at Microsoft :) I've been working with Derrick Stolee over the past few weeks on optimizing git in the Office mono-repo. We found some issues around slow checkouts on mono-repos in an automated environment:

When leaving a detached HEAD, git checkout and git switch call orphaned_commit_warning() to warn about commits that are not reachable from any existing ref or the destination HEAD. This requires enumerating refs and walking commit history, which can be expensive in large repositories.

In an internal mono-repo, Trace2 showed tree unpacking finishing approximately 12 seconds into checkout, but the process continuing for roughly another 193 seconds. A stack dump captured the main thread inside  orphaned_commit_warning(), enumerating loose refs and opening ref files. Running checkout with --quiet, which bypasses this routine, reduced the observed runtime to approximately 14 seconds. However, --quiet also suppresses unrelated output.

This two-commit series adds a targeted opt-out and enables it in Scalar:

1. Make orphaned-commit checks configurable. Add  checkout.orphanedCommitWarning , enabled by default, for both checkout and switch. Setting it to false skips the entire check while retaining the previous HEAD description and normal checkout output. When an enabled check takes more than two seconds, advice suggests the opt-out and explains the consequence. This advice is controlled by  advice.checkoutOrphanedCommitWarning . A new Trace2 region exposes the routine’s duration.
2. Disable the check in Scalar’s recommended configuration. Recommend checkout.orphanedCommitWarning=false through Scalar’s existing configuration mechanism, preserving explicit user settings. Update documentation and add coverage for clone, register, reconfigure, and user overrides.

Ordinary Git retains its existing default behavior. Users who disable the check lose the immediate warning about commits left behind; those commits can still be recovered through the HEAD reflog while the relevant entries and objects remain available.

CC: gitster@pobox.com, stolee@gmail.com, wyattj.holden@gmail.com

[gitgitgadget]

Welcome to GitGitGadget

Hi @wyatt-holden, and welcome to GitGitGadget, the GitHub App to send patch series to the Git mailing list from GitHub Pull Requests.

Please make sure that either:

• Your Pull Request has a good description, if it consists of multiple commits, as it will be used as cover letter.
• Your Pull Request description is empty, if it consists of a single commit, as the commit message should be descriptive enough by itself.

You can CC potential reviewers by adding a footer to the PR description with the following syntax:

CC: Revi Ewer <revi.ewer@example.com>, Ill Takalook <ill.takalook@example.net>

NOTE: DO NOT copy/paste your CC list from a previous GGG PR's description,
because it will result in a malformed CC list on the mailing list. See
example.

Also, it is a good idea to review the commit messages one last time, as the Git project expects them in a quite specific form:

• the lines should not exceed 76 columns,
• the first line should be like a header and typically start with a prefix like "tests:" or "revisions:" to state which subsystem the change is about, and
• the commit messages' body should be describing the "why?" of the change.
• Finally, the commit messages should end in a Signed-off-by: line matching the commits' author.

It is in general a good idea to await the automated test ("Checks") in this Pull Request before contributing the patches, e.g. to avoid trivial issues such as unportable code.

Contributing the patches

Before you can contribute the patches, your GitHub username needs to be added to the list of permitted users. Any already-permitted user can do that, by adding a comment to your PR of the form /allow. A good way to find other contributors is to locate recent pull requests where someone has been /allowed:

• Search: is:pr is:open "/allow"

Both the person who commented /allow and the PR author are able to /allow you.

An alternative is the channel #git-devel on the Libera Chat IRC network:

<newcontributor> I've just created my first PR, could someone please /allow me? https://github.com/gitgitgadget/git/pull/12345
<veteran> newcontributor: it is done
<newcontributor> thanks!

Once on the list of permitted usernames, you can contribute the patches to the Git mailing list by adding a PR comment /submit.

If you want to see what email(s) would be sent for a /submit request, add a PR comment /preview to have the email(s) sent to you. You must have a public GitHub email address for this. Note that any reviewers CC'd via the list in the PR description will not actually be sent emails.

After you submit, GitGitGadget will respond with another comment that contains the link to the cover letter mail in the Git mailing list archive. Please make sure to monitor the discussion in that thread and to address comments and suggestions (while the comments and suggestions will be mirrored into the PR by GitGitGadget, you will still want to reply via mail).

If you do not want to subscribe to the Git mailing list just to be able to respond to a mail, you can download the mbox from the Git mailing list archive (click the (raw) link), then import it into your mail program. If you use GMail, you can do this via:

curl -g --user "<EMailAddress>:<Password>" \
    --url "imaps://imap.gmail.com/INBOX" -T /path/to/raw.txt

To iterate on your change, i.e. send a revised patch or patch series, you will first want to (force-)push to the same branch. You probably also want to modify your Pull Request description (or title). It is a good idea to summarize the revision by adding something like this to the cover letter (read: by editing the first comment on the PR, i.e. the PR description):

Changes since v1:
- Fixed a typo in the commit message (found by ...)
- Added a code comment to ... as suggested by ...
...

To send a new iteration, just add another PR comment with the contents: /submit.

Need help?

New contributors who want advice are encouraged to join git-mentoring@googlegroups.com, where volunteers who regularly contribute to Git are willing to answer newbie questions, give advice, or otherwise provide mentoring to interested contributors. You must join in order to post or view messages, but anyone can join.

You may also be able to find help in real time in the developer IRC channel, #git-devel on Libera Chat. Remember that IRC does not support offline messaging, so if you send someone a private message and log out, they cannot respond to you. The scrollback of #git-devel is archived, though.

[gitgitgadget]

There is an issue in commit 44df4e9:
checkout: make orphaned commit checks configurable

• Commit not signed off

[gitgitgadget]

There is an issue in commit 00d3fc0:
scalar: disable orphaned commit checks by default

• Commit not signed off

[gitgitgadget]

There is an issue in commit 1e3e99b:
checkout: make orphaned commit checks configurable

• Commit not signed off

[gitgitgadget]

There is an issue in commit 8d9f70d:
scalar: disable orphaned commit checks by default

• Commit not signed off

[dscho]

/allow

[gitgitgadget]

User wyatt-holden is now allowed to use GitGitGadget.

WARNING: wyatt-holden has no public email address set on GitHub; GitGitGadget needs an email address to Cc: you on your contribution, so that you receive any feedback on the Git mailing list. Go to https://github.com/settings/profile to make your preferred email public to let GitGitGadget know which email address to use.

[derrickstolee]

/allow

[gitgitgadget]

User wyatt-holden already allowed to use GitGitGadget.

[derrickstolee]

I have some recommendations, but here are some top-level thoughts:

1. You use your @microsoft.com email address, but we can't use that to reply to the mailing list. Have you subscribed to the mailing list somewhere? (I use my Gmail account, have a rule to put the mailing list messages in a folder, and use Thunderbird to navigate and reply to those messages in plaintext.)

2. Your cover letter could include an introduction of yourself (and I'll chime in to vouch for you). But be sure that any justification of the change also happens in the commit messages. You'll want to add cc: gitster@pobox.com and cc: stolee@gmail.com to CC the maintainer and me.

3. "In an affected Scalar-managed repository" you could say "In an internal monorepo" as this could impact a non-Scalar repo. I do wonder how much Scalar and the GVFS Protocol could have impacted this situation, but probably the repo didn't have a commit-graph file (yet) which would have sped-up this check. It might be worth spending some time to understand what exactly was slow about it: did the client use the full endpoint and thus was walking from a huge list of refs? Was it missing commits and thus downloading thousands of missing commits one-by-one? (that last one won't really impact upstream Git clients) It can be worth mentioning that this was something in an automated environment, so we didn't care about abandoning the detached HEAD commit. Users might want to know this information, but I suspect that most ignore it and aren't willing to wait for it.

4. You should mention these two commits directly (and in this format):

06b324c1d71 (status: add status.aheadbehind setting, 2019-06-18)
0a53561a622 (status: warn when a/b calculation takes too long, 2019-06-18)

One thing that is different between those two changes and your change is that git status got a --no-ahead-behind argument that is referenced by the warning, but you're only adding a config option. I think that this is the right call, but it might be worth pointing out that difference as something you know about and is open for changing if reviewers think it's important.

[wyatt-holden]

I have some recommendations, but here are some top-level thoughts:

1. You use your @microsoft.com email address, but we can't use that to reply to the mailing list. Have you subscribed to the mailing list somewhere? (I use my Gmail account, have a rule to put the mailing list messages in a folder, and use Thunderbird to navigate and reply to those messages in plaintext.)
2. Your cover letter could include an introduction of yourself (and I'll chime in to vouch for you). But be sure that any justification of the change also happens in the commit messages. You'll want to add cc: gitster@pobox.com and cc: stolee@gmail.com to CC the maintainer and me.
3. "In an affected Scalar-managed repository" you could say "In an internal monorepo" as this could impact a non-Scalar repo. I do wonder how much Scalar and the GVFS Protocol could have impacted this situation, but probably the repo didn't have a commit-graph file (yet) which would have sped-up this check. It might be worth spending some time to understand what exactly was slow about it: did the client use the full endpoint and thus was walking from a huge list of refs? Was it missing commits and thus downloading thousands of missing commits one-by-one? (that last one won't really impact upstream Git clients) It can be worth mentioning that this was something in an automated environment, so we didn't care about abandoning the detached HEAD commit. Users might want to know this information, but I suspect that most ignore it and aren't willing to wait for it.
4. You should mention these two commits directly (and in this format):

06b324c1d71 (status: add status.aheadbehind setting, 2019-06-18)
0a53561a622 (status: warn when a/b calculation takes too long, 2019-06-18)

One thing that is different between those two changes and your change is that git status got a --no-ahead-behind argument that is referenced by the warning, but you're only adding a config option. I think that this is the right call, but it might be worth pointing out that difference as something you know about and is open for changing if reviewers think it's important.

@derrickstolee -
1 - Okay - just got done subscribing on wyattj.holden@gmail.com. I'll CC myself in the cover/PR description as well
2 - Okay, got it - I'll push modified commits soon
3 - After looking through my PR, it looks like i was using the full endpoint - so this could explain a large number of refs being populated locally on the machine? Though, I'm not entirely sure if the commit-graph file existed or not from the dumps/logs i still do have.
But we for sure know that enumerating on loose ref files was happening. we captured dumps at 15 (2-3 seconds after unpack_trees completed), 45, and 75 seconds into the checkout process, and orpahned_commit_warning() is in all three of the dumps main thread - showing filesystem operations are happening on loose ref files (opening, reading and closing them).
4. Cool, I'll mention those in the modified commit messages