replaces the hard-coded dialect list in the tokenizer with a supports_hash_comment dialect method

[s5dsn-eqee]

Closes: #2673

Dialects: ClickHouse

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.28%. Comparing base (1aabd12) to head (34a5648).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2674      +/-   ##
==========================================
+ Coverage   81.25%   81.28%   +0.03%     
==========================================
  Files          42       42              
  Lines       33910    33949      +39     
  Branches    33910    33949      +39     
==========================================
+ Hits        27553    27595      +42     
+ Misses       2792     2789       -3     
  Partials     3565     3565              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[LucaCappelletti94]

ClickHouse requires an ASCII space or ! immediately after # to start a line comment, while at this time the tokenizer also accepts SELECT 1 #comment, SELECT 1 ####### comment, SELECT 1 #, and # followed by a tab, all of which the official lexer rejects.

Please do add the red tests and ensure the tokenizer properly tackles these cases.

[LucaCappelletti94]

LGTM!

cc @alamb