Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,11 @@ if(BUILD_TESTING)
loglens_enable_warnings(test_parser_properties)
add_test(NAME parser_properties COMMAND test_parser_properties)

add_executable(test_parser_streams tests/test_parser_streams.cpp)
target_link_libraries(test_parser_streams PRIVATE loglens_lib)
loglens_enable_warnings(test_parser_streams)
add_test(NAME parser_streams COMMAND test_parser_streams)

add_executable(test_timestamp_boundaries tests/test_timestamp_boundaries.cpp)
target_link_libraries(test_timestamp_boundaries PRIVATE loglens_lib)
loglens_enable_warnings(test_timestamp_boundaries)
Expand Down
51 changes: 51 additions & 0 deletions docs/parser-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,57 @@ month length and leap-year validity are checked afterward. Out-of-range days
produce `unknown_timestamp` / `invalid calendar date` warnings rather than
wrapping into valid dates. See [the boundary test report](../FUZZING-OR-BOUNDARY-TEST-REPORT.md).

## Stream completion and read failures (unreleased correction)

`parse_stream` returns a `ParseReport` only after the input reaches ordinary EOF
without a reported read failure. With the default stream exception mask, empty
input, newline-terminated input and a valid final record without a newline all
complete normally. The final record retains its event and original line number.
Malformed or unsupported complete lines retain their existing warning taxonomy;
they are content failures, separate from an underlying stream failure.

The parser checks stream state before acquisition and after the `getline` loop:

- `badbit` is an error, including when `eofbit` is also set.
- `failbit` without `eofbit` is an error. The `fail()` predicate also includes
`badbit`; it does not identify `failbit` alone.
- EOF without `badbit`, including the normal `eofbit | failbit` exhaustion state,
is accepted. A stream already exhausted by its caller returns an empty report;
the parser does not clear or rewind it.

Reported non-EOF failures throw `std::ios_base::failure` rather than returning
partial events or adding a parser warning. Caller-enabled stream exceptions
propagate and the exception mask is unchanged: a buffer exception may be
rethrown when `badbit` exceptions are enabled. Masks including `failbit` or
`eofbit` can also throw on ordinary EOF, as before; callers requiring normal
EOF reports should use the default exception mask. These distinctions follow
the C++20 unformatted-input and string-`getline` rules
([working draft](https://isocpp.org/files/papers/N4860.pdf),
`[istream.unformatted]`, `[string.io]`).

Counterexample: a buffer that throws after two complete valid records sets
`badbit`; previously the loop ended and returned those two events as a successful
report. A mid-record failure could similarly hide the remaining evidence.
`tests/test_parser_streams.cpp` injects faults at exact character positions in
both input modes and checks normal EOF, content warnings, initial state and
exception masks. Its focused executable also accepts a case-name substring.
Mutation controls must reject dropping the completion check, suppressing a read
exception into partial success, and rejecting an ordinary unterminated last line.

`parse_file` retains open-error handling and delegates read validation to this
boundary. The CLI catches parsing exceptions before detection and report writing:
it exits nonzero, emits an error diagnostic, prints no success summary and leaves
existing reports unchanged. The CLI regression uses directory input: on the
tested Linux file stream this opens but fails on read; other runtimes can reject
it at open. Injected failures after a valid prefix are tested through the stream
API rather than an operating-system-specific CLI interception hook.

The parser trusts the stream's EOF/error signals; it cannot detect a buffer that
reports false EOF. This does not authenticate log provenance, guarantee an atomic
file snapshot or durable storage, impose resource limits, or make multi-file report
writes transactional. Successful-input events, coverage counts, detector rules,
finding identities and released report contracts are unchanged.

## Recognized event families

The parser currently recognizes common authentication evidence from:
Expand Down
2 changes: 2 additions & 0 deletions docs/quality-gates.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ The main review principle is:
| Claim under review | Evidence surface | Automated or repeatable gate | Good stopping point |
| --- | --- | --- | --- |
| Supported input formats are explicit | [`parser-contract.md`](./parser-contract.md), [`parser-conformance-matrix.md`](./parser-conformance-matrix.md) | `ctest --test-dir build --output-on-failure` through `test_parser`; fixture anchors in `assets/parser_fixture_matrix_syslog.log` and `assets/parser_fixture_matrix_journalctl_short_full.log` | Reviewer can name the two supported formats and see known/unknown line behavior in fixtures |
| Stream failures cannot return successful partial evidence | [`parser-contract.md`](./parser-contract.md#stream-completion-and-read-failures-unreleased-correction), [`tests/test_parser_streams.cpp`](../tests/test_parser_streams.cpp), [`tests/test_cli.cpp`](../tests/test_cli.cpp) | `parser_streams` injects deterministic read faults and checks EOF/exception policy; `cli` checks input-failure exit, stdout and report preservation | Reviewer can distinguish content warnings, ordinary EOF and transport errors without claiming atomic acquisition or transactional output |
| Parser coverage is visible | [`parser-coverage-notes.md`](./parser-coverage-notes.md), [`tests/fixtures/parser_matrix/noisy_auth_expected.json`](../tests/fixtures/parser_matrix/noisy_auth_expected.json) | `test_parser` compares noisy-auth coverage output to the checked-in expected summary | Reviewer can see parsed lines, skipped blanks, warnings, failure categories, and unknown-pattern buckets |
| Parser handler dispatch remains robust | [`parser-contract.md`](./parser-contract.md), [`tests/test_parser_properties.cpp`](../tests/test_parser_properties.cpp), [`tests/fuzz/README.md`](../tests/fuzz/README.md) | `test_parser_properties` checks all handler-order permutations and generated failure invariants; Ubuntu CI runs a bounded Clang libFuzzer smoke corpus | Reviewer can see that registry order does not change results and malformed or arbitrary bytes remain bounded by parser result invariants |
| Unsupported evidence does not silently become detector evidence | [`parser-contract.md`](./parser-contract.md), [`rule-catalog.md`](./rule-catalog.md), [`case-study-linux-auth-bruteforce.md`](./case-study-linux-auth-bruteforce.md) | `test_parser` covers unknown-pattern warnings; `test_detector` covers signal-boundary behavior | Reviewer can explain why unsupported lines remain warnings instead of findings |
Expand All @@ -33,6 +34,7 @@ Use the smallest command set that answers the review question:
| Multi-config local test run | `ctest --test-dir build -C Debug --output-on-failure` |
| Report-contract snapshot verification | `ctest --test-dir build -C Debug -R report_contracts --output-on-failure` |
| Parser property checks | `ctest --test-dir build -C Debug -R parser_properties --output-on-failure` |
| Stream read/EOF fault controls | `ctest --test-dir build -C Debug -R parser_streams --output-on-failure` |
| Bounded parser fuzz smoke (Clang) | See [`tests/fuzz/README.md`](../tests/fuzz/README.md) |
| Performance envelope reproduction | `pwsh -File scripts/benchmark-performance-envelope.ps1` |
| Fast performance smoke check | `pwsh -File scripts/benchmark-performance-envelope.ps1 -LineCounts 1000 -Runs 1 -WarmupRuns 0 -SkipBuild` |
Expand Down
10 changes: 10 additions & 0 deletions src/parser.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,14 @@ std::optional<Event> AuthLogParser::parse_line(std::string_view line,
}

ParseReport AuthLogParser::parse_stream(std::istream& input) const {
const auto check_read_state = [&input] {
// EOF may set failbit after the last record; badbit is never clean EOF.
if (input.bad() || (input.fail() && !input.eof())) {
throw std::ios_base::failure("unable to read input log");
}
};
check_read_state();

ParseReport result;
result.metadata.input_mode = config_.input_mode;
result.metadata.timezone_present = config_.input_mode == InputMode::JournalctlShortFull;
Expand Down Expand Up @@ -140,6 +148,8 @@ ParseReport AuthLogParser::parse_stream(std::istream& input) const {
++category_count.second;
}

check_read_state();

if (result.quality.total_lines != 0) {
result.quality.parse_success_rate =
static_cast<double>(result.quality.parsed_lines) / static_cast<double>(result.quality.total_lines);
Expand Down
30 changes: 30 additions & 0 deletions tests/test_cli.cpp
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
#include <array>
#include <cstdlib>
#include <filesystem>
#include <fstream>
Expand Down Expand Up @@ -250,6 +251,35 @@ int main(int argc, char* argv[]) {
expect(!std::filesystem::exists(journalctl_out / "warnings.csv"),
"did not expect journalctl warnings.csv without explicit csv flag");

const auto directory_input = output_dir / "directory_input";
std::filesystem::create_directories(directory_input);
const std::array report_names{"report.md", "report.json", "findings.csv", "warnings.csv"};
for (const bool existing : {false, true}) {
const auto failed_output = output_dir / (existing ? "read_failure_existing" : "read_failure_new");
if (existing) {
std::filesystem::create_directories(failed_output);
for (const auto* name : report_names) {
std::filesystem::copy_file(csv_out / name, failed_output / name);
}
}
const auto failure_stdout = output_dir / "read_failure_stdout.txt";
const auto failure_stderr = output_dir / "read_failure_stderr.txt";
const int failure_exit = std::system(build_command(
quote_argument(loglens_exe) + " --mode syslog --year 2026 --csv "
+ quote_argument(directory_input) + " " + quote_argument(failed_output),
&failure_stdout, &failure_stderr).c_str());
expect(failure_exit != 0, "expected directory input read/open failure");
expect(read_file(failure_stdout).empty(), "input failure printed a success summary");
expect(read_file(failure_stderr).find("LogLens failed:") != std::string::npos,
"expected input failure diagnostic");
for (const auto* name : report_names) {
expect(existing ? read_file(failed_output / name) == read_file(csv_out / name)
: !std::filesystem::exists(failed_output / name),
"input failure created or changed a report artifact");
}
expect(existing || !std::filesystem::exists(failed_output), "input failure created output directory");
}

const auto missing_year_out = output_dir / "missing_year";
std::filesystem::create_directories(missing_year_out);
const int missing_year_exit = std::system(build_command(
Expand Down
2 changes: 1 addition & 1 deletion tests/test_parser.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1369,7 +1369,7 @@ void test_mixed_auth_first_pam_unix_session_closed_is_unsupported_telemetry() {
const auto full = parser.parse_file(asset_path("mixed_auth_corpus.log"));
const auto warning_it = std::find_if(
full.warnings.begin(), full.warnings.end(),
[kFixtureLineNumber](const loglens::ParseWarning& warning) {
[](const loglens::ParseWarning& warning) {
return warning.line_number == kFixtureLineNumber;
});
expect(warning_it != full.warnings.end(),
Expand Down
Loading
Loading