[wtn]

After a max_field_size error, the field-scanning parser (parse_quotable_robust) skips only to the end of the physical line. If a later quoted field in the record spans lines, the rest of the record comes back as the next row:

require "csv"

csv = CSV.new(%Q{abcde,"x\ny"\nc,d\n}, max_field_size: 4, liberal_parsing: true)
csv.shift rescue nil
csv.line   # => "abcde,\"x\n"
csv.shift  # => ["y\""]

Without liberal_parsing, the field-scanning parser is used after a row with a quoted field that spans lines. There, the rest of the record raises Illegal quoting on the next shift.

Fix

parse_quotable_robust passes a block to validate_field_size, which calls it in place of ignore_broken_line. The block calls the new ignore_broken_record, which reads the remaining fields of the record with parse_column_value and then the row end. CSV#line is then the whole broken record, and the line number goes up by one. The line-splitting parser (parse_quotable_loose) and parse_no_quote are unchanged.

If the rest of the record is malformed, ignore_broken_record drops the rest of the physical line, and the error is still the field size error. An unclosed quote after the long field now reads to the end of the input, as it does without max_field_size. On main, abcde,"x\nc,d\n with liberal_parsing returned ["c", "d"] after the error.

[kou]

How about doing this in all parse modes like #382?

[wtn]

Closing in favor of #382.

Thank you.