Repository navigation
Test failures with lxml 5.4.0 #24
Description
After upgrading to lxml 5.4.0 we are seeing the following tests regressing on nixpkgs.
======================================================================
FAIL: test_host_whitelist_invalid (tests.test_clean.CleanerTest.test_host_whitelist_invalid)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/build/source/tests/test_clean.py", line 343, in test_host_whitelist_invalid
self.assertEqual(expected, cleaner.clean_html(html))
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: '<div></div>' != '<div></div></div>'
- <div></div>
+ <div></div></div>
======================================================================
FAIL: test_host_whitelist_sneaky_userinfo (tests.test_clean.CleanerTest.test_host_whitelist_sneaky_userinfo)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/build/source/tests/test_clean.py", line 350, in test_host_whitelist_sneaky_userinfo
self.assertEqual(expected, cleaner.clean_html(html))
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: '<div></div>' != '<div></div></div>'
- <div></div>
+ <div></div></div>
======================================================================
FAIL: test_host_whitelist_valid (tests.test_clean.CleanerTest.test_host_whitelist_valid)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/build/source/tests/test_clean.py", line 337, in test_host_whitelist_valid
self.assertEqual(expected, cleaner.clean_html(html))
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: '<div><iframe src="https://example.com/page"></iframe></div>' != '<div><iframe src="https://example.com/page"></div></iframe></div>'
- <div><iframe src="https://example.com/page"></iframe></div>
+ <div><iframe src="https://example.com/page"></div></iframe></div>
? ++++++++++++
----------------------------------------------------------------------
Ran 28 tests in 0.019s
FAILED (failures=3, skipped=1)
Activity
frenzymadness commentedon Jun 18, 2025
Thank you for the report. So far, I'm not able to reproduce the issue. When I build the latest version of lxml_html_clean in Fedora rawhide with lxml 5.4.0, it works fine and all tests pass. The same happens to me when I run
tox -e py313with lxml 5.4.0 installed from PyPI.Acconrding to the changelog of 5.4.0, they updated wheels with libxml2 and libxslt and it seems that libxml2 in Fedora is older (2.12.10) than what upstream uses currently (2.13.8). This might be potentially the cause.
vcunat commentedon Jun 18, 2025
In nixpkgs we get this with libxml2 2.14
frenzymadness commentedon Jun 18, 2025
The update to 2.14 is WIP in Fedora: https://src.fedoraproject.org/rpms/libxml2/pull-request/16
frenzymadness commentedon Sep 29, 2025
If I understand this problem correctly, the change happened on lxml/libxml2 level. With the following reproducer:
The latest lxml (6.0.2) with vendored libxml2 2.14.6 prints out:
while the older one (5.3.2) with libxml2 2.12.10 prints out:
Let me check #25 whether the updated tests still make sense.
overlays: drop the doCheck overrides for rapidocr-onnxruntime and lxm…