Skip to content

Speed up listing of large directories - #166

Open
czachor wants to merge 5 commits into
artgris:masterfrom
czachor:perf/listing
Open

czachor wants to merge 5 commits into
artgris:masterfrom
czachor:perf/listing

Conversation

@czachor

@czachor czachor commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Refs #75

Context

Opening the manager on a directory with thousands of files is slow. Part of the cost is on the server: indexAction does work for every request that is either never displayed or repeated. This PR removes that work without changing what the user sees.

The larger part of the cost is in the browser (full-size originals used as 22 px previews, and a cache-buster that forces them to be downloaded on every load). The cache-buster part is addressed in #167; full-size previews will follow in a separate PR.

Changes

1. Directory tree is built only when it is displayed

Cause: indexAction always called retrieveSubDirectories(), which walks the whole directory tree recursively (and counts files in every folder when show_file_count is on). It did this even with tree: false, where the tree is never rendered.

Change: the tree is built only when $fileManager->getTree() is true; otherwise treeData is null. The JavaScript already initializes the tree only when tree === true, so nothing else changes.

2. Image dimensions are read once per file

Cause: File::getDimension() called getimagesize() on every call, and every call opens and parses the file. With orderby=dimension, usort() calls it twice per comparison, so each image was read O(n log n) times.

Change: the result is memoized in the File object. While touching this method:

  • the image check uses an exact extension list, File::IMAGE_EXTENSIONS, instead of an unanchored regex;
  • getimagesize() runs through Symfony's ErrorHandler::call() instead of the @ operator. A corrupt or unreadable image still yields false, as before, but unrelated errors are no longer silenced.

3. File previews are built lazily

Cause: the File constructor called FileTypeService::preview(), which renders a Twig template for images, for every entry in the directory.

Change: the preview is built on the first getPreview() call and reused afterwards. isImage() goes through getPreview(). Today every listed file is rendered, so the gain is small, but it means only entries that are actually displayed pay for the render (useful for pagination later).

4. JSON badge does not rescan the directory

Cause: the JSON response (used to refresh the list after an upload) computed badge with $finderFiles->count(). Finder::count() iterates the Finder again, which scans the directory a second time.

Change: count($fileArray) is used. The list is built from the same Finder, so the number is the same.

Refactoring

  • Image extensions are defined once, in File::IMAGE_EXTENSIONS, and reused by File, FileTypeService and FileManager::getRegex().
  • FileTypeService::fileIcon() compares extensions exactly instead of with unanchored regexes. Before, any extension ending with a known one matched too (e.g. xpdf got the PDF icon). tgz is now listed explicitly, because it used to match only through the gz suffix. The extension is read with SplFileInfo::getExtension() (always a string) instead of pathinfo() (array|string).
  • Small controller cleanups: str_starts_with() instead of 0 !== mb_strpos() in the rename and delete path checks (same result, also when realpath() returns false), a strict === comparison, a static closure for the dimension sort and a void return type on dispatch().

Backward compatibility

No configuration, template or public API changes. The only visible difference: with tree: false, the JSON response returns treeData: null instead of an unused tree.

Tests

  • vendor/bin/simple-phpunit: 17 tests, 41 assertions, OK (Symfony 7.4).
  • New controller tests: JSON response with and without the tree (treeData, badge) and orderby=dimension.
  • New Tests/Helpers/FileTest.php: dimensions of a valid, a corrupt and a non-image file, and the preview being built only when needed and only once.

indexAction always built the full recursive directory tree, even with
tree: false, where it is never displayed. The tree walk is now done
only when the tree is enabled; treeData is null otherwise. The
JavaScript already initializes the tree only when tree === true.
Sorting by dimension calls getDimension() on every usort comparison,
so getimagesize() read each image O(n log n) times. The result is now
computed once per file.

The image check uses an exact extension list (File::IMAGE_EXTENSIONS)
instead of an unanchored regex, and getimagesize() runs through
ErrorHandler::call() instead of the @ operator: a corrupt or unreadable
image still yields false, without silencing unrelated errors.
FileManager::getRegex() builds the "image" type pattern from the same
list.
fileIcon() matched extensions with unanchored regexes, so any
extension ending with a known one matched too (e.g. "xpdf" got the PDF
icon). Extensions are now lowercased once and compared against exact
lists; images reuse File::IMAGE_EXTENSIONS. "tgz" is listed explicitly,
as it was previously matched only through the "gz" suffix.

The extension is read with SplFileInfo::getExtension(), which always
returns a string, instead of pathinfo(), whose return type is
array|string. IMAGE_SIZE gets explicit public visibility.
The File constructor rendered the preview Twig template for every
listed entry, whether or not the template displayed it. The preview is
now built on the first getPreview() call and reused afterwards;
isImage() goes through getPreview().
The JSON response computed the badge with $finderFiles->count(), which
iterates the Finder again and rescans the directory. The file list is
already built from the same Finder, so its size is used instead.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant