Skip to content

Language: Maintain the translations of language modules in PO files - #12261

Open
c-knof wants to merge 103 commits into
ILIAS-eLearning:trunkfrom
kroepelin-projekte:feature/trunk/ck/po-files-migration
Open

c-knof wants to merge 103 commits into
ILIAS-eLearning:trunkfrom
kroepelin-projekte:feature/trunk/ck/po-files-migration

Conversation

@c-knof

@c-knof c-knof commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Implements the accepted feature request "PO files for improving language handling"
(https://docu.ilias.de/go/wiki/wpage_8951_1357), including both requested changes (native gettext, identifier enums).

Depends on #12090 (Language on the Component Revision)

What it does

  • A component can ship a language module as lang/<module>.pot plus <module>_<lang>.po instead of lines in
    lang/ilias_<lang>.lang. Setup (and composer dump-autoload) builds them into .mo catalogs below artifacts/language/,
    ilLanguage serves them through native gettext.
  • Local changes, remarks and plural forms of migrated modules live in an overlay below the client data directory;
    lng_data/lng_modules stay a rollback copy. Local changes of earlier versions are taken over once on the first update.
  • Plural forms: txt($id, $n). Generated LanguageIdentifier enums per component: txt(PollLanguageIdentifier::POLL_ADD).
  • The language administration (edit, import, export, maintenance) works on both kinds of modules.
  • Pilot modules in this PR: tos, poll, cmps, lng. All other modules follow in the wave PRs below.

Breaking change

Language::txt(string|LanguageIdentifier $a_topic, ?int $n = null). The fallback module parameter and ntxt() are removed, all 13 core callers are adapted (one commit per component: Exercise, Survey, WebServices, OrgUnit, Forum, Repository, WebDAV, UI/Refinery test doubles).

Changes outside the Language component

TermsOfService, Poll, Component (module files and contribution), the callers listed above, composer.json/composer.lock (gettext/gettext), docs/development/language.md, lang/ilias_*.lang (pilot modules removed, new lng_* texts).

Requirements and operation

  • ext-gettext (CLI and web server) and the C.utf8 locale.
  • composer dump-autoload or setup build after every change of a shipped .po; Setup runs as the web server user.
  • Unchanged builds only print a count of build warnings, details are in artifacts/language/<build>/manifest.json.
  • Rollback per module: revert its migration commit, composer dump-autoload, setup update (see README, "Rollback").

Behaviour changes

  • A missing value of an identifier that exists in several modules shows -identifier- instead of the value of another
    loaded module.
  • Disallowed markup in local changes is refused instead of stripped.
  • Imports and customizing files refuse identifiers that exist in another module (rule 2 of docs/development/language.md).
  • "Merge local changes" writes into the shipped .po files of the components.

c-knof and others added 30 commits October 23, 2026 16:49
The seven Activities now get `ILIAS\AccessControl\PublicInterface\RBAC` via `$use` instead of
reaching into `$GLOBALS['DIC']->rbac()`, and call `system()` only in `isAllowedToPerform()`.
The Setup agent uses `InstallLanguage::forSetup()`/`UpdateLanguage::forSetup()`, since Setup
checks no permissions and needs no RBAC.
`InstalledLanguageDatabaseRepository`, `LanguageInstallationManager`, `AddLanguageEntry` and the
`ilSetupLanguage` of the web Activities now get `ILIAS\Database\Connection` via `$pull` instead of
resolving `$GLOBALS['ilDB']`. `ilSetupLanguage` takes an optional connection; Setup keeps handing
in its database via `setDbHandler()`, also for the metrics, and the file-only checks need no
database.
The Setup CLI asks `ilLanguagesInstalledAndUpdatedObjective` for its label before `achieve()`,
so `getLabel()` only worked through a leftover `$GLOBALS['ilDB']`. `isApplicable()`, which the
CLI calls first, now reads the languages from the Setup database and `getLabel()` lists them
without database access.
The repository and the installation manager in the component graph now get the root from
`ILIAS\Environment\Configuration\Installation\Directories` at resolve time instead of a path
computed in `init()`. Callers outside the graph keep passing the path themselves.
… container

`AllModernComponents` bound `$DIC[ILIAS\Language\Language::class]` to the `$use`d language,
whose legacy adapter itself delegates to `$DIC->language()`. Nothing reads the key, so the
constructor parameter and the binding are removed.
No component pulls `InstalledLanguageRepository` or `LanguageInstallationManager`, so they stay
internal services of the component; `ilSetupLanguage` remains the entry point for callers
outside it.
The Activities took the ref_id of the language folder as `int|\Closure` defaulting to 0. They
now require `ILIAS\Language\LanguageFolder`, whose `LanguageFolderLegacyProxy` looks the ref_id up
only when a permission is checked.
Keep only what the code does not say itself (the why, a constraint or a
pitfall), mostly in one line, and drop the history of how it got there.
…vities

A denied permission is now a `DomainException` with code 403 instead of a localized string, and
`Failure::of()` masks every error outside the 4xx range as a generic 500, so callers other than the
GUI no longer see internal messages. `InvalidInputException` carries 400,
`AmbiguousLanguageTitleException` 409; `RendersActivityErrors` decides by code, which replaces the
`SafeToDisplayActivityError` marker.
…the File Activities

`LanguageActivity::isAllowedToPerform()` now rejects unknown and anonymous users before asking RBAC,
as `FileUpload\Activities\AccessCheck` does for the File Activities. `maybePerformAs()` hands the
trusted user id to `perform()` directly, which replaces the `additionalPerformParameters()` hook.
…ections like the File Activities

The descriptions are built with `ILIAS\Data\Factory` instead of an own markdown helper, so the
Activities no longer need the Refinery. The input of every Activity is a titled `section` instead of
an untitled `group`, as in the File Activities.
…Provider` like the File Activities

The language objects, language entries, the module cache, the user lookup and the settings are
reached through one `RepositoryProvider` instead of optional closures with legacy defaults in every
Activity. `UninstallLanguage`, `RemoveLocalLanguageChanges`, `AddLanguageEntry` and the two settings
Activities take it as their only dependency besides language, RBAC and language folder; the tests
mock it.
A `PDOException` carries its SQLSTATE as string code, and `'42S02'` compares as a 4xx code, so
`Failure::of()` and `RendersActivityErrors` showed the database message as input error. Both now
require an integer code. The test doubles no longer call the removed `markdown()` helper.
Keeps only what the code does not say itself, as in the File component: the reasons in
`Language.php` shrink to the two real traps (no `final` Activities, the order of the contributions),
and the tests lose their history notes.
`LanguageActivity` no longer takes `Language`: only `AddLanguageEntry` needs it, for the labels of
its translation fields. The other six Activities, their `forSetup()` variants and the wiring drop
the parameter.
…st CSRF

The table actions were read from the query by `executeCommand()` without a
command, so ilCtrl never checked a token. `ilObjLanguageFolderGUI` now
implements `ilCtrlSecurityInterface`, runs the table actions only through the
unsafe command `executeTableAction` and dispatches an exact allowlist of
commands, since ilCtrl compares commands case-sensitively but PHP calls
methods case-insensitively. Takes effect after the ctrl security artifact is
rebuilt (`composer dump-autoload` or `setup build`).
…ObjLanguageFolderGUI`

Listing and downloading the deprecated entries failed with an error, because
`ilObjLanguageFolderGUI` still included `ilLangDeprecated` from the former
`Services/Language` directory.
…uageFolderGUI`

`confirmRefreshSelectedObject()`, `confirmUninstallObject()`,
`confirmUninstallChangesObject()` and the table action `edit` could no longer
be reached since the table actions confirm in modals, and their confirmations
failed for lack of language ids.
Named arguments are not used in the core (review criteria).
PHPUnit 14 no longer allows `with()` without `expects()`, so the mocks in
`SetLanguageTranslationEnabledTest`, `AddLanguageEntryTest` and
`LanguageInstallationManagerTest` now expect their calls once.
…es as `@internal`

`TranslationCache`, `InstallationRoot`, `LanguageSettings` and their implementations
are only wired in `$internal` and used inside the Language component, like the
language file directories that already carry the marker.
Reverts the PO pilot of the File component from the base branch: the
`file` entries move back into lang/ilias_*.lang and File no longer
contributes a LanguageFileDirectory. The PO migration is piloted with
TermsOfService and Poll instead.
c-knof added 27 commits October 23, 2026 16:50
A module migrated to PO files was still read from
lng_modules/lng_data whenever it was not found in the build: without a
build, for a language the build lacks, for a language not installed, or
in Setup and CLI, where the module was only recognised through the
component graph in $DIC. lng_data/lng_modules are written as rollback
fallback only and may be stale.

A module now counts as shipped if the build lists it or a component
ships its lang/<module>.pot (found without $DIC, cached). Such a module
is served from the build only and otherwise has no texts (-identifier-);
a missing build or an unbuilt shipped .po is reported. loadLanguageModule(),
_lookupEntry(), txtlng(), exists() and txt() with a LanguageIdentifier
read the database only for the other modules (plugins).

Tests that asserted the database fallback for shipped modules now
expect no texts.
Setup served the shipped modules from whatever build was there, or from
none if "setup build" had not run. ShippedLanguageFilesCompiledObjective
is now a precondition of installing and updating the languages (same
hash as the build objective, so it runs once), and forgets the read
state afterwards so the following objectives serve the new build.
Instances with the default directories only (plugin objectives) get no
build: it would serve no module at all.
All modules of the global language files are migrated, the global
lang/ilias_<lang>.lang files only hold their header, and the database
serves plugin modules only. Native gettext and C.utf8 are mandatory, the
Setup builds before installing languages, and a language can no longer be
rolled back by removing its .po.
Cover the setup and CLI cases: a shipped module is served from the build
without CLIENT_DATA_DIR and without the language file directories of the
container, with the overlay once the client data directory is known;
without a build, for a language not in the build or not installed it has
no identifiers and the database is not queried; a plugin module is still
read from the database. Test finding the shipped templates, the build as
precondition of installing languages and the reset of the served build
after compiling. Rename the tests that still named the database fallback.
The administration still read a shipped module from lng_modules/lng_data
if its overlay could not be read, without client data directory or
without the language file directories in $DIC, and saving then wrote
that database state into the overlay.

For a shipped module the values shown and the base of a save are now its
shipped .po plus overlay, with an unreadable overlay or without client
data directory the shipped .po alone (reported). If the shipped .po
cannot be read either, nothing is shown and the save of that module is
aborted: it is reported and listed as not written. Plugin modules are
read from the database as before.

Tests that asserted the database fallback for shipped modules now expect
the shipped values or the aborted save; those for modules read from the
database use a plugin module instead of "common".
Before CLIENT_DATA_DIR is defined the client data directory was always
composed with "[clients] default" of ilias.ini.php, also when CLIENT_ID
was already defined (e.g. by the plugin Setup objectives or a CLI run
for another client). The defined client id wins now, like it does in
ilInitialisation.
…uild

ilLanguage kept the client data directory resolved from ilias.ini.php for
the whole process, also when none was found, and cached per module
whether it is shipped. MigratedTranslations::reset() (e.g. after the
build in the Setup) did not clear either, so objectives later in the same
process (install creates the client data directory) kept serving without
overlay.

The client data directory is now resolved by MigratedTranslations, a
found one is kept until reset(), a missing one is looked for again. The
per-module cache in ilLanguage is replaced by an index of the language
file directories, rebuilt when the directory manager in $DIC changes.
Without the component graph a module was recognised as shipped by a
glob over components/*/*/lang/*.pot, which also ran in every web
request for each plugin module, and missed a contribution whose template
is named or placed differently.

The build now records every contributed module in current.json
("shipped"); with a build that lists them only these count, the glob is
the fallback without one (or with a current.json written before). The
build warns about a contribution without a lang/<prefix>.pot, which
Setup and CLI would not recognise without a build.
Cover the editable state of a shipped module (shipped .po with or without
overlay, unreadable files), that the administration shows, saves,
deletes and adds entries of shipped modules without lng_data, the
contributed modules recorded in current.json with the build warning for
a missing template, the client data directory resolved by CLIENT_ID and
forgotten on reset, and the web path with a filled module cache and the
identifier path without a build. Rename the tests that still named the
database fallback.
…languages

Every module is shipped in PO files now, so lang/ilias_<lang>.lang holds
only the header up to `<!-- language file start -->`. The check of all
languages in the administration took the empty content behind the
header for a missing header and reported every file as invalid ("wrong
header"). Only a missing start marker is a wrong header.
If the content of a shipped module could not be read, saving in the
administration and "add new variable" still wrote the lng_data rows with
local_change before giving up on lng_modules and the overlay; a later
setup update took them over into the overlay unnoticed. And with an
unreadable overlay the save was based on the shipped values alone, which
dropped the overlay's other local changes.

The content is now checked per module before anything is written: a
module whose shipped .po or overlay cannot be read is skipped entirely
and reported as not written. Only the display still falls back to the
shipped values for an unreadable overlay.
An invalid `shipped` entry in current.json no longer invalidates the whole
pointer: only the list is dropped, and shipped modules are recognised by
their templates instead. A contribution with an invalid file name pattern
is listed as well.
…uild is reset

Resolving the client data directory reads ilias.ini.php; while the directory
does not exist yet (setup install), do not read it again for every lookup
but only after MigratedTranslations::reset(). Keep the memo of shipped
.po files without a build in MigratedTranslations as well, so reset()
clears it too.
Setup read the local changes of a shipped module from lng_data on every
update and let them win over the overlay, so stale or partially written
rows overrode the overlay. lng_data is only the rollback copy.

The local changes in lng_data are now taken over into the overlay once,
while the module has no overlay for the language (upgrade from ILIAS
10/11). Afterwards, recorded per language in
<client data dir>/lang/taken-over.<lang>.json, and whenever an overlay
exists, the overlay is the only source of local changes and remarks; an
overlay that cannot be read is still rebuilt from lng_data, the only
copy left. Uninstalling a language forgets the record. Plugin modules
are unchanged, the dual-write stays.

Tests that let a lng_data local change win over the overlay now expect
the overlay's value; installing without local changes writes the record.
…nges once

Cover that the administration and AddLanguageEntry write nothing of a
module whose shipped .po or overlay cannot be read (and keep the local
changes of other keys), that setup update takes over the local changes in
lng_data only for a module without overlay and records it, that it leaves
a module with an unreadable overlay unchanged and reports it, and the
remaining cache and pointer details. Rename the tests that still named the
database as source of local changes.
…ers in Setup

An overlay or a `taken-over.<lang>.json` entry of a module that is no longer
shipped (e.g. a reverted migration) would win over the changes made in
`lng_data` once the module is migrated again, so `ilLanguagesInstalledAndUpdatedObjective`
now reports such modules, as well as local changes whose take-over could not be
recorded. Nothing is removed. The README explains the rollback steps.
…nguage file is read-only

Merging the local changes into the shipped files ran only if
`lang/ilias_<lang>.lang` was writable, although `ShippedPoMerger` checks each
`.po` itself. `ilObjLanguageExtGUI` now always merges into the `.po` files,
writes the `.lang` file only if it is writable and reports both results.
Close the array shape of `ShippedPoMerger::merge()`, declare the exceptions of
`MigratedLanguageFileSync::overlayIfShipped()` and `markMergedIntoShipped()`, add
missing value types, drop `array_values()` calls on lists and let
`GrindingInputAccess::assertInternal()` assert the input type.
…rror

Installing or updating a language with invalid language files threw an
`InvalidInputException` (400) whose English text the administration showed
untranslated. `InstallLanguage` and `UpdateLanguage` now throw
`InvalidLanguageFilesException` (409), and `RendersActivityErrors` shows the
new text `lng_invalid_language_files` with the language keys.
`scan_shipped_cross_module_duplicates.php` only read `lang/ilias_*.lang`, which
no longer holds any module once all modules are maintained in PO files. The build
reports collisions between modules itself.
Every `setup update` and `composer dump-autoload` repeated all warnings of the
language build, about 340 lines. `ShippedLanguageFilesCompiledObjective` now
prints them in detail only when the build is written, with collisions grouped by
modules, and otherwise a single line counting them with the path of the manifest.
…tGUI`

ilCtrl compares the unsafe commands case-sensitively, but PHP calls methods
case-insensitively, so `cmd=maintainexecute` ran the maintenance without a
token. `ilObjLanguageExtGUI` now only calls the commands it uses and shows the
list for any other command.
Introduce the read-only `TranslationRepository` with `TranslationGettextRepository`
as its implementation, which takes over the read path of `MigratedTranslations`.
`ilLanguage` accepts a repository as optional second argument, so unit tests can
replace the build and gettext with an in-memory repository.
…build

`ShippedTranslationsBuild::catalogCompatibility()` left out `TranslationCatalog`,
`TranslationEntry` and `PluralForms`, so a `.po` that does not compile could be
served from the catalog of a build compiled by other code. The compatibility and
the build id now hash the same list `CATALOG_CLASSES`.
…module

Importing a language file could add an identifier that already existed in
another module, whose value then replaced the one of that module. Like a new
entry, `ilObjLanguageExt::importLanguageFile()` now refuses such entries: an
uploaded file is rejected as a whole, the customizing file skips them, and both
name the entries with the new text `lng_identifier_not_unique`.
…stomizing file

`InstallLanguage` in the mode `install_local` reported invalid shipped language
files only as an aborted action. It now checks the languages like a plain install
and throws `InvalidLanguageFilesException`, so the administration names them.
…s in another module

Installing or updating a language applied the lines of
`lang/customizing/ilias_<lang>.lang.local` even if their identifier belonged to
another module. `LanguageInstallationManager` now skips such lines like lines
with disallowed markup, and Setup and the administration name them.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement php Pull requests that update Php code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants