Skip to content

JSONML.toString() and XML.toString() omit escaping for literal newlines (\n), causing whitespace loss in downstream XML parsers (Regression from #1070) #1092

Open

Description

@S-wald

Hello,

When a JSONObject containing a string with newline characters (\n) is converted to XML using JSONML.toString() (or XML.toString()), the newlines are output as literal newline characters instead of being XML-escaped (e.g., to 
).
Because literal newlines in attribute values or specific text nodes are subject to W3C Newline Normalization by downstream XML parsers, the structural formatting and whitespace are completely lost or altered when the generated XML is parsed by another system.
This behavior is a side effect of the fix introduced in PR #1070 (which solved #1059). While PR #1070 correctly allowed whitespace numeric character references (
) to be accepted during parsing (deserialization), it altered XML.mustEscape(int cp) to return false for 0xA, 0xD, and 0x9. As a result, the serialization path (XML.toString / JSONML.toString) now skips escaping for literal newlines, causing a regression in how whitespaces are preserved during export.

Reproducible Example / Code Snippet

import org.json.JSONObject;
import org.json.JSONML;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertTrue;

public class XMLNewlineEscapingTest {

    @Test
    public void testJSONMLToStringPreservesNewlines() {
        String jsonString = "{\"tagName\":\"MyTag\", \"value\":\"Lettuce\\nTomato\\nMustard\"}";
        JSONObject jo = new JSONObject(jsonString);

        String xmlOutput = JSONML.toString(jo);
        System.out.println("Generated XML:\n" + xmlOutput);

        // Assert
        // Expected: Newlines should be escaped as 
 to survive XML normalization
        // Actual: It outputs raw, literal newlines <MyTag value="Lettuce\nTomato..."/>
        assertTrue(xmlOutput.contains("&#10;"), "XML should escape literal newlines to preserve them.");
    }
}

Expected Behavior

When serializing to XML, literal control characters like \n (0xA), \r (0xD), and \t (0x9) within string values should be escaped into their corresponding numeric character entities (&#10;, &#13;, &#9;) to prevent downstream XML parsers from stripping or normalizing them.

Actual Behavior

The generated XML string contains raw literal line breaks, which are collapsed or lost according to standard XML whitespace processing rules when loaded by any standard XML parser.

Suggested Root Cause & Fix Hint

The serialization path (XML.escape) heavily relies on XML.mustEscape(int cp). Currently, XML.mustEscape combines both "is this character legally allowed in an XML document?" (W3C range) and "does this character need to be escaped in text/attributes?".
The library needs to differentiate between the deserialization path (where &#10; is valid and shouldn't throw an exception) and the serialization path (where a literal 0xA needs to be actively escaped to ensure data integrity).

Best regards,
Sebastian

Activity

  1. stleary commented on Oct 14, 2026

    @stleary
    Owner

    @S-wald Thanks for raising this issue. Since it requests a behavior change, can you clarify whether this is causing a problem in someone's code, or if it is to bring the code up to the spec?
    FYI For context, please see the Readme section on Requirements for code changes

  2. S-wald commented on Oct 20, 2026

    @S-wald
    Author

    We only observed this problem in our code after we upgraded to the release containing #1070.

    So I think both is true, it's causing a problem in existing code and it's bringing the code up to the spec.

  3. stleary commented on Oct 20, 2026

    @stleary
    Owner

    @S-wald Thank you for explaining, OK to proceed with a fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions