Skip to content

CLDR-18722 Explicit numberSystem attribute #4974

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

CLDR-18722 Explicit numberSystem attribute #4974

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
09366c0
CLDR-18722 Explicit numberSystem attribute
btangmu Aug 18, 2025
8dbbcf6
CLDR-18722 Correction for el.xml
btangmu Aug 18, 2025
112a943
CLDR-18722 Un-deprecate decimal/group numberSystem and
btangmu Aug 18, 2025
cb070b2
CLDR-18722 Un-deprecate decimal/group numberSystem in ldml.dtd
btangmu Aug 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions common/dtd/ldml.dtd
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1441,6 +1441,8 @@ CLDR data files are interpreted according to the LDML specification (http://unic
<!--@DEPRECATED:true, false-->
<!ATTLIST pattern references CDATA #IMPLIED >
<!--@METADATA-->
<!ATTLIST pattern numberSystem CDATA #IMPLIED >
<!--@MATCH:bcp47/nu-->

<!ELEMENT datetimeSkeleton ( #PCDATA ) >
<!ATTLIST datetimeSkeleton numbers CDATA #IMPLIED >
Expand Down Expand Up @@ -2062,7 +2064,7 @@ CLDR data files are interpreted according to the LDML specification (http://unic
<!ATTLIST decimal references CDATA #IMPLIED >
<!--@METADATA-->
<!ATTLIST decimal numberSystem CDATA #IMPLIED >
<!--@DEPRECATED-->
<!--@MATCH:bcp47/nu-->

<!ELEMENT group ( #PCDATA ) >
<!ATTLIST group alt NMTOKENS #IMPLIED >
Expand All @@ -2073,7 +2075,7 @@ CLDR data files are interpreted according to the LDML specification (http://unic
<!ATTLIST group references CDATA #IMPLIED >
<!--@METADATA-->
<!ATTLIST group numberSystem CDATA #IMPLIED >
<!--@DEPRECATED-->
<!--@MATCH:bcp47/nu-->

<!ELEMENT list ( #PCDATA ) >
<!ATTLIST list alt NMTOKENS #IMPLIED >
Expand Down
6 changes: 3 additions & 3 deletions common/main/ca.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6509,13 +6509,13 @@ Warnings: All cp values have U+FE0F characters removed. See /annotationsDerived/
<symbol draft="contributed">↑↑↑</symbol>
</currency>
<currency type="ESP">
<pattern>¤ #,##0.00</pattern>
<pattern numberSystem="latn">¤ #,##0.00</pattern>
<displayName>pesseta espanyola</displayName>
<displayName count="one">pesseta espanyola</displayName>
<displayName count="other">pessetes espanyoles</displayName>
<symbol>₧</symbol>
<decimal>,</decimal>
<group>.</group>
<decimal numberSystem="latn">,</decimal>
<group numberSystem="latn">.</group>
</currency>
<currency type="ETB">
<displayName>birr etíop</displayName>
Expand Down
6 changes: 3 additions & 3 deletions common/main/el.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6581,13 +6581,13 @@ Warnings: All cp values have U+FE0F characters removed. See /annotationsDerived/
<displayName count="other">εκουέλε Ισημερινής Γουινέας</displayName>
</currency>
<currency type="GRD">
<pattern>#,##0.00 ¤</pattern>
<pattern numberSystem="latn">#,##0.00 ¤</pattern>
<displayName>Δραχμή Ελλάδας</displayName>
<displayName count="one">δραχμή Ελλάδας</displayName>
<displayName count="other">δραχμές Ελλάδας</displayName>
<symbol>Δρχ</symbol>
<decimal>,</decimal>
<group>.</group>
<decimal numberSystem="latn">,</decimal>
<group numberSystem="latn">.</group>
</currency>
<currency type="GTQ">
<displayName>Κουετσάλ Γουατεμάλας</displayName>
Expand Down
2 changes: 1 addition & 1 deletion common/main/en_150.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ CLDR data files are interpreted according to the LDML specification (http://unic
</currencyFormats>
<currencies>
<currency type="EUR">
<pattern>¤#,##0.00</pattern>
<pattern numberSystem="latn">¤#,##0.00</pattern>
</currency>
</currencies>
</numbers>
Expand Down
2 changes: 1 addition & 1 deletion common/main/en_AT.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ CLDR data files are interpreted according to the LDML specification (http://unic
</currencyFormats>
<currencies>
<currency type="EUR">
<pattern>¤ #,##0.00</pattern>
<pattern numberSystem="latn">¤ #,##0.00</pattern>
</currency>
</currencies>
</numbers>
Expand Down
6 changes: 3 additions & 3 deletions common/main/eu.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13967,14 +13967,14 @@ Warnings: All cp values have U+FE0F characters removed. See /annotationsDerived/
<symbol draft="contributed">↑↑↑</symbol>
</currency>
<currency type="ESP">
<pattern>¤ #,##0.00</pattern>
<pattern numberSystem="latn">¤ #,##0.00</pattern>
<displayName>pezeta espainiarra</displayName>
<displayName count="one">↑↑↑</displayName>
<displayName count="other">pezeta espainiar</displayName>
<symbol>₧</symbol>
<symbol alt="narrow" draft="contributed">↑↑↑</symbol>
<decimal>,</decimal>
<group>.</group>
<decimal numberSystem="latn">,</decimal>
<group numberSystem="latn">.</group>
</currency>
<currency type="ETB">
<displayName>birr etiopiarra</displayName>
Expand Down
6 changes: 3 additions & 3 deletions common/main/gl.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5385,13 +5385,13 @@ Warnings: All cp values have U+FE0F characters removed. See /annotationsDerived/
<displayName draft="contributed">Peseta española (conta convertible)</displayName>
</currency>
<currency type="ESP">
<pattern>#,##0.00 ¤</pattern>
<pattern numberSystem="latn">#,##0.00 ¤</pattern>
<displayName draft="contributed">Peseta española</displayName>
<displayName count="one" draft="contributed">peseta</displayName>
<displayName count="other" draft="contributed">pesetas</displayName>
<symbol>₧</symbol>
<decimal>,</decimal>
<group>❰NBTSP❱</group>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is something wrong here; unrelated to this ticket. The [❰❱] characters should never occur in values for our data files. It looks like vetters might be pasting them into values. I'll file a separate ticket about this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@macchiati Did you file that ticket yet?

<decimal numberSystem="latn">,</decimal>
<group numberSystem="latn">❰NBTSP❱</group>
</currency>
<currency type="ETB">
<displayName>birr etíope</displayName>
Expand Down
6 changes: 3 additions & 3 deletions common/main/it.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6095,13 +6095,13 @@ Warnings: All cp values have U+FE0F characters removed. See /annotationsDerived/
<symbol alt="narrow">↑↑↑</symbol>
</currency>
<currency type="ITL">
<pattern>¤ #,##0.00</pattern>
<pattern numberSystem="latn">¤ #,##0.00</pattern>
<displayName>lira italiana</displayName>
<displayName count="one" draft="contributed">↑↑↑</displayName>
<displayName count="other" draft="contributed">lire italiane</displayName>
<symbol draft="contributed">↑↑↑</symbol>
<decimal>,</decimal>
<group>.</group>
<decimal numberSystem="latn">,</decimal>
<group numberSystem="latn">.</group>
</currency>
<currency type="JMD">
<displayName>dollaro giamaicano</displayName>
Expand Down
2 changes: 1 addition & 1 deletion common/main/tr.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7655,7 +7655,7 @@ Warnings: All cp values have U+FE0F characters removed. See /annotationsDerived/
<symbol draft="contributed">↑↑↑</symbol>
</currency>
<currency type="TRY">
<pattern>¤#,##0.00</pattern>
<pattern numberSystem="latn">¤#,##0.00</pattern>
<displayName>Türk lirası</displayName>
<displayName count="one">↑↑↑</displayName>
<displayName count="other">↑↑↑</displayName>
Expand Down
7 changes: 1 addition & 6 deletions tools/cldr-code/src/main/java/org/unicode/cldr/test/CheckNumbers.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -259,12 +259,8 @@ public CheckCLDR handleCheck(
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I take it that NumericType still gets these items, so they aren't skipped by line 258.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the late response; this PR was put on hold pending branching maint/maint-48.

I don't quite understand the question.

This is more than 100 lines into handleCheck and this PR only does two things here, a few lines further down:

  1. It re-enables the "Number formats must have an explicit numberSystem attribute." check.
  2. It makes the check more general, since the numberSystem attribute can occur in different places, not necessary index 2.

Is the concern that some paths that would now match would be skipped prematurely because they have NumericType.NOT_NUMERIC or NumericType.RATIONAL? I could investigate that...

XPathParts parts = XPathParts.getFrozenInstance(path);

// TODO: re-enable this commented-out check
// Reference: https://unicode-org.atlassian.net/browse/CLDR-18722
/*
// Check that number formats are for an explicit number system.
String numberSystem = parts.getAttributeValue(2, "numberSystem");
if (numberSystem == null) {
if (!parts.containsAttribute("numberSystem")) {
result.add(
new CheckStatus()
.setCause(this)
Expand All @@ -273,7 +269,6 @@ public CheckCLDR handleCheck(
.setMessage(
"Number formats must have an explicit numberSystem attribute."));
}
*/

// Do tests that need to split the values

Expand Down
4 changes: 2 additions & 2 deletions tools/cldr-code/src/test/resources/org/unicode/cldr/unittest/data/common/dtd/ldml.dtd
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2051,7 +2051,7 @@ CLDR data files are interpreted according to the LDML specification (http://unic
<!ATTLIST decimal references CDATA #IMPLIED >
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering why this file exists; I've noticed that it causes errors when it drifts way from the regular ldml.dtd. If we keep it, we should have a test that it is identical (and maybe we do, and that's what alerted you to it?)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In eclipse I see the following error with that file:

Description Resource Path Location Type
The element 'gmtUnknownFormat' has not been declared. ldml.dtd /cldr-code/src/test/resources/org/unicode/cldr/unittest/data/common/dtd line 1723 DTD Problem

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused by that too. There are 3 ldml.dtd files:

  1. common/dtd/ldml.dtd
  2. tools/cldr-code/src/test/resources/org/unicode/cldr/unittest/data/common/dtd/ldml.dtd
  3. tools/cldr-code/target/test-classes/org/unicode/cldr/unittest/data/common/dtd/ldml.dtd

I was accidentally editing the wrong one before I noticed that...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact I did that again even after realizing there are 3 files; this PR still fails maybe for that reason. I'll make another (4th) commit shortly.

@srl295 do you have insight about how the ldml.dtd files relate to each other?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@srl295 I forget if we discussed this in the infrastructure meeting or slack. Although it's tangential to this ticket, maybe there should be a new ticket for it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No it's already on my list, just ignore that file i have an issue to correct it

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i recommend to revert changes to this file

<!--@METADATA-->
<!ATTLIST decimal numberSystem CDATA #IMPLIED >
<!--@DEPRECATED-->
<!--@MATCH:bcp47/nu-->
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
<!--@MATCH:bcp47/nu-->
<!--@DEPRECATED-->


<!ELEMENT group ( #PCDATA ) >
<!ATTLIST group alt NMTOKENS #IMPLIED >
Expand All @@ -2062,7 +2062,7 @@ CLDR data files are interpreted according to the LDML specification (http://unic
<!ATTLIST group references CDATA #IMPLIED >
<!--@METADATA-->
<!ATTLIST group numberSystem CDATA #IMPLIED >
<!--@DEPRECATED-->
<!--@MATCH:bcp47/nu-->
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
<!--@MATCH:bcp47/nu-->
<!--@DEPRECATED-->


<!ELEMENT list ( #PCDATA ) >
<!ATTLIST list alt NMTOKENS #IMPLIED >
Expand Down
Loading