You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We have different ways of comparing strings, none of which are "correct". Now that's we've updated to a version of endo with endojs/endo#2008, we can use its compareByCodePoints throughout.
To Reproduce
Incorrect string comparison
a < b or any variation. This results in a comparison by UTF-16 code units.
[a, b].sort() similarly uses code units
a.localeCompare() compares by environment specific local rules. The SES shim is supposed to tame this by default, but I have seen surprising results (see Consistent store order #10299 test and undo localeCompare change).
Expected behavior
Comparison by codepoints which we're trying to standardize on, and is the behavior for any implementation that compares utf-8 encoded bytes like SQLite, golang (doesn't strictly requires that string be utf-8, but literals and range iterations are), and JSON (which specifies utf-8 as the default encoding).
Additional context
#10299 fixes the more egregious of these cases, which caused a discrepancy in the order of Stores when they're heap or virtual/durable, and whether they're in a real or fake liveslots environment. In comments, @gibson042 also points out at other cases of mistaken comparisons (likely not exhaustive).
The text was updated successfully, but these errors were encountered:
Describe the bug
We have different ways of comparing strings, none of which are "correct". Now that's we've updated to a version of endo with endojs/endo#2008, we can use its
compareByCodePoints
throughout.To Reproduce
Incorrect string comparison
a < b
or any variation. This results in a comparison by UTF-16 code units.[a, b].sort()
similarly uses code unitsa.localeCompare()
compares by environment specific local rules. The SES shim is supposed to tame this by default, but I have seen surprising results (see Consistent store order #10299 test and undo localeCompare change).Expected behavior
Comparison by codepoints which we're trying to standardize on, and is the behavior for any implementation that compares utf-8 encoded bytes like SQLite, golang (doesn't strictly requires that string be utf-8, but literals and range iterations are), and JSON (which specifies utf-8 as the default encoding).
Additional context
#10299 fixes the more egregious of these cases, which caused a discrepancy in the order of Stores when they're heap or virtual/durable, and whether they're in a real or fake liveslots environment. In comments, @gibson042 also points out at other cases of mistaken comparisons (likely not exhaustive).
The text was updated successfully, but these errors were encountered: