first commit

This commit is contained in:
2026-09-08 20:28:54 +08:00
commit 2ada8d3d5a
37380 changed files with 4886169 additions and 0 deletions
+82
View File
@@ -0,0 +1,82 @@
# Change Log
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [4.1.3](https://github.com/messageformat/messageformat/compare/messageformat-parser@4.1.2...messageformat-parser@4.1.3) (2020-04-12)
**Note:** Version bump only for package messageformat-parser
## [4.1.2](https://github.com/messageformat/messageformat/compare/messageformat-parser@4.1.1...messageformat-parser@4.1.2) (2019-07-17)
### Bug Fixes
* Update dependencies ([b4907b5](https://github.com/messageformat/messageformat/commit/b4907b5))
## [4.1.1](https://github.com/messageformat/messageformat/compare/messageformat-parser@4.1.0...messageformat-parser@4.1.1) (2019-05-02)
### Bug Fixes
* Update dependencies ([11fd29b](https://github.com/messageformat/messageformat/commit/11fd29b))
* **parser:** Allow non-lowercase custom formatter names, as long as they do not mask plural/select/selectordinal ([a38b181](https://github.com/messageformat/messageformat/commit/a38b181)), closes [#230](https://github.com/messageformat/messageformat/issues/230)
# 4.1.0 (2019-03-03)
### Bug Fixes
* **parser:** Use a stack of values for inPlural (Fixes [#226](https://github.com/messageformat/messageformat/issues/226)) ([71b8400](https://github.com/messageformat/messageformat/commit/71b8400))
### Features
* Harmonise code style with Prettier & add linting with ESLint ([#220](https://github.com/messageformat/messageformat/issues/220)) ([18bc474](https://github.com/messageformat/messageformat/commit/18bc474))
* **parser:** Annotate rules for better errors ([cc010ac](https://github.com/messageformat/messageformat/commit/cc010ac)), closes [#215](https://github.com/messageformat/messageformat/issues/215)
# 4.0.0 (2018-12-05)
The changes included in this release are documented in PR [#8](https://github.com/messageformat/parser/pull/8):
* Allow MessageFormat content in function parameters
* Replace `strictNumberSign` option with broader `strict`
# 3.0.0 (2018-04-04)
* To conform with the ICU MessageFormat standard, escaping with a `\` is no longer supported (so e.g. the strings `\{` and `\u0123` are now passed through without changes). Instead, use ['quotes'](http://userguide.icu-project.org/formatparse/messages#TOC-Quoting-Escaping) to escape, e.g. `'{braces}'`. To help deal with this change, we provide a [codemod](https://github.com/messageformat/parser/blob/v3.0.0/codemod-fix-backslash-escapes.js) to handle the change for you.
* The optional parameter passed to a function is now a string, rather than an array of strings. So with e.g. `{var, func, your param}`, the parameter passed to func would be `' your param'`, including the leading space. As a consequence, we've also dropped the `strictFunctionParams` option.
# 2.0.0 (2017-11-18)
* Closer conformance with the ICU MessageFormat spec with respect to quoting and escaping text ([#3](https://github.com/messageformat/parser/pull/3)) and whitespace ([#6](https://github.com/messageformat/parser/pull/6)), thanks to [@nkovacs](https://github.com/nkovacs).
# 1.1.0 (2017-07-18)
* Add support for `strictFunctionParams`, thanks to [@nkovacs](https://github.com/nkovacs).
# 1.0.0 (2016-09-02)
* See [messageformat/messageformat#138](https://github.com/messageformat/messageformat/pull/138) for more information
+20
View File
@@ -0,0 +1,20 @@
Copyright OpenJS Foundation and contributors, https://openjsf.org/
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+167
View File
@@ -0,0 +1,167 @@
# messageformat-parser
A [PEG.js] parser for [ICU MessageFormat] strings part of [messageformat].
Outputs an AST defined by [parser.pegjs].
The generated `parse(src, [options])` function takes two parameters, first the
string to be parsed, and a second optional parameter `options`, an object with
the following possible keys:
- `cardinal` and `ordinal` Arrays of valid plural categories for the current
locale, used to validate `plural` and `selectordinal` keys. If these are
missing or set to false, the full set of valid [Unicode CLDR] keys is used:
`'zero', 'one', 'two', 'few', 'many', 'other'`. To disable this check, pass in
an empty array.
- `strict` By default, the parsing applies a few relaxations to the ICU
MessageFormat spec. Setting `strict: true` will disable these relaxations:
- The `argType` of `simpleArg` formatting functions will be restricted to the
set of `number`, `date`, `time`, `spellout`, `ordinal`, and `duration`,
rather than accepting any lower-case identifier that does not start with a
number.
- The optional `argStyle` of `simpleArg` formatting functions will not be
parsed as any other text, but instead as the spec requires: "In
argStyleText, every single ASCII apostrophe begins and ends quoted literal
text, and unquoted {curly braces} must occur in matched pairs."
- Inside a `plural` or `selectordinal` statement, a pound symbol (`#`) is
replaced with the input number. By default, `#` is also parsed as a special
character in nested statements too, and can be escaped using apostrophes
(`'#'`). In strict mode `#` will be parsed as a special character only
directly inside a `plural` or `selectordinal` statement. Outside those, `#`
and `'#'` will be parsed as literal text.
The parser only supports the default `DOUBLE_OPTIONAL` [apostrophe mode], in
which a single apostrophe only starts quoted literal text if it immediately
precedes a curly brace `{}`, or a pound symbol `#` if inside a plural format. A
literal apostrophe `'` is represented by either a single `'` or a doubled `''`
apostrophe character.
[icu messageformat]: https://messageformat.github.io/guide/
[messageformat]: https://messageformat.github.io/
[parser.pegjs]: ./parser.pegjs
[peg.js]: http://pegjs.org/
[unicode cldr]: http://cldr.unicode.org/index/cldr-spec/plural-rules
[apostrophe mode]: http://www.icu-project.org/apiref/icu4c/messagepattern_8h.html#af6e0757e0eb81c980b01ee5d68a9978b
## Installation
```sh
npm install messageformat-parser
```
## Usage
```js
> var parse = require('messageformat-parser').parse;
> parse('So {wow}.')
[ 'So ', { type: 'argument', arg: 'wow' }, '.' ]
> parse('Such { thing }. { count, selectordinal, one {First} two {Second}' +
' few {Third} other {#th} } word.')
[ 'Such ',
{ type: 'argument', arg: 'thing' },
'. ',
{ type: 'selectordinal',
arg: 'count',
offset: 0,
cases:
[ { key: 'one', tokens: [ 'First' ] },
{ key: 'two', tokens: [ 'Second' ] },
{ key: 'few', tokens: [ 'Third' ] },
{ key: 'other', tokens: [ { type: 'octothorpe' }, 'th' ] } ] },
' word.' ]
> parse('Many{type,select,plural{ numbers}selectordinal{ counting}' +
'select{ choices}other{ some {type}}}.')
[ 'Many',
{ type: 'select',
arg: 'type',
cases:
[ { key: 'plural', tokens: [ ' numbers' ] },
{ key: 'selectordinal', tokens: [ ' counting' ] },
{ key: 'select', tokens: [ ' choices' ] },
{ key: 'other', tokens: [ ' some',
{ type: 'argument', arg: 'type' } ] } ] },
'.' ]
> parse('{Such compliance')
// SyntaxError: Expected ",", "}" or [ \t\n\r] but "c" found.
> var msg = '{words, plural, zero{No words} one{One word} other{# words}}';
> var englishKeys = { cardinal: [ 'one', 'other' ],
ordinal: [ 'one', 'two', 'few', 'other' ] };
> parse(msg)
[ { type: 'plural',
arg: 'words',
offset: 0,
cases:
[ { key: 'zero', tokens: [ 'No words' ] },
{ key: 'one', tokens: [ 'One word' ] },
{ key: 'other', tokens: [ { type: 'octothorpe' }, ' words' ] } ] } ]
> parse(msg, englishKeys)
// Error: Invalid key `zero` for argument `words`. Valid plural keys for this
// locale are `one`, `other`, and explicit keys like `=0`.
```
For more example usage, please take a look at our [test suite](./test.js).
## Structure
The output of `parse()` is a `Token` array:
```typescript
type Token = string | Argument | Plural | Select | Function
type Argument = {
type: 'argument',
arg: Identifier
}
type Plural = {
type: 'plural' | 'selectordinal',
arg: Identifier,
offset: number,
cases: PluralCase[]
}
type Select = {
type: 'select',
arg: Identifier,
cases: SelectCase[]
}
type Function = {
type: 'function',
arg: Identifier,
key: Identifier,
param: {
tokens: options.strict ? [string] : (Token | Octothorpe)[]
} | null
}
type PluralCase = {
key: 'zero' | 'one' | 'two' | 'few' | 'many' | 'other' | '=0' | '=1' | '=2' | ...,
tokens: (Token | Octothorpe)[]
}
type SelectCase = {
key: Identifier,
tokens: options.strict ? Token[] : (Token | Octothorpe)[]
}
type Octothorpe = {
type: 'octothorpe'
}
type Identifier = string // not containing whitespace or control characters
```
---
[Messageformat](https://messageformat.github.io/) is an OpenJS Foundation project, and we follow its [Code of Conduct](https://github.com/openjs-foundation/cross-project-council/blob/master/CODE_OF_CONDUCT.md).
<a href="https://openjsf.org">
<img width=200 alt="OpenJS Foundation" src="https://messageformat.github.io/messageformat/logo/openjsf.svg" />
</a>
+59
View File
@@ -0,0 +1,59 @@
/**
* codemod for fixing backslash \escapes to quote 'escapes' in MessageFormat strings
*
* messageformat-parser v3 (used by messageformat v2) no longer allows for the
* characters #{}\ to be escaped with a \ prefix, as well as dropping support
* for \u0123 character escapes. This codemod can help fix your MessageFormat
* JSON sources to use ICU MessageFormat 'escapes' instead.
*
* To enable jscodeshift to handle JSON input, you'll need to have an
* appropriate parser available:
*
* npm install --no-save json-estree-ast
*
* Then apply the codemod:
*
* npx jscodeshift -t node_modules/messageformat-parser/codemod-fix-backslash-escapes.js [input]
*
* If your input includes doubled single quotes '', they will need to be
* escaped as well; use the command-line option --doubleSingleQuotes=true to
* enable that. Note that applying the codemod with that option multiple times
* will double your doubled quotes each time.
*/
let doubleSingleQuotes = false;
const fixEscapes = node => {
if (node.type !== 'Literal' || typeof node.value !== 'string') return;
if (doubleSingleQuotes) node.value = node.value.replace(/''+/g, '$&$&');
node.value = node.value.replace(
/('*)\\([#{}\\]|u[0-9a-f]{4})('*)/g,
(_, start, char, end) => {
switch (char[0]) {
case 'u': {
const code = parseInt(char.slice(1), 16);
return start + String.fromCharCode(code) + end;
}
case '\\':
return `${start}\\${end}`;
default:
// Assume multiple ' are already escaped
if (start === "'") start = "''";
if (end === "'") end = "''";
return `'${start}${char}${end}'`;
}
}
);
};
module.exports = ({ source }, { jscodeshift: j }, options) => {
if (options.doubleSingleQuotes) doubleSingleQuotes = true;
const ast = j(source);
ast.find(j.Property).forEach(({ value: { value } }) => fixEscapes(value));
ast
.find(j.ArrayExpression)
.forEach(({ value: { elements } }) => elements.forEach(fixEscapes));
return ast.toSource();
};
module.exports.parser = require('json-estree-ast');
+53
View File
@@ -0,0 +1,53 @@
{
"name": "messageformat-parser",
"version": "4.1.3",
"description": "A PEG.js parser for ICU MessageFormat strings",
"keywords": [
"icu",
"messageformat",
"parser"
],
"contributors": [
"Alex Sexton <alexsexton@gmail.com>",
"Eemeli Aro <eemeli@gmail.com>",
"Nikola Kovacs <nikola.kovacs@gmail.com>",
"Adrian Vogelsgesang <adrian.vogelsgesang@tum.de>"
],
"license": "MIT",
"homepage": "https://messageformat.github.io/",
"repository": {
"type": "git",
"url": "https://github.com/messageformat/messageformat.git",
"directory": "packages/parser"
},
"main": "parser.js",
"files": [
"parser.js",
"codemod-fix-backslash-escapes.js"
],
"eslintConfig": {
"env": {
"commonjs": true,
"es6": true
},
"overrides": [
{
"files": [
"test.js"
],
"env": {
"mocha": true
},
"rules": {
"no-unused-vars": 0
}
}
]
},
"scripts": {
"build": "pegjs parser.pegjs",
"prepublishOnly": "npm test",
"test": "mocha"
},
"gitHead": "dd10833d8d39155b005ea2ba6ec7a87f4daf0cd2"
}
+1991
View File
File diff suppressed because it is too large Load Diff