Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Localization Language Feature #618

Merged
merged 23 commits into from
Sep 17, 2023
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
8ac5b1d
Initial commit for semantic tokenizer.
gdotdesign Jun 12, 2023
60f1bb5
Simplify semantic tokenizer a bit.
gdotdesign Jun 13, 2023
0e865c6
Working language server implementation.
gdotdesign Jun 13, 2023
1c14a5a
Cleanup semantic tokenizer class.
gdotdesign Jun 13, 2023
ca01ad6
Save keywords automatically instead of manually.
gdotdesign Jun 13, 2023
6926d57
Use an array derived from the actual token types.
gdotdesign Jun 13, 2023
29fb79b
Implement suggestions from code review.
gdotdesign Jun 14, 2023
6b25fc8
Update src/ls/semantic_tokens.cr
gdotdesign Jun 14, 2023
0eea575
Implement HTML highlighting.
gdotdesign Jun 14, 2023
1cd96da
Implement highlight directive.
gdotdesign Jun 14, 2023
529117d
Avoid unnecessary interations.
gdotdesign Jun 14, 2023
79c3f11
Implement suggestions from code review.
gdotdesign Jun 14, 2023
6faec26
Use the ast from the workspace semantic tokens.
gdotdesign Jun 25, 2023
37db9b6
Implementation of localization language structures.
gdotdesign Jun 26, 2023
6e51ae0
Update operation.cr
gdotdesign Jul 18, 2023
673b361
Merge branch 'master' into locales
gdotdesign Jul 18, 2023
92833ca
Update test.
gdotdesign Jul 21, 2023
5e9eef7
Merge branch 'master' into locales
gdotdesign Aug 8, 2023
44ab595
Revert change to the operation formatting.
gdotdesign Aug 8, 2023
a8ecd78
Update Locale.md
gdotdesign Aug 8, 2023
d28c7a3
Merge branch 'master' into locales
gdotdesign Sep 6, 2023
d171155
Merge branch 'master' into locales
gdotdesign Sep 7, 2023
3c4010c
Apply suggestions from code review
gdotdesign Sep 11, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions core/source/Locale.mint
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
module Locale {
/* Sets the current locale. */
fun set (locale : String) : Bool {
gdotdesign marked this conversation as resolved.
Show resolved Hide resolved
`_L.set(#{locale})`
}

/* Returns the current locale. */
fun get : Maybe(String) {
`
(() => {
if (_L.locale) {
return #{Maybe::Just(`_L.locale`)}
} else {
return #{Maybe::Nothing}
}
})()
`
}
}
49 changes: 49 additions & 0 deletions documentation/Language/Locale.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Locale

This feature of the language allows specifing localization tokens and values for languages indentified by [ISO 639-1](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) codes.

```mint
locale en {
ui: {
buttons: {
ok: "OK"
}
}
}
gdotdesign marked this conversation as resolved.
Show resolved Hide resolved

locale hu {
ui: {
buttons: {
ok: "Rendben"
}
}
}
```

A locale consists of a tree structure where the keys are lowercase identifiers and the values are expressions.

To localize a value you need to use the locale token:

```
:ui.buttons.ok
gdotdesign marked this conversation as resolved.
Show resolved Hide resolved
```

or if it's a function if can be called:

```
:ui.buttons.ok(param1, param2)
gdotdesign marked this conversation as resolved.
Show resolved Hide resolved
```

To get and set the current locale the `Locale` module can be used:

```
Locale.set("en")
Locale.get() // Maybe::Just("en")
```

Every translation is typed checked:

* all translations of the same key must have the same type
* locale keys must have translations in every defined language

Locales are open like Modules are so they can be defined in multiple places.
25 changes: 25 additions & 0 deletions spec/compilers/locale_key
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
locale en {
test: "Hello"
}

component Main {
fun render : String {
:test
}
}
--------------------------------------------------------------------------------
class A extends _C {
componentWillUnmount() {
_L._unsubscribe(this);
}

componentDidMount() {
_L._subscribe(this);
}

render() {
return _L.t("test");
}
};

A.displayName = "Main";
11 changes: 11 additions & 0 deletions spec/formatters/locale
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
locale en {
ui: {
button: {
ok: "OK"
}
}
}
--------------------------------------------------------------------------------
locale en {
ui: { button: { ok: "OK" } }
}
23 changes: 23 additions & 0 deletions spec/formatters/locale_key
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
locale en {
ui: {
button: {
ok: "OK"
}
}
}

component Main {
fun render : String {
:ui.button.ok
}
}
--------------------------------------------------------------------------------
locale en {
ui: { button: { ok: "OK" } }
}

component Main {
fun render : String {
:ui.button.ok
}
}
11 changes: 11 additions & 0 deletions spec/parsers/locale_key_spec.cr
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
require "../spec_helper"

describe "Locale" do
subject locale_key

expect_ignore "comp"
expect_ignore "asd"
expect_ignore ":"

expect_ok ":ui.button.ok"
end
16 changes: 16 additions & 0 deletions spec/parsers/locale_spec.cr
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
require "../spec_helper"

describe "Locale" do
subject locale

expect_ignore "comp"
expect_ignore "asd"

expect_error "locale", Mint::Parser::LocaleExpectedLanguage
expect_error "locale{", Mint::Parser::LocaleExpectedLanguage
expect_error "locale ", Mint::Parser::LocaleExpectedLanguage
expect_error "locale en", Mint::Parser::LocaleExpectedOpeningBracket
expect_error "locale en {", Mint::Parser::LocaleExpectedClosingBracket

expect_ok "locale en { a: \"\" }"
end
43 changes: 43 additions & 0 deletions spec/type_checking/locale_key
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
locale en {
test: ""
}

component Main {
fun render : String {
:test
}
}
-------------------------------------------------------------TranslationMissing
component Main {
fun render : String {
:test
}
}
-------------------------------------------------------TranslationNotTranslated
locale en {
test: ""
}

locale hu {

}

component Main {
fun render : String {
:test
}
}
------------------------------------------------------------TranslationMismatch
locale en {
test: ""
}

locale hu {
test: 0
}

component Main {
fun render : String {
:test
}
}
22 changes: 19 additions & 3 deletions src/ast.cr
Original file line number Diff line number Diff line change
Expand Up @@ -30,18 +30,21 @@ module Mint
If |
Js

getter components, modules, records, stores, routes, providers
getter suites, enums, comments, nodes, unified_modules, keywords
getter operators
getter components, modules, records, stores, routes, providers, operators
getter suites, enums, comments, nodes, keywords, locales

getter unified_modules, unified_locales

def initialize(@operators = [] of Tuple(Int32, Int32),
@keywords = [] of Tuple(Int32, Int32),
@records = [] of RecordDefinition,
@unified_modules = [] of Module,
@unified_locales = [] of Locale,
@components = [] of Component,
@providers = [] of Provider,
@comments = [] of Comment,
@modules = [] of Module,
@locales = [] of Locale,
@routes = [] of Routes,
@suites = [] of Suite,
@stores = [] of Store,
Expand Down Expand Up @@ -77,6 +80,7 @@ module Mint
@comments.concat ast.comments
@modules.concat ast.modules
@records.concat ast.records
@locales.concat ast.locales
@stores.concat ast.stores
@routes.concat ast.routes
@suites.concat ast.suites
Expand Down Expand Up @@ -111,6 +115,18 @@ module Mint
)
end

@unified_locales =
@locales
.group_by(&.language)
.map do |_, locales|
Locale.new(
input: Data.new(input: "", file: ""),
fields: locales.flat_map(&.fields),
gdotdesign marked this conversation as resolved.
Show resolved Hide resolved
language: locales.first.language,
comment: nil,
from: 0,
to: 0)
end
self
end
end
Expand Down
4 changes: 3 additions & 1 deletion src/ast/component.cr
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@ module Mint
class Component < Node
getter properties, connects, styles, states, comments
getter functions, gets, uses, name, comment, refs, constants
getter? global

getter? global, locales

def initialize(@refs : Array(Tuple(Variable, Node)),
@properties : Array(Property),
Expand All @@ -16,6 +17,7 @@ module Mint
@comment : Comment?,
@gets : Array(Get),
@uses : Array(Use),
@locales : Bool,
@global : Bool,
@name : TypeId,
@input : Data,
Expand Down
15 changes: 15 additions & 0 deletions src/ast/locale.cr
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
module Mint
class Ast
class Locale < Node
getter fields, comment, language

def initialize(@fields : Array(RecordField),
@comment : Comment?,
@language : String,
@input : Data,
@from : Int32,
@to : Int32)
end
end
end
end
13 changes: 13 additions & 0 deletions src/ast/locale_key.cr
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
module Mint
class Ast
class LocaleKey < Node
getter value

def initialize(@value : String,
@input : Data,
@from : Int32,
@to : Int32)
end
end
end
end
2 changes: 1 addition & 1 deletion src/compiler.cr
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ module Mint

delegate lookups, checked, cache, component_records, to: @artifacts
delegate ast, types, variables, resolve_order, to: @artifacts
delegate record_field_lookup, to: @artifacts
delegate record_field_lookup, locales, to: @artifacts

getter js, style_builder, static_components, static_components_pool
getter build, relative
Expand Down
5 changes: 5 additions & 0 deletions src/compilers/component.cr
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,11 @@ module Mint
"componentDidMount" => %w[],
}

if node.locales?
heads["componentWillUnmount"] << "_L._unsubscribe(this)"
heads["componentDidMount"] << "_L._subscribe(this)"
end

node.connects.each do |item|
store =
ast.stores.find(&.name.value.==(item.store.value))
Expand Down
7 changes: 7 additions & 0 deletions src/compilers/locale_key.cr
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
module Mint
class Compiler
def _compile(node : Ast::LocaleKey) : String
%(_L.t("#{node.value}"))
end
end
end
52 changes: 52 additions & 0 deletions src/compilers/top_level.cr
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,22 @@ module Mint
end
end

def compiled_locales
mapped =
locales.each_with_object({} of String => Hash(String, String)) do |(key, data), memo|
data.each do |language, node|
if node.in?(checked)
memo[language] ||= {} of String => String
memo[language]["'#{key}'"] = compile(node)
end
end
end

js.object(mapped.each_with_object({} of String => String) do |(language, tokens), memo|
memo[language] = js.object(tokens)
end)
end

# --------------------------------------------------------------------------

# Wraps the application with the runtime
Expand Down Expand Up @@ -271,6 +287,42 @@ module Mint
const _PV = Symbol("Variable")
const _PS = Symbol("Spread")

class Locale {
constructor(translations) {
this.locale = Object.keys(translations)[0];
this.translations = translations;
this.listeners = new Set();
}

set(locale) {
if (this.locale != locale && this.translations[locale]) {
this.locale = locale;

for (let listener of this.listeners) {
listener.forceUpdate();
}

return true
} else {
return false
}
}

t(key) {
return this.translations[this.locale][key]
}

_subscribe(owner) {
this.listeners.add(owner);
}

_unsubscribe(owner) {
this.listeners.delete(owner);
}
}

const _L = new Locale(#{compiled_locales});

class RecordPattern {
constructor(patterns) {
this.patterns = patterns
Expand Down
Loading