Skip to content

Commit c924822

Browse files
authored
Merge pull request JuliaText#58 from SambhawDrag/Sam/Typos
Fix Typos and Indentation
2 parents 3dfb825 + d8a5910 commit c924822

File tree

1 file changed

+12
-12
lines changed

1 file changed

+12
-12
lines changed

README.md

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -337,12 +337,12 @@ julia> tk = tokenizer(spm, "i love the julia language") #or tk = spm("i love the
337337
"▁julia"
338338
"▁language"
339339

340-
julia> subword = tokenizer(spm, "unfriendly")
341-
2-element Array{String,1}:
340+
julia> subword = tokenizer(spm, "unfriendly")
341+
2-element Array{String,1}:
342342
"▁un"
343343
"friendly"
344344

345-
julia> para = spm("Julia is a high-level, high-performance dynamic language for technical computing")
345+
julia> para = spm("Julia is a high-level, high-performance dynamic language for technical computing")
346346
17-element Array{String,1}:
347347
""
348348
"J"
@@ -366,15 +366,15 @@ julia> tk = tokenizer(spm, "i love the julia language") #or tk = spm("i love the
366366
Indices is usually used for deep learning models.
367367
Index of special tokens in ALBERT are given below:
368368

369-
1 => <pad>
370-
2 => <unk>
371-
3 => [CLS]
372-
4 => [SEP]
373-
5 => [MASK]
369+
1 [PAD]
370+
2 [UNK]
371+
3 [CLS]
372+
4 [SEP]
373+
5 [MASK]
374374

375375

376376
```julia
377-
julia> ids_from_tokens(spm , tk)
377+
julia> ids_from_tokens(spm, tk)
378378
4-element Array{Int64,1}:
379379
32
380380
340
@@ -383,13 +383,13 @@ julia> ids_from_tokens(spm , tk)
383383
817
384384
#we can also get sentences back from tokens
385385
julia> sentence_from_tokens(tk)
386-
"i love the julia language"
386+
"i love the julia language"
387387

388388
julia> sentence_from_token(subword)
389-
"unfriendly"
389+
"unfriendly"
390390

391391
julia> sentence_from_tokens(para)
392-
"Julia is a high-level, high-performance dynamic language for technical computing"
392+
"Julia is a high-level, high-performance dynamic language for technical computing"
393393
```
394394

395395
## Contributing

0 commit comments

Comments
 (0)