Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --offline flag and replace elm-json by elm-solve-deps-wasm #144

Merged
merged 98 commits into from
Oct 24, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
98 commits
Select commit Hold shift + click to select a range
28c5a5a
Add --offline flag
jfmengels Oct 12, 2023
fdf48aa
Add elm-solve-deps-wasm dependency
jfmengels Oct 12, 2023
d706822
Fetch dependencies
jfmengels Oct 12, 2023
7b5261d
Cache listing of versions
jfmengels Oct 12, 2023
c412577
Add initElmSolveDeps
jfmengels Oct 12, 2023
f4c514a
Delete tmp code
jfmengels Oct 12, 2023
ac5d321
Delete tmp code
jfmengels Oct 12, 2023
674e018
Add elm-syntax dependency using elm-solve-deps-wasm
jfmengels Oct 12, 2023
149900f
Move to separate module
jfmengels Oct 12, 2023
1d00c0d
Add dependency provider
jfmengels Oct 12, 2023
aeca52e
Extract variable
jfmengels Oct 12, 2023
4481f1e
Expose elmReviewDependencyCache
jfmengels Oct 12, 2023
c598020
Add online solver
jfmengels Oct 12, 2023
471bd0a
Add getElmJsonFromElmHomePath
jfmengels Oct 12, 2023
0e92719
Pass elm version to get elm home
jfmengels Oct 12, 2023
938ec5f
Put cache under the version
jfmengels Oct 12, 2023
d6b9d4f
Get elm home
jfmengels Oct 12, 2023
bd168e1
Parse dependencies
jfmengels Oct 12, 2023
7de51dd
Declare dependency solver once
jfmengels Oct 12, 2023
a86e70a
Format
jfmengels Oct 12, 2023
905b60d
Pass elmVersion around
jfmengels Oct 12, 2023
826f790
Extract solve function
jfmengels Oct 12, 2023
a90eaa7
Parse in solve
jfmengels Oct 12, 2023
9ee9215
Solve using elm-solve-deps in TemplateDependencies.get
jfmengels Oct 12, 2023
872aefd
Remove dependency solver
jfmengels Oct 12, 2023
20acb08
Pass useTest
jfmengels Oct 12, 2023
3836223
Initialize syncGetWorker when needed
jfmengels Oct 13, 2023
0fa6a44
Use elm-solve-deps to init a new project
jfmengels Oct 13, 2023
1087e13
Use elm-solve-deps for --template
jfmengels Oct 13, 2023
b8154f5
Remove elm-json related code from project
jfmengels Oct 13, 2023
09d6342
Format
jfmengels Oct 13, 2023
00b28e4
Have some commands try online first
jfmengels Oct 13, 2023
15e54ba
Fix online fetching not working
jfmengels Oct 13, 2023
de663ed
Rename TemplateDependencies.add to createNewReviewElmJson
jfmengels Oct 13, 2023
1fac321
Move definition of elmJson
jfmengels Oct 13, 2023
5dc9f53
Compute test dependencies first
jfmengels Oct 13, 2023
b3168c2
Fix generation of review/elm.json during init
jfmengels Oct 13, 2023
658365c
Set dependency provider to null when tearing it down
jfmengels Oct 13, 2023
3e74a4e
Improve teardown of dependencies provider
jfmengels Oct 13, 2023
13ef1da
Fix process staying alive?
jfmengels Oct 13, 2023
0e5af93
Remove useTest argument
jfmengels Oct 13, 2023
a05f750
Fix upgrade of dependencies
jfmengels Oct 13, 2023
4e6d440
Move validation of jfmengels/elm-review version for templates
jfmengels Oct 14, 2023
331bc1e
Only save elm.json file during init, not new-package
jfmengels Oct 14, 2023
8db019d
Upgrade dependencies
jfmengels Oct 14, 2023
b3764ba
Convert types to TypeScript
jfmengels Oct 14, 2023
a613ab9
Remove unused file
jfmengels Oct 14, 2023
033a99e
Remove extraneous arguments
jfmengels Oct 14, 2023
eba60df
Remove TypeScript errors
jfmengels Oct 14, 2023
36cfc45
Fix ESLint errors
jfmengels Oct 14, 2023
fbe074a
Improve TS for sync-get
jfmengels Oct 15, 2023
89e2843
Re-order classes and add documentation
jfmengels Oct 15, 2023
b18a07a
Use plain conversions instead of using Intl.collator
jfmengels Oct 15, 2023
03a6ef6
Update tooling integration documentation
jfmengels Oct 15, 2023
0aff6c1
Pass options to onlineCache.update
jfmengels Oct 15, 2023
9a57023
Don't cache dependencies computation anymore
jfmengels Oct 15, 2023
95912cd
Put dependency cache inside elm-stuff
jfmengels Oct 15, 2023
1323cdc
Pass options to fetchElmJsonOnline
jfmengels Oct 15, 2023
bd48811
Add type annotation
jfmengels Oct 15, 2023
3d8f779
Re-use existing folder to store downloaded elm.json files instead of …
jfmengels Oct 15, 2023
d676356
Flatten functions
jfmengels Oct 15, 2023
ab2cd2c
Remove duplicate writing of elm.json file during init
jfmengels Oct 15, 2023
072e0fb
Always tear down after TemplateDependencies.update()
jfmengels Oct 15, 2023
8e4b8d0
Pass elmVersion as a parameter
jfmengels Oct 15, 2023
6c7d9e6
Add TODO
jfmengels Oct 14, 2023
96566b2
Update CHANGELOG
jfmengels Oct 15, 2023
f93c2bf
2.11.0-beta.1
jfmengels Oct 15, 2023
c91aa08
Fix illegal FS operation on versions cache file
jfmengels Oct 16, 2023
0169a2d
2.11.0-beta.2
jfmengels Oct 16, 2023
b90c446
Remove test dependencies when compiling review/elm.json
jfmengels Oct 16, 2023
bbf6531
Update snapshots
jfmengels Oct 16, 2023
eacf715
2.11.0-beta.3
jfmengels Oct 16, 2023
fa3eb61
Update snapshots
jfmengels Oct 17, 2023
5300678
Document the --offline flag
jfmengels Oct 17, 2023
b2d6a96
Avoid downloading docs.json in offline mode
jfmengels Oct 17, 2023
627e895
Report an error when using both --offline and --template
jfmengels Oct 17, 2023
1755390
Report an error when using both --offline and new-package
jfmengels Oct 17, 2023
5fa9ad6
Extract function
jfmengels Oct 21, 2023
6d93a0c
Store downloaded info about packages into ELM_HOME
jfmengels Oct 21, 2023
82c1822
Update snapshots
jfmengels Oct 21, 2023
b6ca921
Fix TypeScript errors
jfmengels Oct 19, 2023
174e3a9
Add prepare-offline command
jfmengels Oct 18, 2023
83f4872
Define ELM_HOME in tests
jfmengels Oct 19, 2023
b5b16fb
Define ELM_HOME when running tests
jfmengels Oct 19, 2023
8bf5abd
Clean up tmp folder
jfmengels Oct 21, 2023
a941d06
Move package path
jfmengels Oct 21, 2023
93b777f
Add TypeScript types
jfmengels Oct 21, 2023
6802835
Formatting
jfmengels Oct 21, 2023
753c832
Download dependencies of the target project
jfmengels Oct 21, 2023
2987a29
Rename get to addRequiredDependencies
jfmengels Oct 22, 2023
3d4c0b6
Make a nicer error when dependencies can't be updated
jfmengels Oct 22, 2023
4dc64d6
Move creation/loading of elm.json for the parser application
jfmengels Oct 22, 2023
fb1c553
Extract function
jfmengels Oct 24, 2023
58fb68c
Remove unnecessary default value
jfmengels Oct 24, 2023
3736f9b
Extract variable
jfmengels Oct 24, 2023
ddc0614
Move variable
jfmengels Oct 24, 2023
57575cc
Also remove direct dependencies from indirect test-dependencies
jfmengels Oct 24, 2023
5137ae2
Upgrade test dependencies when running init or new-package
jfmengels Oct 24, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,13 @@

## [Unreleased]

- Fixed a crash when using `--template` from the same repository but from two different paths.
## [2.11.0] **BETA** - 2023-10-16

- Add an `offline` mode to prevent `elm-review` from making any HTTP requests. This is useful for CI environments that should not have access to the internet, where you only want to run `elm-review` without arguments.
- Replaced the internally used `elm-json` dependency with `elm-solve-deps-wasm`, which should be more reliable, re-enable support for old MacOS versions as well as improve performance.
- Fixed a crash when using `--template` from the same repository but from two different paths.

This is a **BETA** release, so I expect things to break. Please report any issues you encounter on the #elm-review channel on the Elm Slack.

## [2.10.3] - 2023-09-26

Expand Down
2 changes: 1 addition & 1 deletion documentation/tooling-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ The CLI creates a bunch of cache inside `elm-stuff/generated-code/jfmengels/elm-
- `file-cache/`: Caching of the file's ASTs.
- `review-applications/`: Caching of the project's configuration. This is the application we build by compiling the source code in the CLI's `template/` directory.
- `result-cache/`: Caching of the results of each module's analysis for every rule.
- `dependencies-cache/`: Caching of the dependencies of the project's configuration computed by `elm-json`. `elm-json` is a bit slow, and doesn't work great offline. This is done so we don't have to compute the dependencies again if the configuration changed but not `review/elm.json`.
- `dependencies-cache/`: Caching of data related to Elm dependencies computed by `elm-solve-deps-wasm` (or `elm-json` prior to `2.11.0`).
- `elm-parser/`: Caching of the parser application. This is the application we build by compiling with the user's version of `stil4m/elm-syntax`, and the name of the file reflect the version. This is used to parallelize the parsing of the files, which is kind of slow, at startup.

Namespacing things means that data will unfortunately be duplicated, but it is meant to prevent different tools from stepping on each other's toes by saving the same files at the same time as another, potentially corrupting the files.
Expand Down
84 changes: 26 additions & 58 deletions lib/build.js
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ const TemplateDependencies = require('./template-dependencies');
* @typedef { import("./types/build").BuildResult } BuildResult
* @typedef { import("./types/build").AppHash } AppHash
* @typedef { import("./types/build").ReviewElmJson } ReviewElmJson
* @typedef { import("./types/elm-version").ElmVersion } ElmVersion
* @typedef { import("./types/path").Path } Path
*/

Expand Down Expand Up @@ -159,13 +160,7 @@ I can help set you up with an initial configuration if you run ${chalk.magenta('

const buildResult = await Promise.all([
getElmBinary(options),
createTemplateProject(
options,
reviewElmJsonPath,
userSrc,
buildFolder,
reviewElmJson
)
createTemplateProject(options, userSrc, buildFolder, reviewElmJson)
]).then(([elmBinary]) => {
Debug.log('Compiling review application');
Benchmark.start(options, 'Compile review project');
Expand Down Expand Up @@ -257,7 +252,6 @@ async function buildFromGitHubTemplate(options, template) {
getElmBinary(options),
createTemplateProject(
options,
reviewElmJsonPath,
buildFolder,
path.join(buildFolder, 'project'),
reviewElmJsonWithReplacedParentDirectories
Expand Down Expand Up @@ -294,15 +288,13 @@ async function buildFromGitHubTemplate(options, template) {
/**
* Create a local temporary project to build an Elm application in.
* @param {Options} options
* @param {Path} reviewElmJsonPath
* @param {Path} userSrc
* @param {Path} projectFolder
* @param {ReviewElmJson} reviewElmJson
* @returns {Promise<void>}
*/
async function createTemplateProject(
options,
reviewElmJsonPath,
userSrc,
projectFolder,
reviewElmJson
Expand All @@ -312,17 +304,14 @@ async function createTemplateProject(
const elmJsonPath = path.join(projectFolder, 'elm.json');

// Load review project's elm.json file contents
const [dependencies, previousElmJson] = await Promise.all([
TemplateDependencies.get(
options,
reviewElmJson.dependencies,
reviewElmJsonPath
),
FS.readFile(elmJsonPath).catch(() => null)
const [previousElmJson, dependencies] = await Promise.all([
FS.readFile(elmJsonPath).catch(() => null),
TemplateDependencies.addRequiredDependencies(options, reviewElmJson)
]);
const finalElmJson = updateSourceDirectories(options, userSrc, {
...reviewElmJson,
dependencies
dependencies,
'test-dependencies': {direct: {}, indirect: {}}
});
const finalElmJsonAsString = JSON.stringify(finalElmJson, null, 4);
if (previousElmJson !== finalElmJsonAsString) {
Expand Down Expand Up @@ -558,38 +547,14 @@ async function buildElmParser(options, reviewElmJson) {

Debug.log(`Building parser app for elm-syntax v${elmSyntaxVersion}`);

const parseElmElmJsonPath = path.resolve(parseElmFolder, 'elm.json');
const parseElmElmJson = await FS.readJsonFile(parseElmElmJsonPath).catch(
(error) => {
if (error.code === 'ENOENT') {
return Promise.reject(
new ErrorMessage.CustomError(
// prettier-ignore
'UNEXPECTED INTERNAL ERROR',
// prettier-ignore
`I was expecting to find the "parseElm" project at ${chalk.cyan(parseElmElmJsonPath)} but could not find it.

Please open an issue at the following link:
https://github.com/jfmengels/node-elm-review/issues/new
`,
options.elmJsonPath
)
);
}

return Promise.reject(error);
}
);

const buildFolder = options.buildFolderForParserApp();

const [elmBinary] = await Promise.all([
getElmBinary(options),
createParserElmJsonFile(
options,
buildFolder,
parseElmElmJson,
parseElmElmJsonPath,
reviewElmJson['elm-version'],
elmSyntaxVersion
),
// Needed when the user has `"type": "module"` in their package.json.
Expand All @@ -612,33 +577,36 @@ https://github.com/jfmengels/node-elm-review/issues/new
);
}

/** Create `elm.json` file for the parser application, which will use the exact same version
* of `stil4m/elm-syntax` as the review application.
*
* @param {Options} options
* @param {Path} buildFolder
* @param {ElmVersion} elmVersion
* @param {ElmVersion} elmSyntaxVersion
* @return {Promise<void>}
*/
async function createParserElmJsonFile(
options,
buildFolder,
parseElmElmJson,
parseElmElmJsonPath,
elmVersion,
elmSyntaxVersion
) {
const dependencies = await TemplateDependencies.addElmSyntax(
/** @type {ReviewElmJson} */
const parseElmElmJson = await TemplateDependencies.addElmSyntax(
options,
parseElmElmJsonPath,
elmVersion,
elmSyntaxVersion
);

parseElmElmJson['source-directories'] = parseElmElmJson[
'source-directories'
].map((dir) => path.resolve(parseElmFolder, dir));

await FS.mkdirp(buildFolder);
return FS.writeFile(
path.resolve(buildFolder, 'elm.json'),
JSON.stringify(
{
...parseElmElmJson,
dependencies,
'source-directories': parseElmElmJson['source-directories'].map((dir) =>
path.resolve(parseElmFolder, dir)
)
},
null,
2
)
JSON.stringify(parseElmElmJson, null, 2)
);
}

Expand Down
Loading