Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Robustness against corrupt package store #3199

Open
maleadt opened this issue Sep 13, 2022 · 2 comments
Open

Robustness against corrupt package store #3199

maleadt opened this issue Sep 13, 2022 · 2 comments

Comments

@maleadt
Copy link
Member

maleadt commented Sep 13, 2022

Recently, PkgEval testing of lots of packages broke because of a corrupt entry in .julia/packages (which is shared and persistent). MWE:

$ rm -rf ~/.julia/packages/Distributions/39PV5
$ mkdir ~/.julia/packages/Distributions/39PV5

julia> using Distributions
ERROR: ArgumentError: Package Distributions [31c24e10-a181-5473-b8eb-7969acd0382f] is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.

Actual backtrace:


     Testing Bootstrap
      Status `/tmp/jl_oUIXPn/Project.toml`
  [e28b5b4c] Bootstrap v2.3.3
  [a93c6f00] DataFrames v1.3.5
  [31c24e10] Distributions v0.25.70
  [38e38edf] GLM v1.8.0
  [ce6b1742] RDatasets v0.7.7
  [2913bbd2] StatsBase v0.33.21
  [3eaba693] StatsModels v0.6.31
  [9a3f8284] Random `@stdlib/Random`
  [10745b16] Statistics `@stdlib/Statistics`
  [8dfed614] Test `@stdlib/Test`
      Status `/tmp/jl_oUIXPn/Manifest.toml`
  [e28b5b4c] Bootstrap v2.3.3
  [336ed68f] CSV v0.10.4
  [49dc2e85] Calculus v0.5.1
  [324d7699] CategoricalArrays v0.10.6
  [d360d2e6] ChainRulesCore v1.15.5
  [9e997f8a] ChangesOfVariables v0.1.4
  [944b1d66] CodecZlib v0.7.0
  [34da2185] Compat v4.2.0
  [a8cc5b0e] Crayons v4.1.1
  [9a962f9c] DataAPI v1.10.0
  [a93c6f00] DataFrames v1.3.5
  [864edb3b] DataStructures v0.18.13
  [e2d170a0] DataValueInterfaces v1.0.0
  [b429d917] DensityInterface v0.4.0
  [31c24e10] Distributions v0.25.70
  [ffbed154] DocStringExtensions v0.9.1
  [fa6b7ba4] DualNumbers v0.6.8
  [e2ba6199] ExprTools v0.1.8
  [5789e2e9] FileIO v1.15.0
  [48062228] FilePathsBase v0.9.20
  [1a297f60] FillArrays v0.13.4
  [59287772] Formatting v0.4.2
  [38e38edf] GLM v1.8.0
  [34004b35] HypergeometricFunctions v0.3.11
  [842dd82b] InlineStrings v1.1.4
  [3587e190] InverseFunctions v0.1.7
  [41ab1584] InvertedIndices v1.1.0
  [92d709cd] IrrationalConstants v0.1.1
  [82899510] IteratorInterfaceExtensions v1.0.0
  [692b3bcd] JLLWrappers v1.4.1
  [2ab3a3ac] LogExpFunctions v0.3.18
  [e1d29d7a] Missings v1.0.2
⌃ [78c3b35d] Mocking v0.7.2
  [77ba4419] NaNMath v1.0.1
  [bac558e1] OrderedCollections v1.4.1
  [90014a1f] PDMats v0.11.16
  [69de0a69] Parsers v2.4.0
  [2dfb63ee] PooledArrays v1.4.2
  [21216c6a] Preferences v1.3.0
⌅ [08abe8d2] PrettyTables v1.3.1
  [1fd47b50] QuadGK v2.5.0
  [df47a6cb] RData v0.8.3
  [ce6b1742] RDatasets v0.7.7
  [3cdcf5f2] RecipesBase v1.2.1
  [189a3867] Reexport v1.2.2
  [ae029012] Requires v1.3.0
  [79098fc4] Rmath v0.7.0
  [6c6a2e73] Scratch v1.1.1
  [91c51154] SentinelArrays v1.3.13
  [1277b4bf] ShiftedArrays v1.0.0
  [a2af1166] SortingAlgorithms v1.0.1
  [276daf66] SpecialFunctions v2.1.7
  [82ae8749] StatsAPI v1.5.0
  [2913bbd2] StatsBase v0.33.21
  [4c63d2b9] StatsFuns v1.0.1
  [3eaba693] StatsModels v0.6.31
  [3783bdb8] TableTraits v1.0.1
  [bd369af6] Tables v1.7.0
  [f269a46b] TimeZones v1.9.0
  [3bb67fe8] TranscodingStreams v0.9.9
  [ea10d353] WeakRefStrings v1.4.2
  [efe28fd5] OpenSpecFun_jll v0.5.5+0
  [f50d1b31] Rmath_jll v0.3.0+0
  [0dad84c5] ArgTools v1.1.1 `@stdlib/ArgTools`
  [56f22d72] Artifacts `@stdlib/Artifacts`
  [2a0f44e3] Base64 `@stdlib/Base64`
  [ade2ca70] Dates `@stdlib/Dates`
  [f43a241f] Downloads v1.6.0 `@stdlib/Downloads`
  [7b1f6079] FileWatching `@stdlib/FileWatching`
  [9fa8497b] Future `@stdlib/Future`
  [b77e0a4c] InteractiveUtils `@stdlib/InteractiveUtils`
  [4af54fe1] LazyArtifacts `@stdlib/LazyArtifacts`
  [b27032c2] LibCURL v0.6.3 `@stdlib/LibCURL`
  [76f85450] LibGit2 `@stdlib/LibGit2`
  [8f399da3] Libdl `@stdlib/Libdl`
  [37e2e46d] LinearAlgebra `@stdlib/LinearAlgebra`
  [56ddb016] Logging `@stdlib/Logging`
  [d6f4376e] Markdown `@stdlib/Markdown`
  [a63ad114] Mmap `@stdlib/Mmap`
  [ca575930] NetworkOptions v1.2.0 `@stdlib/NetworkOptions`
  [44cfe95a] Pkg v1.8.0 `@stdlib/Pkg`
  [de0858da] Printf `@stdlib/Printf`
  [3fa0cd96] REPL `@stdlib/REPL`
  [9a3f8284] Random `@stdlib/Random`
  [ea8e919c] SHA v0.7.0 `@stdlib/SHA`
  [9e88b42a] Serialization `@stdlib/Serialization`
  [6462fe0b] Sockets `@stdlib/Sockets`
  [2f01184e] SparseArrays `@stdlib/SparseArrays`
  [10745b16] Statistics `@stdlib/Statistics`
  [4607b0f0] SuiteSparse `@stdlib/SuiteSparse`
  [fa267f1f] TOML v1.0.0 `@stdlib/TOML`
  [a4e569a6] Tar v1.10.0 `@stdlib/Tar`
  [8dfed614] Test `@stdlib/Test`
  [cf7118a7] UUIDs `@stdlib/UUIDs`
  [4ec0a83e] Unicode `@stdlib/Unicode`
  [e66e0078] CompilerSupportLibraries_jll v0.5.2+0 `@stdlib/CompilerSupportLibraries_jll`
  [deac9b47] LibCURL_jll v7.84.0+0 `@stdlib/LibCURL_jll`
  [29816b5a] LibSSH2_jll v1.10.2+0 `@stdlib/LibSSH2_jll`
  [c8ffd9c3] MbedTLS_jll v2.28.0+0 `@stdlib/MbedTLS_jll`
  [14a3606d] MozillaCACerts_jll v2022.2.1 `@stdlib/MozillaCACerts_jll`
  [4536629a] OpenBLAS_jll v0.3.21+0 `@stdlib/OpenBLAS_jll`
  [05823500] OpenLibm_jll v0.8.1+0 `@stdlib/OpenLibm_jll`
  [bea87d4a] SuiteSparse_jll v5.10.1+0 `@stdlib/SuiteSparse_jll`
  [83775a58] Zlib_jll v1.2.12+3 `@stdlib/Zlib_jll`
  [8e850b90] libblastrampoline_jll v5.1.1+0 `@stdlib/libblastrampoline_jll`
  [8e850ede] nghttp2_jll v1.48.0+0 `@stdlib/nghttp2_jll`
  [3f19e933] p7zip_jll v17.4.0+0 `@stdlib/p7zip_jll`
        Info Packages marked with ⌃ and ⌅ have new versions available, but those with ⌅ are restricted by compatibility constraints from upgrading.
     Testing Running tests...
ERROR: LoadError: ArgumentError: Package Distributions [31c24e10-a181-5473-b8eb-7969acd0382f] is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.

Stacktrace:
 [1] _require(pkg::Base.PkgId, env::String)
   @ Base ./loading.jl:1390
 [2] _require_prelocked(uuidkey::Base.PkgId, env::String)
   @ Base ./loading.jl:1281
 [3] macro expansion
   @ ./loading.jl:1261 [inlined]
 [4] macro expansion
   @ ./lock.jl:267 [inlined]
 [5] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1224
 [6] include
   @ ./Base.jl:434 [inlined]
 [7] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
   @ Base ./loading.jl:1638
 [8] top-level scope
   @ stdin:1
in expression starting at /home/pkgeval/.julia/packages/Bootstrap/6veet/src/Bootstrap.jl:1
in expression starting at stdin:1
ERROR: LoadError: Failed to precompile Bootstrap [e28b5b4c-05e8-5b66-bc03-6f0c0a0a06e0] to /home/pkgeval/.julia/compiled/v1.9/Bootstrap/jl_ZDIgbE.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:35
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
    @ Base ./loading.jl:1789
  [3] compilecache
    @ ./loading.jl:1733 [inlined]
  [4] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1421
  [5] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1281
  [6] macro expansion
    @ ./loading.jl:1261 [inlined]
  [7] macro expansion
    @ ./lock.jl:267 [inlined]
  [8] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1224
  [9] include(fname::String)
    @ Base.MainInclude ./client.jl:478
 [10] top-level scope
    @ ~/.julia/packages/Bootstrap/6veet/test/runtests.jl:8
 [11] include(fname::String)
    @ Base.MainInclude ./client.jl:478
 [12] top-level scope
    @ none:6
in expression starting at /home/pkgeval/.julia/packages/Bootstrap/6veet/test/test-non-parametric.jl:1
in expression starting at /home/pkgeval/.julia/packages/Bootstrap/6veet/test/runtests.jl:6

It would be great if Julia/Pkg were robust against such corruption, reinstalling the affected package when needed. Alternatively, avoiding the corruption in the first place would be great as well. I think it was either caused by multiple testers writing to the package store at the same time (which would require locking), or a well-timed cancellation signal leaving the folder in a bad state (which would require the final operation to be atomic). I'm not familiar with Pkg so don't know if either of these is already the case.
@fredrikekre
Copy link
Member

Related: #1683, #1717.

@rfourquet
Copy link
Member

Another case where corruption happens is when the disk becomes full while downloading a package.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@maleadt @rfourquet @fredrikekre