Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

For a 0.15.10 release #469

Merged
merged 24 commits into from
Aug 22, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
56a3feb
add activate_registry_environment()
ablaom May 11, 2022
0f76851
re-organize registry src and test code
ablaom May 11, 2022
1e8cd9c
update doc-string for @update and README.md
ablaom May 11, 2022
4e278cd
fix scope issue with registry_project()
ablaom May 12, 2022
8590209
rm redundant code
ablaom May 13, 2022
4f4fabd
Merge pull request #462 from JuliaAI/load-fix
ablaom May 16, 2022
51f2a69
relax the type signature for `load_path` to include any named tuple
ablaom May 23, 2022
793c07a
refactor check_registry to make use of new pkg MLJTestIntegration.jl
ablaom May 23, 2022
f880634
Merge pull request #463 from JuliaAI/ecosystem-integration
ablaom May 23, 2022
ad93039
fix one hot encoder when first value in column is missing
olivierlabayle Aug 3, 2022
7832888
Merge pull request #468 from JuliaAI/fix_ohe_first_missing
ablaom Aug 3, 2022
74750f5
bump 0.15.10
ablaom Aug 4, 2022
9123a5f
add hack to allow ci pass
OkonSamuel Aug 22, 2022
a3cdfca
temporarily allow check_registry tests to run
OkonSamuel Aug 22, 2022
e92f1e3
Update check_registry.yml
OkonSamuel Aug 22, 2022
9f6e442
Update check_registry.yml
OkonSamuel Aug 22, 2022
e91933f
Update check_registry.yml
OkonSamuel Aug 22, 2022
43e3d7a
Update check_registry.yml
OkonSamuel Aug 22, 2022
d9113fd
Update check_registry.yml
OkonSamuel Aug 22, 2022
60204d1
Update check_registry.yml
OkonSamuel Aug 22, 2022
c1cdfca
rm temporary lines for testing hack
ablaom Aug 22, 2022
4cbf86f
update model registry
ablaom Aug 22, 2022
fc9c954
remove a space
ablaom Aug 22, 2022
adf82cb
Merge pull request #471 from JuliaAI/betaml-fixes
ablaom Aug 22, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 16 additions & 1 deletion .github/workflows/check_registry.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,11 @@ jobs:
arch:
- x64
env:
PYTHON: ""
PYTHON: Conda
# remove next line (and others marked below) when
# JuliaAI/MLJScikitLearnInterface.jl#42 properly resolved
LD_LIBRARY_PATH: /home/runner/.julia/conda/3/lib

steps:
- uses: actions/checkout@v2
- uses: julia-actions/setup-julia@v1
Expand All @@ -34,9 +38,20 @@ jobs:
${{ runner.os }}-test-${{ env.cache-name }}-
${{ runner.os }}-test-
${{ runner.os }}-
- uses: julia-actions/julia-buildpkg@v1
- run: julia -e 'using Pkg; Pkg.Registry.update()'
- run: julia -e 'using Pkg; Pkg.develop(Pkg.PackageSpec(path = pwd()))'
- run: julia -e 'using Pkg; Pkg.add("Test")'

# remove next eight lines (and one other marked above) when
# JuliaAI/MLJScikitLearnInterface.jl#42 gets properly resolved
- name: "Install Conda"
run: julia -e 'using Pkg; Pkg.add("Conda");'
- name: "Install Scikit-learn"
run: |
julia -e 'using Conda; Conda.add("scikit-learn");'\
cd $LD_LIBRARY_PATH #just to check that the path is valid

- run: julia -e 'using Pkg; Pkg.update()'
- run: julia -e 'using Pkg; Pkg.precompile()'
- run: julia -e 'using Pkg; Pkg.status()'
Expand Down
2 changes: 1 addition & 1 deletion Project.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name = "MLJModels"
uuid = "d491faf4-2d78-11e9-2867-c94bc002c0b7"
authors = ["Anthony D. Blaom <[email protected]>"]
version = "0.15.9"
version = "0.15.10"

[deps]
CategoricalArrays = "324d7699-5711-5eae-9e2f-1d82baa6b597"
Expand Down
33 changes: 3 additions & 30 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,34 +70,7 @@ MLJModels contains:

Generally model registration is performed by administrators. If you
have an interface you would like registered, open an issue
[here](https://github.com/alan-turing-institute/MLJ.jl/issues).
[here](https://github.com/alan-turing-institute/MLJ.jl/issues).

**Administrator instructions.** To register all the models in
GreatNewPackage with MLJ:

- In the dev branch of a clone of the dev branch of MLJModels, change
to the `/src/registry/` directory and, in the latest version of julia, activate the
environment specified by the Project.toml there, after checking the
[compat] conditions there are up to date. **Do not use**
`Revise`.

- Add `GreatNewPackage` to the environment.

- In some environment to which your MLJModels clone has been added
(using `Pkg.dev`) execute `using MLJModels; MLJModels.@update`. This updates
`src/registry/Metadata.toml` and `src/registry/Models.toml` (the
latter is generated for convenience and not used by MLJ).

- Quit your REPL session and make a trivial commit to your MLJModels
branch to force pre-compilation in a new julia session when you run
`using MLJModels`. (For technical reasons the registry is not loaded
in `__init__`()`, so without pre-compiliation the new ]registry is not
available.)

- Test that the interfaces load properly with
`MLJModels.check_registry()`. (CI will fail on dev -> master if
this test fails.)

- Push your changes to an appropriate branch of MLJModels to make
the updated metadata available to users of the next MLJModels tagged
release.
**Administrator instructions.** These are given in the
`MLJModels.@update` document string.
7 changes: 4 additions & 3 deletions src/MLJModels.jl
Original file line number Diff line number Diff line change
Expand Up @@ -74,14 +74,15 @@ const MODEL_TRAITS_IN_REGISTRY = model_traits_in_registry(INFO_GIVEN_HANDLE)
# model search and registry code:
include("model_search.jl")
include("loading.jl")
include("registry/src/info_dict.jl")
include("registry/src/Registry.jl")
include("registry/src/check_registry.jl")
import .Registry.@update
using .Registry

# load built-in models:
include("builtins/Constant.jl")
include("builtins/Transformers.jl")
include("builtins/ThresholdPredictors.jl")

# finalize:
include("init.jl")

end # module
2 changes: 1 addition & 1 deletion src/builtins/Transformers.jl
Original file line number Diff line number Diff line change
Expand Up @@ -860,7 +860,7 @@ function MMI.fit(transformer::OneHotEncoder, verbosity::Int, X)
if T <: allowed_scitypes && ftr in specified_features
ref_name_pairs_given_feature[ftr] = Pair{<:Unsigned,Symbol}[]
shift = transformer.drop_last ? 1 : 0
levels = classes(first(col))
levels = classes(col)
fitted_levels_given_feature[ftr] = levels
if verbosity > 0
@info "Spawning $(length(levels)-shift) sub-features "*
Expand Down
6 changes: 6 additions & 0 deletions src/init.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
function __init__()
project = open(joinpath(@__DIR__, "registry", "Project.toml")) do io
readlines(io)
end
global REGISTRY_PROJECT = Ref{Vector{String}}(project)
end
74 changes: 40 additions & 34 deletions src/loading.jl
Original file line number Diff line number Diff line change
Expand Up @@ -15,25 +15,9 @@ function _append!(program, ex, doprint::Bool, tick_early::Bool)
end

function _import(modl, api_pkg, pkg, doprint)
# can be removed once MLJModel #331 is resolved:
if pkg == :NearestNeighbors
doprint && print("import NearestNeighbors")
try
modl.eval(:(import MLJModels))
catch
try
modl.eval(:(import MLJ.MLJModels))
catch
error("Problem putting MLJModels into scope. ")
end
end
modl.eval(:(import NearestNeighbors))
doprint && println(" \u2714")
else
doprint && print("import $api_pkg")
modl.eval(:(import $api_pkg))
doprint && println(" \u2714")
end
doprint && print("import $api_pkg")
modl.eval(:(import $api_pkg))
doprint && println(" \u2714")
end

function _eval(modl, path::Union{Expr,Symbol})
Expand All @@ -44,19 +28,26 @@ end
## OVERLOADING load_path

"""
load_path(model::String, pkg=nothing)
load_path(model_name::String, pkg=nothing)

Return the load path for model type with name `model_name`, specifying
the algorithm=providing package name `pkg` to resolve name conflicts,
if necessary.

load_path(proxy::NamedTuple)

Return the load path for model type with name `model`, specifying the
package name `pkg` to resolve name conflicts if necessary.
Return the load path for the model whose name is `proxy.name` and whose
algorithm-providing package has name `proxy.package_name`. For example,
`proxy` could be any element of the vector returned by `models()`.

load_path(model)

Return the load path of a `model` instance or type. Usually requires
necessary model code to have been separately loaded. Supply a string
necessary model code to have been separately loaded. Supply strings
as above if code is not loaded.

"""
function MLJModelInterface.load_path(proxy::ModelProxy)
function MLJModelInterface.load_path(proxy::NamedTuple)
handle = (name=proxy.name, pkg=proxy.package_name)
return INFO_GIVEN_HANDLE[handle][:load_path]
end
Expand Down Expand Up @@ -213,15 +204,30 @@ function _load(modl, name_ex, kw_exs...; interactive=false)
end


## NO LONGER SUPPORTED
"""
MLJModels.load(name; pkg=nothing, add=false, verbosity=0, mod=Main)

Experimental method.
Currently private.

Loads model code into specified module `mod` at run time, as opposed
to `@load` which loads coad into calling module at time of invokation.

"""
function load(
name::String;
pkg::Union{String,Nothing}=nothing,
add::Bool=false,
verbosity::Integer=0,
mod=Main
)
ex = if isnothing(pkg)
:(@load $name add=$add verbosity=$verbosity)
else
:(@load $name pkg=$pkg add=$add verbosity=$verbosity)
end
mod.eval(ex)
end

_deperror() = error(
"The `load` function is no longer supported. "*
"Use the `@load` macro instead, as in "*
"`@load RandomForestRegressor pkg = DecisionTree`.\n"*
"For explicit importing, you can discover a model's "*
"full load path with the `load_path` function, as in "*
"`load_path(\"RandomForestRegressor\", pkg=\"DecisionTree\")`. )")

load(proxy::ModelProxy; kwargs...) = _deperror()
load(name::String; kwargs...) = _deperror()
#load(proxy, ...) = ...
Loading