Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add indexes methods #329

Closed
wants to merge 2 commits into from
Closed

Add indexes methods #329

wants to merge 2 commits into from

Conversation

rikhuijzer
Copy link
Contributor

@rikhuijzer rikhuijzer commented Feb 20, 2021

I was trying to use CategoricalArrays with Turing.jl. Surprisingly, it worked after I added some CategoricalArrays specialized type signatures for Base.checkindex and Base.to_index. According to the docstrings of both, these extensions are allowed, see ?Base.checkindex and ?Base.to_index.

With this PR, I suggest to add these specialized type signatures.

By the way, this PR is slightly related to #296, which is about a specialization for InvertedIndices.

@nalimilan
Copy link
Member

Thanks, but I don't think that's right. CategoricalValue is not supposed to be used as a numeric index. In particular, CategoricalValue{Int} would be ambiguous as it's not clear whether you'd like to use the reference code corresponding to the level, or the wrapped numeric value.

Can you develop what's the problem you get with Turing.jl?

@rikhuijzer
Copy link
Contributor Author

Fair enough; good point.

Can you develop what's the problem you get with Turing.jl?

I'm afraid that I don't understand this sentence. You mean to show the exact error here or to see whether the problem can be fixed at Turing.jl?

@nalimilan
Copy link
Member

I guess both, but seeing the error would probably help.

@rikhuijzer
Copy link
Contributor Author

Thanks, but I don't think that's right. CategoricalValue is not supposed to be used as a numeric index. In particular, CategoricalValue{Int} would be ambiguous as it's not clear whether you'd like to use the reference code corresponding to the level, or the wrapped numeric value.

On second thought, isn't this PR a more specific implementation of the existing design? Even though numeric indexing isn't supposed to be used, it is implemented:

julia> using CategoricalArrays

julia> v = categorical([1, 1, 2])
3-element CategoricalArray{Int64,1,UInt32}:
 1
 1
 2

julia> v[3]
CategoricalValue{Int64, UInt32} 2

I guess both, but seeing the error would probably help.

When I use CategoricalArrays for df.culture in this model, the error is as follows:

Sampling 100%|████████████████████████████████████████| Time: 0:00:01
ERROR: LoadError: ArgumentError: unable to check bounds for indices of type CategoricalValue{String, UInt32}
Stacktrace:
  [1] checkindex(#unused#::Type{Bool}, inds::Base.OneTo{Int64}, i::CategoricalValue{String, UInt32})
    @ Base ./abstractarray.jl:665
  [2] checkindex
    @ ./abstractarray.jl:680 [inlined]
  [3] checkbounds
    @ ./abstractarray.jl:595 [inlined]
  [4] checkbounds
    @ ./abstractarray.jl:610 [inlined]
  [5] _getindex
    @ ./multidimensional.jl:831 [inlined]
  [6] getindex
    @ ./abstractarray.jl:1164 [inlined]
  [7] #1
    @ ~/git/TuringModels.jl/scripts/spatial-autocorrelation-oceanic.jl:49 [inlined]
  [8] (::var"#1#2")(_rng::Random._GLOBAL_RNG, _model::DynamicPPL.Model{var"#1#2", (:Dmat, :society, :logpop, :total_tools), (), (), Tuple{Matrix{Float64}, CategoricalVector{String, UInt32, String, CategoricalValue{String, UInt32}, Union{}}, Vector{Float64}, Vector{Int64}}, Tuple{}}, _varinfo::DynamicPPL.ThreadSafeVarInfo{DynamicPPL.UntypedVarInfo{DynamicPPL.Metadata{Dict{DynamicPPL.VarName, Int64}, Vector{Distribution}, Vector{DynamicPPL.VarName}, Vector{Real}, Vector{Set{DynamicPPL.Selector}}}, Float64}, Vector{Base.RefValue{Float64}}}, _sampler::DynamicPPL.SampleFromUniform, _context::DynamicPPL.DefaultContext, Dmat::Matrix{Float64}, society::CategoricalVector{String, UInt32, String, CategoricalValue{String, UInt32}, Union{}}, logpop::Vector{Float64}, total_tools::Vector{Int64})
    @ Main ./none:0
  [9] macro expansion
    @ ~/.julia/packages/DynamicPPL/wf0dU/src/model.jl:0 [inlined]
 [10] _evaluate
    @ ~/.julia/packages/DynamicPPL/wf0dU/src/model.jl:154 [inlined]
...

After a method with a specialized type on CategoricalArrays for Base.checkindex, the error becomes:

ERROR: LoadError: ArgumentError: invalid index: CategoricalValue{String, UInt32} "Malekula" of type CategoricalValue{String, UInt32}
Stacktrace:
  [1] to_index(i::CategoricalValue{String, UInt32})
    @ Base ./indices.jl:297
  [2] to_index(A::Vector{Float64}, i::CategoricalValue{String, UInt32})
    @ Base ./indices.jl:274

@rikhuijzer rikhuijzer closed this Feb 27, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants