Multivariant regression?

[arturgower]

Hi there, nice package. Any plans/capabilities for multiple regression, several independent variables, or multivariant regression (more than one dependent variable) ?

[panlanfeng]

There is a multivariate regression function localconstant that allows multiple independent variables (or covariates, the X).
Right now I don't have plan to add function dealing with multiple response variables but I am open to any suggestions. And you are very welcome to open pull requests.

[arturgower]

Thanks Pan, great I'll test out the multiple independent variable capabilities. Afraid I can't help develop much at the moment as am new to nonparametric regression.

[arturgower]

Any tips how to adjust the bias/variance trade off for localconstant(x,y)? Also guessing that passing localconstant(x,y; eval=newx), will predict the labels for newx right?

Also ran into this:

julia>n=50
julia>x = 2pi*rand(n,2); y = sin(x[:,1]).*cos(x[:,2]) + 0.2*randn(n);
julia>localconstant(x,y)
ERROR: MethodError: no method matching bwlocalconstant(::Array{Float64,2}, ::Array{Float64,1}, ::Array{KernelEstimator.#gaussiankernel,1})
Closest candidates are:
  bwlocalconstant(::AbstractArray{T<:Real,2}, ::AbstractArray{T<:Real,1}) at /home/art-man/.julia/v0.5/KernelEstimator/src/bandwidth.jl:316
  bwlocalconstant(::AbstractArray{T<:Real,2}, ::AbstractArray{T<:Real,1}, ::Array{Function,1}) at /home/art-man/.julia/v0.5/KernelEstimator/src/bandwidth.jl:316
  bwlocalconstant(::AbstractArray{T<:Real,1}, ::AbstractArray{T<:Real,1}) at /home/art-man/.julia/v0.5/KernelEstimator/src/bandwidth.jl:214
  ...
 in localconstant(::Array{Float64,2}, ::Array{Float64,1}) at /home/art-man/.julia/v0.5/KernelEstimator/src/regression.jl:98

which I worked around by changing line 96 of regression.jl from
kernel::Array{Function, 1} = [gaussiankernel for i in 1:size(xdata)[2]]
to
kernel::Array{Function, 1} = Array{Function}([gaussiankernel for i in 1:size(xdata)[2]])

[panlanfeng]

localconstant will choose bandwidth via cross validation, which is the process of bias and variance trade off.

Yes the function will predict the y on newx.

Thanks for catching this bug. It is fixed in the master branch.

[UserQuestions]

Is there a plan to extend to multivariate local linear regression? My understanding is that such regressions have substantially better properties than local constant ones. I also think that there is a similar implementation in https://github.com/mcreel/Econometrics.jl

[panlanfeng]

Thank you for interested in the package. Local linear regression is much more complicated and computationally intensive but it is on the plan.
As for the Econometrics package, I briefly browsed their code and I highly suspect their implementation is not real nonparametric local linear regression.