Revise failure with KernelAbstractions

[leios]

I have been playing around with the following code:

module revise_tests
    export vadd

    using KernelAbstractions, CUDA
    if has_cuda_gpu()
        CUDA.allowscalar(false)
    end

    @kernel function vadd_kernel(c, a, b)

        cI = @index(Global)

        c[cI] = a[cI] + b[cI]
    end

    function vadd(c, a, b)
        if isa(c, Array)
            kernel! = vadd_kernel(CPU(), 8) 
        else
            kernel! = vadd_kernel(CUDADevice(),256)
        end
        kernel!(c, a, b, ndrange=size(c)) 
    end

end # module

If I do

julia> a = ones(100)
julia> c = zeros(100)
julia> revise_tests.vadd(c, a, a)
KernelAbstractions.CPUEvent(Task (done) @0x00007f673bfd2710)

c is correctly a set of 100 2s. If I modify the line above such that c[cI] = a[cI] + b[cI] is now c[cI] = a[cI] + 2*b[cI] (or somethign similar, I get the following error:

julia> revise_tests.vadd(c, a, a)
ERROR: MethodError: no method matching vadd_kernel(::KernelAbstractions.CPU, ::Int64)
Stacktrace:
 [1] vadd(::Array{Float64,1}, ::Array{Float64,1}, ::Array{Float64,1}) at /home/leios/projects/revise_tests/src/revise_tests.jl:18
 [2] top-level scope at REPL[7]:1
 [3] run_backend(::REPL.REPLBackend) at /home/leios/.julia/packages/Revise/BqeJF/src/Revise.jl:1184
 [4] top-level scope at REPL[2]:0

As a note, changing the kernel names seems to be fine. If we %s/vadd_kernel/check/g, Revise does know the code is changed.

I don't mind poking around a bit, but do not really know where to look.

[vchuravy]

So we emit the kernels in

KernelAbstractions.jl/src/macros.jl

Lines 51 to 63 in 4ab11f2

	constructors = quote
	if $(name isa Symbol ? :(!@isdefined($name)) : true)
	Core.@__doc__ $name(dev::$Device) = $name(dev, $DynamicSize(), $DynamicSize())
	$name(dev::$Device, size) = $name(dev, $StaticSize(size), $DynamicSize())
	$name(dev::$Device, size, range) = $name(dev, $StaticSize(size), $StaticSize(range))
	function $name(::Device, ::S, ::NDRange) where {Device<:$CPU, S<:$_Size, NDRange<:$_Size}
	return $Kernel{Device, S, NDRange, typeof($cpu_name)}($cpu_name)
	end
	function $name(::Device, ::S, ::NDRange) where {Device<:$GPU, S<:$_Size, NDRange<:$_Size}
	return $Kernel{Device, S, NDRange, typeof($gpu_name)}($gpu_name)
	end
	end
	end

What I suspect is happening is that Revise will delete a function, but the macro will not recreate it, because we check if the name has been defined before. I added that 2896ef7 in order to allow two kernels to share the name.

@timholy any ideas how we can make macros defining functions compatible with revise?

[timholy]

I'm not sure I fully understand, but let me try: the macro expands to new definitions that run inside an isdefined check, and as a consequence it doesn't do anything if it re-evaluates. I'm not certain of the best way forward, but I'd consider maybe adding a hasmethod check to this conditional. That way, if Revise deletes the method then it should know that it needs to re-execute the block.

[vchuravy]

Ah that makes sense, the isdefined checks were for ensuring that one can overload `@kernel` methods without getting redefined methods warning. We can change that to `hasmethod` for Revise compatibility. @leios do you want to give that a try?

…

On Thu, Jul 30, 2020, 23:45 Tim Holy ***@***.***> wrote: I'm not sure I fully understand, but let me try: the macro expands to new definitions that run inside an isdefined check, and as a consequence it doesn't do anything if it re-evaluates. I'm not certain of the best way forward, but I'd consider maybe adding a hasmethod check to this conditional <https://github.com/JuliaGPU/KernelAbstractions.jl/blob/4ab11f29b615e72b5ec2112935593fb56309633a/src/macros.jl#L52>. That way, if Revise deletes the method then it should know that it needs to re-execute the block. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#111 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABDO2RZAN2JQRCMTMWEBLDR6HSWLANCNFSM4OWVTJHA> .

[leios]

I am not really sure what happened between the opening of this issue and now, but I am unable to replicate this error with the provided code. Revise seems to work (at least in KA#master). For the record, I tried simple modifications of the provided module with dispatching on both the CPU and GPU.

For now, I will close this issue, but may re-open it in the future if the need arises.

[agdestein]

I run into this issue quite often with CUDABackend.

1. I define a kernel @kernel function dissipation!(...) ... end in a module tracked by Revise, and it runs fine with dissipation!(CUDABackend(), 64)(args...). In the REPL it shows

julia> dissipation!
dissipation! (generic function with 4 methods)

2. When then change the content of the kernel and run it again, I get the error

ERROR: MethodError: no method matching dissipation!(::CUDABackend, ::Int64)
The function `dissipation!` exists, but no method is defined for this combination of argument types.

The kernel now has zero methods:

julia> dissipation!
dissipation! (generic function with 0 methods)

A quick solution is to rename it by replacing all occurences with newname!, but that becomes tedious in larger codebases and frequent changes.