Architecture-specific configuration

[AntonOresten]

See https://docs.nvidia.com/cuda/cutile-python/performance.html

x-ref #111 (comment)

iirc, the optimization hints (#25, e.g. latency and allow_tma) should also support this but I don't know if it's being used anywhere.

[AntonOresten]

Maybe a ByTarget type with a ByTarget(; default, arch_value...) constructor? Could be cheap to construct by wrapping a NamedTuple, then we could convert sm_arch::String to Symbol.

Maybe there's a way to create it once and just cache it.

[maleadt]

Sounds like something we should support. We already shard the cache on these, and I think we can keep this:

cuTile.jl/ext/CUDAExt.jl

Lines 198 to 201 in 44c4823

	# Create cache view with compilation options as sharding keys
	opts = (sm_arch=sm_arch, opt_level=opt_level, num_ctas=num_ctas, occupancy=occupancy,
	bytecode_version=bytecode_version)
	cache = CacheView{CuTileResults}((:cuTile, opts), world)

We'll have to introduce something like ByTarget to annotate kernels with, and at launch time resolve that given the current active device to a set of optimization flags to construct the cache view.

I guess the real problem is where we write this down? Would be trivial to pass to the launch API, but it's cleaner to attach to the definition like Python does... Maybe we can abuse Expr(:meta) or define some other intrinsic that survives until code generation such that we can do something a la

function kernel(...)
  @ByTarget(...)
  ...
end