Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Autotuner doesn't respond and fails #499

Open
chsasank opened this issue Feb 29, 2024 · 4 comments
Open

Autotuner doesn't respond and fails #499

chsasank opened this issue Feb 29, 2024 · 4 comments

Comments

@chsasank
Copy link

Hi I have been trying to autotune so that I get good perf on Intel Arc 770. However after building autotune and running it I see no progress and ultimate it fails as following:

./tune_nn 4096 4096 4096 4 strided
Device: Intel(R) Arc(TM) A770 Graphics
Native API failed. Native API returns: -5 (PI_ERROR_OUT_OF_RESOURCES) -5 (PI_ERROR_OUT_OF_RESOURCES)
Enqueue process failed. -59 (PI_ERROR_INVALID_OPERATION)
Enqueue process failed. -59 (PI_ERROR_INVALID_OPERATION)
Enqueue process failed. -59 (PI_ERROR_INVALID_OPERATION)
Enqueue process failed. -59 (PI_ERROR_INVALID_OPERATION)
Enqueue process failed. -59 (PI_ERROR_INVALID_OPERATION)
Enqueue process failed. -59 (PI_ERROR_INVALID_OPERATION)
Enqueue process failed. -59 (PI_ERROR_INVALID_OPERATION)
Enqueue process failed. -59 (PI_ERROR_INVALID_OPERATION)
Enqueue process failed. -59 (PI_ERROR_INVALID_OPERATION)
terminate called after throwing an instance of 'sycl::_V1::runtime_error'
  what():  Enqueue process failed. -59 (PI_ERROR_INVALID_OPERATION)
Aborted (core dumped)

I restarted tuning with smaller numbers: ./tune_nn 1024 1024 1024 4 strided. Unfortunately if fails for that too with the same error.

I wish the tuner showed some sort of progress or the results it is seeing on each kernel. Would've been more educational if not anything. Happy to send PRs that shows both progress and with optimized configurations if you can handhold me a bit :)

@muhammad-tanvir-1211
Copy link
Collaborator

Hello,
Thank you for expressing your interest in using the auto-tuner tool. The auto-tuner is able to run a diverse list of tiled GEMM configurations on the underlying hardware and give out information like average time spent per GEMM configuration or the average flops achieve per GEMM configuration for the same input matrix size.
We can definitely try and help you with this error and see if we can replicate the same issue on our end. Could you please share the cmake command you used to build the library? Thanks.

@chsasank
Copy link
Author

chsasank commented Mar 5, 2024

The instructions to build are detailed in #498 (the same one you commented recently on)

@s-Nick
Copy link
Collaborator

s-Nick commented Apr 11, 2024

Hello @chsasank ,
Sorry for taking so long. We confirm the issue with the auto-tuner and we will look in to it in the future.

@chsasank
Copy link
Author

Thanks! Looking forward to see this fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants