Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NPU] Support non-const parameter for decoder layers when keep_ir=True #12789

Merged
merged 6 commits into from
Feb 8, 2025

Conversation

rnwang04
Copy link
Contributor

@rnwang04 rnwang04 commented Feb 7, 2025

Description

1. Why the change?

2. User API changes

  • For decoder layers, we default make layernorm & bias as const, but for now, we can make them as input by passing keep_ir=True or set IPEX_LLM_NPU_CONST_PARAMETER=0

3. Summary of the change

  • Update parameter name from layernorm_const to const_parameter
  • Set const_parameter to False when passing keep_ir=True or set IPEX_LLM_NPU_CONST_PARAMETER=0
  • Make layernorm & bias as input when const_parameter is True

4. How to test?

  • N/A
  • Unit test: Please manually trigger the PR Validation here by inputting the PR number (e.g., 1234). And paste your action link here once it has been successfully finished.

@rnwang04 rnwang04 marked this pull request as draft February 7, 2025 09:49
@rnwang04 rnwang04 changed the title [NPU] Support layernorm=False for decoder layers when keep_ir=True [NPU] Support const parameter for decoder layers when keep_ir=True Feb 7, 2025
@rnwang04 rnwang04 force-pushed the ln_const branch 2 times, most recently from 27eadad to e26dc9a Compare February 7, 2025 10:00
@rnwang04 rnwang04 marked this pull request as ready for review February 7, 2025 10:00
@rnwang04 rnwang04 requested a review from jason-dai February 7, 2025 10:05
@rnwang04 rnwang04 changed the title [NPU] Support const parameter for decoder layers when keep_ir=True [NPU] Support non-const parameter for decoder layers when keep_ir=True Feb 7, 2025
@rnwang04 rnwang04 merged commit e90a9ad into intel:main Feb 8, 2025
1 check passed
@rnwang04 rnwang04 deleted the ln_const branch February 8, 2025 01:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants