Code

separate prefill into a process#11787

Code

Merged

sgwhat merged 24 commits intointel-analytics:mainintel-analytics/ipex-llm:mainfrom yangw1234:prefillyangw1234/ipex-llm:prefillCopy head branch name to clipboardAug 19, 2024

+1,579-867

Commits

Commits on Aug 15, 2024

• seperate prefill into a process

  yangw1234

  committed
  8f749f7

• using model.share_memory()

  yangw1234

  committed
  7d14013

• might work

  yangw1234

  committed
  61e5848

• worked

  yangw1234

  committed
  dcd6c73

• use long prompt

  yangw1234

  committed
  906ce53

• refactor

  yangw1234

  committed
  0d0a2d6

• cleanup

  yangw1234

  committed
  b538ed6

Commits on Aug 16, 2024

• fix bug

  yangw1234

  committed
  ce01fdf

• clean up

  yangw1234

  committed
  bfe8e85

• Merge branch 'prefill' of https://github.com/yangw1234/ipex-llm into prefill

  yangw1234

  committed
  c454003

• changable inter and intra process stages

  yangw1234

  committed
  15d82a2

• small fix

  ivy-lv11

  committed
  79e4ebf

Commits on Aug 17, 2024

• refactor

  yangw1234

  committed
  cb3f6a8

• fix style

  yangw1234

  committed
  2b9fcec

• fix style

  yangw1234

  committed
  f9bc5db

• fix style

  yangw1234

  committed
  9c358db

• fix style

  yangw1234

  committed
  a639eeb

• add max output len

  yangw1234

  committed
  f2bafa8

• fix style

  yangw1234

  committed
  1f510f7

• fix style

  yangw1234

  committed
  26b8496

• fix bug

  yangw1234

  committed
  6287452

Commits on Aug 19, 2024

• fix npu_model changes that may cause generate down

  sgwhat

  committed
  f6dce1a

• fix npu_model generate import error

  sgwhat

  committed
  4a874f8

• fix generare forward error

  sgwhat

  committed
  0818a41