Fix nightly spr & core perf test

[liu-shaojun]

Description

1. Why the change?

This submission install pydantic for spr & core perf test to fix nightly perf test failure:
https://github.com/intel-analytics/ipex-llm/actions/runs/9700292272/job/26771341907
https://github.com/intel-analytics/ipex-llm/actions/runs/9700292272/job/26771342167

[plusbang]

It seems that import error of pydantic exists when install [all] option, maybe we also need to fix here or add pydantic in setup.py.

[Oscilloscope98]

I don't think pydantic is a required option for cpu, as it is required for pp serving. We may update here :)