ASoC: SOF: Add support for on-demand DSP boot for LNL+ by ujfalusi · Pull Request #5427 · thesofproject/linux

ujfalusi · 2025-05-23T12:38:43Z

On system suspend / resume we always power up the DSP and boot the
firmware, which is not strictly needed as right after the firmware booted
up we power the DSP down again on suspend and we also power it down after
resume after some inactivity.

Out of caution, add a new platform descriptor flag to enable on-demand
DSP boot since this might not work without changes to platform code on
certain platforms.

With the on-demand dsp boot enabled we will not boot the DSP and firmware
up on system or rpm resume, just enable audio subsystem since audio IPs,
like HDA and SoundWire might be needed (codecs suspend/resume operation).
Only boot up the DSP during the first hw_params() call when the DSP is
really going to be needed.

In this way we can handle the audio related use cases:
normal audio use (rpm suspend/resume)
system suspend/resume without active audio
system suspend/resume with active audio
system suspend/resume without active audio, and audio start before the rpm
suspend timeout

Suspend resume is tested on LNL laptop with pm-graph/sleepgraph.py -rtcwake 15 -m freeze
Baseline (w/o active audio):
sof-audio-pci-intel-lnl @ 0000:00:1f.3 {sof-audio-pci-intel-lnl} (Total Suspend: 106.434 ms Total Resume: 157.955 ms)

With this PR, without active audio:
sof-audio-pci-intel-lnl @ 0000:00:1f.3 {sof-audio-pci-intel-lnl} (Total Suspend: 18.351 ms Total Resume: 31.468 ms)

With this PR, with active audio:
sof-audio-pci-intel-lnl @ 0000:00:1f.3 {sof-audio-pci-intel-lnl} (Total Suspend: 116.535 ms Total Resume: 32.763 ms)

ujfalusi · 2025-05-23T13:47:24Z

Changes since v1:

no need to check the on demand flag in hw_parms(), we have state check for that
sof-probe client now works with on-demand DSP boot
IPC message injector is not yet, neither compressed

ujfalusi · 2025-05-26T06:27:55Z

Changes since v2:

do a put module if the dsp boot returns with error in sof_client_core_module_get()

ujfalusi · 2025-05-26T10:46:07Z

Changes since v3:

Cover corner case users to handle on-demand DSP boot mode: compressed, debug, ipc3-dtrace, client drivers.
Ready for review.

sound/soc/sof/pcm.c

kv2019i

Code looks good, lot of helpful explanations in commit messages. There are a few sentences in commits had hard language to follow. No blockers on my side, but given the magnitude of change, I wonder if we should do a partial deployment of this change (made the same comment inline)?

sound/soc/sof/pm.c

sound/soc/sof/sof-client-ipc-flood-test.c

kv2019i · 2025-05-26T12:21:28Z

sound/soc/sof/intel/mtl.c

@@ -323,6 +323,9 @@ static int mtl_dsp_post_fw_run(struct snd_sof_dev *sdev)
 		}


The commit is a bit hard to follow " must be enabled since ... they are only enabled" ?

right, what it is trying to say is that the interrupts on MTL are enabled as in step 4 of mtl_dsp_cl_init(), which is part of the firmware booting, but we need interrupts to be enabled for SDW/HDA to work, and since we don't boot the firmware, we need to do it on post fw run time.

can we inline comment this need to enable IRQs so its easier to follow.

I will drop the MTL support for now.

sound/soc/sof/intel/pci-mtl.c

dbaluta · 2025-05-26T12:46:28Z

@lgirdwood out of curiosity I wonder why this booting the firmware at system boot was needed. Beside a some sort of self-check at boot.

ujfalusi · 2025-05-26T12:53:36Z

@lgirdwood out of curiosity I wonder why this booting the firmware at system boot was needed. Beside a some sort of self-check at boot.

for self check, to be sure that the fw has the right key, to grab information about the booted firmware configuration and we are also placing it to IMR for faster bootup on consequent power ons.

lgirdwood

Re BRA - this should only be used for active codec use cases. i.e. we are streaming audio.

@shumingfan @charleskeepax do we always need to load codec FW for jack detect use cases ?

We should only need to power up codec and load codec FW when we have real streaming use cases otherwise at system resume we usually power up codec, load codec FW then after a few seconds enter RTD3 and power audio codec and DSP off again. i.e. a wasted effort transition if audio is inactive.

sound/soc/sof/sof-client-ipc-kernel-injector.c

lgirdwood · 2025-05-26T16:56:42Z

sound/soc/sof/intel/mtl.c

@@ -323,6 +323,9 @@ static int mtl_dsp_post_fw_run(struct snd_sof_dev *sdev)
 		}


can we inline comment this need to enable IRQs so its easier to follow.

lyakh · 2025-05-27T06:40:02Z

sound/soc/sof/pm.c

-		dev_warn(sdev->dev,
-			 "warning: failed to init trace after resume %d\n",
-			 ret);
+		dev_warn(sdev->dev, "%s: failed to resume trace%d\n",


a space is missing

Oh, will fix.

lyakh · 2025-05-27T06:40:11Z

sound/soc/sof/pm.c

 		ret = tplg_ops->set_up_all_pipelines(sdev, false);
 		if (ret < 0) {
-			dev_err(sdev->dev, "Failed to restore pipeline after resume %d\n", ret);
+			dev_err(sdev->dev, "%s: failed to restore pipeline%d\n",


lyakh · 2025-05-27T07:11:58Z

sound/soc/sof/compress.c

+	 * Make sure that the DSP is booted up, which might not be the
+	 * case if the on-demand DSP boot is used
+	 */
+	ret = snd_sof_dsp_power_up(sdev, true);


Do I understand it correctly, that currently powering up the DSP is a part of the (runtime) PM flow? I.e. is performed on some .resume() paths? And this commit makes additional locations like this one, which previously relied on the PM to bring up the DSP, do it explicitly now? Wouldn't it be possible and a better approach to keep the call as a part of the PM flow but just add a check in .resume() whether the DSP has to be booted?

@lyakh, currently we are powering up the audio subsystem and booting up the DSP firmware, they are hard-linked.

With this PR we can detach the DSP firmware boot from the audio subsystem power up.

I have already reworked this API, I will have snd_sof_boot_dsp_firmware(sdev); only as a function, in pm.c it is:

if (sdev->pdata->desc->on_demand_dsp_boot) { /* Only change the fw_state to PREPARE but skip booting */ sof_set_fw_state(sdev, SOF_FW_BOOT_PREPARE); return 0; } return snd_sof_boot_dsp_firmware(sdev);

for the resume.

ujfalusi · 2025-05-27T08:56:51Z

Changes since v4:

Change the API from snd_sof_dsp_power_up(sdev, true); to snd_sof_boot_dsp_firmware(sdev);
Add protection for the DSP firmware boot to avoid possible races
Do not enable on-demand DSP boot for MTL (Only for LNL+)
Handle BRA also
Typos fixed

kv2019i

Looks great!

kv2019i · 2025-05-27T11:31:05Z

@ujfalusi One export is missing though in the new BRA patch:
ERROR: modpost: "snd_sof_boot_dsp_firmware" [sound/soc/sof/intel/snd-sof-intel-hda-sdw-bpt.ko] undefined!

…_library The initial library loading is happening during topology loading, which is already protected with pm_runtime_resume_and_get() via pcm.c The redundant rpm code can be dropped from sof_ipc4_load_library() Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>

…nted Test earlier for the existence of ext_volatile_get callback and if it is missing, skip the rpm calls to avoid needles DSP power on. No change in functionality, we just skip the DSP power on in the unlikely case when the ext_volatile _get is not supported and yet the topology adds such control. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>

On system suspend / resume we always power up the DSP and boot the firmware, which is not strictly needed as right after the firmware booted up we power the DSP down again on suspend and we also power it down after resume after some inactivity. Out of caution, add a new platform descriptor flag to enable on-demand DSP boot since this might not work without changes to platform code on certain platforms. With the on-demand dsp boot enabled we will not boot the DSP and firmware up on system or rpm resume, just enable audio subsystem since audio IPs, like HDA and SoundWire might be needed (codecs suspend/resume operation). Only boot up the DSP during the first hw_params() call when the DSP is really going to be needed. In this way we can handle the audio related use cases: normal audio use (rpm suspend/resume) system suspend/resume without active audio system suspend/resume with active audio system suspend/resume without active audio, and audio start before the rpm suspend timeout Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>

With the introduction of on-demand DSP boot the rpm status not necessary tells that the DSP firmware is booted up. Introduce the sof_client_boot_dsp() which can be used to make sure that the DSP is booted and it can handle IPCs. Update the client drivers to use the new function where it is expected that the DSP is booted up. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>

If on-demand DSP boot is used we need to make sure that the DSP is booted up - which might not be the case - since we need ChainDMA in normal, non DSPless mode for the BRA to work. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>

LNL can be used with on-demand DSP booting, set the flag to enable it. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>

PTL and WCL can be used with on-demand DSP booting, set the flag to enable it. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>

ujfalusi · 2025-05-27T11:37:49Z

Changes since v5:

add missed EXPORT_SYMBOL

kv2019i

Looks good.

ranj063 · 2025-05-28T16:24:24Z

@ujfalusi this PR is OK for now but I have a suggestion to make this a bit more elegant. How about we create a new auxiliary device for the DSP and use the runtime PM get/put functions to do the DSP boot or transition to D3 in those? That way instead of forcing the DSP to boot up, we can call the get/put to make sure that the DSP is booted when we need it in the params calls

lgirdwood · 2025-05-29T14:40:55Z

@ujfalusi this PR is OK for now but I have a suggestion to make this a bit more elegant. How about we create a new auxiliary device for the DSP and use the runtime PM get/put functions to do the DSP boot or transition to D3 in those? That way instead of forcing the DSP to boot up, we can call the get/put to make sure that the DSP is booted when we need it in the params calls

We do have time here, so if aligned and this request can be done easily then pls proceed.

ujfalusi · 2025-05-30T08:44:30Z

@ujfalusi this PR is OK for now but I have a suggestion to make this a bit more elegant. How about we create a new auxiliary device for the DSP and use the runtime PM get/put functions to do the DSP boot or transition to D3 in those? That way instead of forcing the DSP to boot up, we can call the get/put to make sure that the DSP is booted when we need it in the params calls

The problem is that it will look even more hackish as we have an aux device which does nothing, but to traverse back to use sdev and the audio device to boot up the DSP and then to turn it off.

What would be the clean solution is to have the audio device as it is now, create the aux devices for DSP and the interfaces, give them the resources t hat they need in order to be operational and use these devices instead of the audio device.
So, if the DSP is requested to be on then we would boot it up, if sdw is needed, we enable it, etc.

This also means that the topology will not be loaded using the main sdev, but the auxdev of the DSP, this would traverse down to machine drivers, sdw stack, and other places as the device is no longer the audio device, but an auxdevice.

I'm not sure if hacking an auxdevice to hook back to the main device is cleaner than an API which does this.
It would be useful of the auxdev is used for the component and rely on ASoC to keep what's needed to be powered, but that would mean that we boot the DSP on resume likely.

While I think it would be great to do this, I honestly think that it will take a better part of the rest of the year to get it somehow right and then we have not even considered the vendors using SOF...

lgirdwood · 2025-05-30T10:38:01Z

While I think it would be great to do this, I honestly think that it will take a better part of the rest of the year to get it somehow right and then we have not even considered the vendors using SOF...

ok, if this is big then lets keep backport effort low.

ujfalusi · 2025-05-30T10:41:08Z

While I think it would be great to do this, I honestly think that it will take a better part of the rest of the year to get it somehow right and then we have not even considered the vendors using SOF...

ok, if this is big then lets keep backport effort low.

I think it is rather big, but I still think it worth a look in the medium to long run.

ujfalusi · 2025-06-10T12:03:20Z

Let's merge this for now to give broader testing

ujfalusi force-pushed the peter/sof/pr/on-demand-dsp-boot-01 branch from 5e7ebe3 to 5349348 Compare May 23, 2025 13:45

ujfalusi force-pushed the peter/sof/pr/on-demand-dsp-boot-01 branch from 5349348 to f66c08e Compare May 26, 2025 06:26

ujfalusi force-pushed the peter/sof/pr/on-demand-dsp-boot-01 branch from f66c08e to 08107a4 Compare May 26, 2025 10:44

ujfalusi changed the title ~~[RFC] [DNM] ASoC: SOF: Add support for on-demand DSP boot for MTL+~~ ASoC: SOF: Add support for on-demand DSP boot for MTL+ May 26, 2025

ujfalusi marked this pull request as ready for review May 26, 2025 10:46

ujfalusi requested review from bardliao, dbaluta, kv2019i, lgirdwood, lyakh, plbossart and ranj063 as code owners May 26, 2025 10:46

bardliao reviewed May 26, 2025

View reviewed changes

sound/soc/sof/pcm.c Outdated Show resolved Hide resolved

kv2019i reviewed May 26, 2025

View reviewed changes

lgirdwood reviewed May 26, 2025

View reviewed changes

lyakh reviewed May 27, 2025

View reviewed changes

ujfalusi force-pushed the peter/sof/pr/on-demand-dsp-boot-01 branch from 08107a4 to 32dab67 Compare May 27, 2025 08:54

kv2019i previously approved these changes May 27, 2025

View reviewed changes

ujfalusi added 5 commits May 27, 2025 14:36

ujfalusi added 2 commits May 27, 2025 14:36

ASoC: SOF: Intel: pci-lnl: Set on_demand_dsp_boot for LNL

0e130d3

LNL can be used with on-demand DSP booting, set the flag to enable it. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>

ASoC: SOF: Intel: pci-ptl: Set on_demand_dsp_boot for PTL and WCL

e355d8a

PTL and WCL can be used with on-demand DSP booting, set the flag to enable it. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>

ujfalusi dismissed kv2019i’s stale review via e355d8a May 27, 2025 11:37

ujfalusi force-pushed the peter/sof/pr/on-demand-dsp-boot-01 branch from 32dab67 to e355d8a Compare May 27, 2025 11:37

ujfalusi changed the title ~~ASoC: SOF: Add support for on-demand DSP boot for MTL+~~ ASoC: SOF: Add support for on-demand DSP boot for LNL+ May 27, 2025

bardliao approved these changes May 27, 2025

View reviewed changes

kv2019i approved these changes May 27, 2025

View reviewed changes

lgirdwood approved these changes May 30, 2025

View reviewed changes

ujfalusi merged commit 80c0814 into thesofproject:topic/sof-dev Jun 10, 2025
8 of 16 checks passed

		@@ -323,6 +323,9 @@ static int mtl_dsp_post_fw_run(struct snd_sof_dev *sdev)
		}

Conversation

ujfalusi commented May 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ujfalusi commented May 23, 2025

Uh oh!

ujfalusi commented May 26, 2025

Uh oh!

ujfalusi commented May 26, 2025

Uh oh!

Uh oh!

kv2019i left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dbaluta commented May 26, 2025

Uh oh!

ujfalusi commented May 26, 2025

Uh oh!

lgirdwood left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ujfalusi commented May 27, 2025

Uh oh!

kv2019i left a comment

Choose a reason for hiding this comment

Uh oh!

kv2019i commented May 27, 2025

Uh oh!

ujfalusi commented May 27, 2025

Uh oh!

kv2019i left a comment

Choose a reason for hiding this comment

Uh oh!

ranj063 commented May 28, 2025

Uh oh!

lgirdwood commented May 29, 2025

Uh oh!

ujfalusi commented May 30, 2025

Uh oh!

lgirdwood commented May 30, 2025

Uh oh!

ujfalusi commented May 30, 2025

Uh oh!

ujfalusi commented Jun 10, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

ujfalusi commented May 23, 2025 •

edited

Loading