Add CEF audio rerouting to audio_data signal

[pixaline]

Hello there,
I found myself wanting to route the audio produced by CEF in to Godot. I implemented some prototype code from skimming other open source projects. I intend this to be a starting point for something better, perhaps added comments, audio buffer, or a toggle to toggle it on or off. The code only plays mono audio for now.

You can use this in Godot with the following example code:

audioPlayer = AudioStreamPlayer3D.new()
audioPlayer.stream = AudioStreamGenerator.new()
audioPlayer.playing = true

var browser = cef.create_browser(...)
browser.connect("audio_data", self, "on_audio_data")

func on_audio_data(data : PoolVector2Array):
	var playback = audioPlayer.get_stream_playback()
	if playback.can_push_buffer(data.size()):
		playback.push_buffer(data)

[Lecrapouille]

please use "m_" (m_audioBuffer) for member variables.

[Lecrapouille]

Cannot be size_t and set to 0 instead of int ?

[Lecrapouille]

Please indent to follow my coding style

[Lecrapouille]

Cannot channels be size_t instead ?

[Lecrapouille]

Please use //! \brief instead of /// and move this inside the header file on the associated method

[Lecrapouille]

Are you sure of Vector2(data[0][i], data[0][i]) ? Should not be Vector2(data[i][0], data[i][1]) instead ?

[Lecrapouille]

Can you break the line before { to follow my coding style ?

[Lecrapouille]

I would prefer:
if ((data == nullptr) || (frames < 0) ||( audioChannelLayoutSize == 0))
because:

• frames is int but used as unsigned
• audioChannelLayoutSize shall not be 0 when no channel used ?

[Lecrapouille]

audioBuffer.resize(size_t(frames)); else the compiler probably complains of possible negative values

[Lecrapouille]

(int) is C cast, not C++, the compiler will complain.
Why not audioChannelLayoutSize = size_t(params.channel_layout);
and audioChannelLayoutSize seems not be used.

[Lecrapouille]

@parulina Thank you for your pull request! I'm not sure to understand what is the utility of rerouting the sound (for doing signal processing ?), but why not ! I made reviews. Feel free to modify the 2D demo https://github.com/Lecrapouille/gdcef/tree/master/addons/gdcef/demos/2D by adding your example. The 2D demo is made for showing the most features of this module, like a sandbox.

PS: I give a try: I hear no sound :( and I have an error inside your callback when halting the demo2D. In addition, the rerouting pipeline should be set only if desired by the user when creating a browser var browser = $CEF.create_browser(url, name, S.x, S.y, {"sound_rerouting":true}), else let the default sound pipeline like currently made, this will avoid forcing people to add a Stream node a callback for something they have freely without any additional line of code :)

[Lecrapouille]

This code will force using your audio callback. I do not like that since your code is still experimental and this force to add a AudioStreamPlayer3D node in all projects. I would return this if and only if godot::Dictionary browser_settings has a field {"audio_rerouting", STATE_ENABLED} else return nullptrto use the default sound pipeline.

See:

gdcef/addons/gdcef/gdcef/src/gdcef.hpp

Lines 243 to 264 in 4b0fac5

	// -------------------------------------------------------------------------
	//! \brief Create a browser view and store its instance inside the internal
	//! container. Return the browser identifier or return nullptr in case of failure.
	//! \param[in] url the page link (by copy needed by Godot).
	//! \param[in] name the browser name (by copy needed by Godot).
	//! \param[in] w the width dimension of the document.
	//! \param[in] h the height dimension of the document.
	//! \param[in] Return the address of the newly created browser (or nullptr
	//! in case of error).
	//! \param[in] browser_settings dictionary of Browser config with default values:
	//! - {"frame_rate", 30}
	//! - {"javascript", STATE_ENABLED}
	//! - {"javascript_close_windows", STATE_DISABLED}
	//! - {"javascript_access_clipboard", STATE_DISABLED}
	//! - {"javascript_dom_paste", STATE_DISABLED}
	//! - {"image_loading", STATE_ENABLED}
	//! - {"databases", STATE_ENABLED}
	//! - {"webgl", STATE_ENABLED}
	//! Wherer STATE_DISABLED / STATE_ENABLED == false / true
	// -------------------------------------------------------------------------
	GDBrowserView* createBrowser(godot::String const url, godot::String const name,
	int w, int h, godot::Dictionary browser_settings);

[pixaline]

Thank you for the detailed feedback, useful to know what parts exactly to improve. I agree about the optional parameter. I can get back with the added improvements in a bit and double check if the demo would work.

I'm not sure to understand what is the utility of rerouting the sound (for doing signal processing ?)

Similar to showing the CEF texture on a mesh in-game, this allows you to route the audio CEF produces to AudioStreamPlayer or AudioStreamPlayer3D, making the audio play in the game instead of from the subprocess, I give an example:
https://user-images.githubusercontent.com/6361957/236702464-669c51d6-ce41-45f0-a8da-660e548ad939.mp4

This is a game recording, normally this recording would exclude the played audio since CEF is a separate process. I route the CEF audio to my game's audio bus that also has spectrum analyzer, so that's how I can add visualizer at the bottom left as well as do subtle camera effects like screen shake or bloom depending on the lower frequencies like drums etc.

Routing the audio ingame also allows the user to change the volume in-game rather than having to adjust the volume of the subprocess (at least in Windows you have to right click audio device > volume mixer, find the cryptic "gdcefSubprocess.exe" with no icon, adjust its volume that it will also forget) making it more user friendly.

[Lecrapouille]

@pixaline Any news ?

[Lecrapouille]

@pixaline sorry I crapped the merged conflict, I do not how to use this poor Github UX. Commited manually