-
-
Notifications
You must be signed in to change notification settings - Fork 30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to audio data stream to the speakers on node server ? #3
Comments
Hiya, The opus decoder is exist on npm, so the main problem is for decoding the But you can output the stream as a file and use var fs = require('fs');
// Event listener
io.on('connection', function(socket){
var chunks = [];
/* Presenter */
socket.on('bufferHeader', function(packet){
chunks.push(packet.data); // The header must be at first index
});
// Save the received buffer
socket.on('stream', function(packet){
if(chunks.length !== 0 && chunks.length <= 20){
chunks.push(packet[0]);
// Write to file after 2sec (If one chunk duration is 100ms)
if(chunks.length === 20){
fs.writeFile("test.webm", Buffer.concat(chunks), "binary", function(){
// Exit process after the write was completed
process.exit();
});
}
}
});
}); |
But if the browser support recording in MediaRecorder.isTypeSupported('audio/ogg;codecs="opus"') === true
MediaRecorder.isTypeSupported('audio/ogg;codecs="vorbis"') === true You can change the Presenter mimeType var presenter = new ScarletsMediaPresenter(...);
presenter.options.mimeType = 'audio/ogg;codecs="opus"'; And on the server, you need to
|
I'm doing a little experiment. It is the audio stream direct from the Microphone of the chrome of Android Phone (use SFMediaStream) to raspberrypi speaker (not browser, only nodejs). I understand the steps you talk on server about ogg decoder and opus decoder. But I'm not good at programming nodejs. Can you write a few lines of code that can work? Thanks. |
I have learned a lot on the Internet. |
Hmm, I'm not sure chrome can recording with // Event listener
io.on('connection', function(socket){
var decoder = null;
/* Presenter */
socket.on('bufferHeader', function(packet){
decoder = opusDecoder(packet.data);
});
// Broadcast the received buffer
socket.on('stream', function(packet){
if(decoder !== null)
decoder.write(packet[0]);
});
});
function opusDecoder(headerBuffer){
var opus = require('node-opus');
var ogg = require('ogg');
var stream = require('stream');
var speaker = require('speaker');
speaker = new speaker();
return {
write:function(chunk){
var decoder = new ogg.Decoder();
decoder.on('stream', function(stream){
var opusDecoder = new opus.Decoder();
opusDecoder.on('format', function(format){
if(!format.signed && format.bitDepth !== 16)
throw new Error('unexpected format: ' + JSON.stringify(format));
// Send the data to speaker
opusDecoder.pipe(speaker);
});
opusDecoder.on('error', console.error);
stream.pipe(opusDecoder);
});
var bufferStream = new stream.PassThrough();
bufferStream.end(Buffer.concat([headerBuffer, chunk]));
bufferStream.pipe(decoder);
}
}
} |
I have access link: https://kbumsik.io/opus-media-recorder/
|
Oh, i have just change the Presenter mimeType
Then Chrome show console error: DOMException: Failed to construct 'MediaRecorder': Failed to initialize native MediaRecorder the type provided (audio/ogg;codecs="opus") is not supported. I will explore this issue. Thanks for your support. |
Yeah, maybe initializing the polyfill before you starting the recording could work. |
Hello, |
@ashkmn I just updated the example, but sadly the opus decoder dependency on NodeJS was not able to decode the stream. And currently the example is default to write to a file instead to the speaker. I'm curious if there are a media player that able to play the file that still being written, or maybe I should concat every buffer start from the header so the opus decoder will not complain about data corrupted. But the speaker library doesn't have media seek so it will always play from the beginning. Well, maybe you can set an interval in NodeJS to rename the written file after some second and start writing to new file. So you can play the renamed file with a media player. It's a bit crazy tho on File I/O, but it's the only solution for now. |
Instead of using default browser's media recorder, I just using @tinamore is suggestion and now it's working with |
@StefansArya Hi ! Thanks for your work in this exemple. I try to test it but i'm stuck. With Chrome, it's work well with stream to file function, but with stream to server's speakers i got this : With Firefox, both stream to file and stream to server's speakers gave me the same error : Have you got any idea to make it work ? |
@katarpilar Hi, thanks for notify me. It seems the example wasn't work when switch into "Stream to speaker" after the page was loaded. I have fixed it, could you try on your browser? |
@StefansArya It work grate now thanks :D |
@katarpilar you're free to open new issue as long it isn't a duplicate with other open issue. But it's still related with this issue so it's OK to continue our discussion here.. Actually that "pop" also happen on my PC but currently I don't have solution about that 😅
I think you can reduce that "pop" by increasing the latency, but yeah.. that's not a best idea. My another alternative solution is using WebRTC because I also want to add it for this library in the past. |
@StefansArya Yea i noticed you commented some lines in speaker.js to decode opus chunk in real time and i tryed to test them but all i get is the node process segfaulting when it try to decode the stream :/ I still have some hope by using ogg instead of webm container but with chrome we need to use the OpusMediaRecorder, and when i try to change it i get "TypeError: Failed to execute 'fetch' on 'WorkerGlobalScope': Failed to parse URL from /OggOpusEncoder.wasm" I dont understand why, probably because i'm a big noob in JS ^^ but i tryed to load the wasm file as specified here https://github.com/kbumsik/opus-media-recorder , without success. |
Hi,
Your project is very interesting.
I wonder how to output the speaker on the server node?
Example:
Thanks
The text was updated successfully, but these errors were encountered: