So many features flags...

[Johannesd3]

Why does librespot provide three different vorbis decoders, two dns-sd libraries, and backends that could simply be replaced by using a pipe (like pulseaudio using pacat)? Is it worth the effort?

[roderickvd]

With regards to the audio backends, answers may lie in the more general question "why would anyone use libraries over inter-process pipes?"

I'd say libraries have the potential of offering higher performance, better error handling and more granular control, at the expense of larger code size, higher complexity and maintenance requirements.

On the other hands, inter-process pipes offer a very simple interface, completely encapsulating backends. The downside is there can be hard-to-debug interfacing and timing issues, like the one you are working on I believe at #659.

In particular for the ALSA backend there is the capability of doing volume control in hardware by ways of ALSA's mixer. That's one important advantage over using aplay.

[herrernst]

Another important feature (for me) of using ALSA via library and not via pipe is that when the ALSA sink gets closed, it disables the SPDIF signal on my Raspberry PI and my active speakers go to standby after some idle time to save energy (and turn on again, when audio starts again). Also, when using direct hardware access, it frees the device and it can be used by other porcesses. Both of theses things don't work when using a pipe.

[sashahilton00]

To somewhat answer the question:

Lewton was added as a pure rust alternative to the libvorbis bindings.

The audio back ends are a bit of a mess. It is planned that someday librespot is separated properly into a library and a daemon, at which point all those back ends move to the daemon and the library just provides the PCM samples.

Not sure why there are two dns-sd implementations, likely for cross compatibility.

[Johannesd3]

Is it right that libmdns spawns its own async task which responds to incoming requests, while dns-sd only registers the service on a bonjour or avahi daemon which is running on the system? So they probably both have their raison d'etre.

I also looked up the difference between vorbis and tremor: tremor seems to use fixed-point arithmetics, so it works better on devices without FPU than libvorbis.

In #652 @leshow mentioned that features should be additive. Now since there's an AudioDecoder trait (#569), this should be easy for decoders (btw, why does librespot bother to expose thin wrappers around vorbis decoders which are only used by librespot-playback? Should we move it?).

There's no such abstraction for dns-sd (yet). It would be cool if libmdns had this abstraction, so you could use the daemon if possible and as fallback a tokio task.

[roderickvd]

Is it right that libmdns spawns its own async task which responds to incoming requests, while dns-sd only registers the service on a bonjour or avahi daemon which is running on the system? So they probably both have their raison d'etre.

That makes sense.

I also looked up the difference between vorbis and tremor: tremor seems to use fixed-point arithmetics, so it works better on devices without FPU than libvorbis.

Only recently I thought that none of the ARMv6 chips had floating point support, but then @kingosticks pointed out that all Raspberry Pi boards do have hard float support including the Pi Zero. So I am not sure which hardware we are actually targeting here in practice?

Besides doing fixed-point arithmetic, Tremor is said to be a very different decoder architecture. At some point more than ten years ago Xiph.org considered also making a floating point version of it that would eventually replace the "vanilla" Vorbis. In between the lines you could read that the Tremor architecture was deemed better. I don't believe this was pursued any further though.

In #652 @leshow mentioned that features should be additive. Now since there's an AudioDecoder trait (#569), this should be easy for decoders (btw, why does librespot bother to re-export thin wrappers around vorbis decoders which are only used by librespot-playback? Should we move it?).

Don't know if other software out there depends on librespot-audio for this purpose?

[Johannesd3]

Don't know if other software out there depends on librespot-audio for this purpose?

If so, they should rather use their favourite decoder directly. But I don't believe it's the case, at least not at crates.io.

[roderickvd]

@JasonLG1979 continuing from #660:

Debating the technical merits of the backends are really above my pay grade but from a novice's perspective I wonder how many people actually use all of these backends? Like do the dev's ever say "No" when someone wants to add one?

I'm tempted to say "exactly this".

Myself, I'm new to this project and having a lot of fun hacking away at the backends. Really, I do. But what you'd really want is to have some sort of ownership over the backends. Now it seems like someone scratches an itch, prepares a PR (great!) and then pretty much leaves the burden of maintainership with the core team. That's not very sustainable and especially not for specialty cases.

Now I'm really eager to get #660 going and who knows what the future holds. And by matter of principle I believe I should not cause regressions in what worked before. I'm even open to invest time in setting up JACK, PortAudio, GStreamer and what not just for the merits of cursory testing. Though personally, I'm an Alsa man and don't phanthom doing anything else for any number of years.

Or: split the backends into a Tier 1 "core backends" owned by the core team, and a Tier 2 "specialty backend" with a dedicated maintainer. No maintainer? Backend goes into disrepair, it might function, it might not, but don't keep it from pushing new developments.

I'm a pure Linux user, I don't use Windows or macOS at all on any of my computers. In my mind I'm not sure what the point of middle-manning the audio though gstreamer is when you could just send it straight to ALSA? The PulseAudio backend is about as redundant. PulseAudio is basically desktop system specific. How many people run librespot on a Linux desktop system when there's a desktop client? Even then why wouldn't you just use the ALSA backend when it's basically functionally identical.

Not saying I don't agree with you but there's always arguments to be made.

• GStreamer supports all sorts of plugins, equalizers, visualizations
• PulseAudio has networking support and per-application volume control
• and so on.

I think Rodio became standard exactly to do away with all the individual backends and upkeep. Yet as I said elsewhere, I don't like that automagic stuff because when it doesn't work (like F32 on Alsa hosts) you're pretty much lost. Same could be said for GStreamer.

TL;DR if you'd ask me, split backends into two tiers with "core, guaranteed, well-maintained" backends and another tier with "best-effort" backends. Stop doing CI on Tier 2.

[leshow]

Myself, I'm new to this project and having a lot of fun hacking away at the backends. Really, I do. But what you'd really want is to have some sort of ownership over the backends. Now it seems like someone scratches an itch, prepares a PR (great!) and then pretty much leaves the burden of maintainership with the core team. That's not very sustainable and especially not for specialty cases.

I'd be more inclined to invest time if there was a clear path forward so that I don't end up doing a bunch of work that might not get merged, would make things easier to divvy up also.

[JasonLG1979]

The questions become:

1. Is librespot simply trying to match features and functionality with libspotify or is it trying to do more than that?

There needs to be stated goals.

2. What is the lowest spec real target device?

Basically they need to pick a potato and base decisions like the floating point math thing based on that potato. Like is anyone actually running librespot on a lower spec device than a Pi Zero?

3. Do we need all these backends? How many people actually use them? Do they exist to scratch one person's itch? Are they the product of fire and forget PR's? Do they belong in the main repo or should the person who uses them just maintain a repo with their niche backend?

One of the stated goals I hear over and over again is that they want to decouple the core from playback but they keep accepting new backends? I have some very limited experience maintaining a couple projects and in my experience you got to be able to say "no".

[sashahilton00]

All these are valid points. Issues relating to the audio backends are the most common, and it's somewhat unsurprising given that a lot of them are, as you say, best effort.

Like do the dev's ever say "No" when someone wants to add one?

👀... Not that I can remember. Essentially it been a pretty laid back approach to allowing audio backends to be added, primarily for 2 reasons:

1. Whilst the long-awaited daemon has yet to be created, many people default to using librespot directly, and want to get the audio in their application of choice. To this end, several audio backends have been added.
2. All the code for the backends is constrained to the librespot-audio crate, so the potential 'damage' is limited to that crate for the most part. The recent introduction of Ogg passthrough is a bit of a fly in the ointment in the sense that it spans across multiple librespot crates.

I hear over and over again is that they want to decouple the core from playback but they keep accepting new backends? I have some very limited experience maintaining a couple projects and in my experience you got to be able to say "no".

This is ultimately what should happen. Ideally the daemon becomes the home for all of this audio processing, along with other features that could be considered 'bells and whistles'. Once this happens, we'll strip out all the backends from the core librespot library, and only support one officially, likely rodio.

The main problem, as with so many things, is time. Unfortunately this project isn't really used 'in production' for anything, and thus there's a limited amount of time that is spent on maintaining it. Added to this, to a certain extent we are perpetually playing catchup; Spotify is changing its APIs from Hermes to gRPC/HTTP, and there are numerous features (eg. retrieving files from CDN and session reconnection) that have been outstanding for a while, but require a larger restructuring of the project.

Is librespot simply trying to match features and functionality with libspotify or is it trying to do more than that?

I'm not sure this is the question. Librespot isn't trying to be a drop in replacement for libspotify, it merely occupies the same role in that it is a client library. All it is intended to do is provide a means to interface with Spotify, with the additional support for acting as a Spotify Connect receiver. The thing that would be 'trying to do more than that' would be the daemon, which is where additionl functionality would be added, such as an metadata/events server, multiple audio backends, multiroom synchronisation, etc.

[JasonLG1979]

it is definitely an important feature for many, who are using librespot directly as a Connect receiver.

Not that I disagree with anything else about it but librespot works just fine as a connect receiver without discovery. If you use the --disable-discovery flag and provide a username and password to librespot it basically turns librespot into a private connect receiver. Only you can see it and no one else since it dials home to spotify at startup but doesn't advertise itself on the network via mDNS.

[sashahilton00]

That's true enough, though one of the big upsides to connect is that it is essentially zero touch; anyone on the local wifi can just beam music to a nearby speaker. I'm not sure how many people would use it if they had to manually enter their credentials.

[JasonLG1979]

Well a use case would be in a shared wifi situation where you maybe don't want your neighbor blasting kpop randomly over your stereo,lol!!!

But really I would think that most people use librespot as a service and aren't manually entering in creds every time it runs. I use it as a system service via systemd with an environment file as a config file. I've got a crappy old fire 7 tablet running an ancient version of Android that does nothing but act as a remote control for spotify connect. If a guest wants to listen to whatever I just hand them the remote. There's really no need to make it discoverable. I also really don't know anyone else that actually pays for Spotify so even if it were discoverable they couldn't use it anyway.

[roderickvd]

Thanks for chiming in!

If we do move to Spotifyd being 'the daemon', then it would make sense to have a transition period for the backends to be moved from librespot to Spotifyd where desired. As for which ones would remain, ideally I would only like to see one included with librespot, purely to provide basic playback functionality. With this in mind, my preference is to do one of the following:

Perhaps going on a limb here, but you could even consider just keeping pipe around. When piping into the likes of aplay and paplay you would be 100% sure you're testing your sample output and not some backend implementation.

1. Drop rodio in favour of cpal, as we already handle the vorbis processing with lewton/libvorbis/tremor, and do not support any other formats currently, hence the additional utility provided by using rodio over cpal is minimal. This leaves us to handle processing and playback of audio.

I'm no fan of either of them as I think their code is highly convoluted. Though I understand your point you'd rather not deal with backends at all and instead have that black box approach.

2. Commit to rodio, on the basis that more formats are coming (eg. Spotify HiFi), and we're not interested in handling all the audio processing/playback logic ourselves. This will provide us with less flexibility, but if we take the view that Spotifyd exists to handle the flexibility side of things, and that librespot is vanilla playback only, then this tradeoff probably makes sense. We should also continue to provide the passthrough option in the core library, as it would be difficult to implement externally.

If the reason is wide and care-free codec support, then we should check if such extra codecs are within the Rodio project goals. It's entirely thinkable that Spotify HiFi will use some lossless codec like FLAC, which is currently not in Rodio -- but will it be?

It really is an optional feature. I don't even use it as my router is a POS and doesn't do mDNS worth a crap.

I think it could definitely be separated out into it's own crate, as it has external dependencies and not everyone uses it, but it is definitely an important feature for many, who are using librespot directly as a Connect receiver. It's not something that we should be looking to offload to Spotifyd.

I'm with @Johannesd3 here for separation of concerns. Isn't daemonizing a Connect receiver the point of spotifyd?

[Johannesd3]

It want to express myself more clearly: I suggested that the repository which contains librespot-core, librespot-audio and maybe-librespot-discovery should not support any kind of playback. The issues and pull requests of this repository would mostly relate to reverse engineering questions and necessary bug fixes. This would of course only pay out if you find someone else who wants to maintain that stuff that is now librespot-playback (and I suggested that Spotifyd's maintainers are one possibility). But it would cover the back of you and other experienced contributors e.g. to implement the new api.

I don't know whether it all makes sense. Maybe there could be a separate crate for now-librespot-playback which could be shared by ncspot and spotifyd and other projects. Maybe there's a way to create a neat abstraction over the connect api without relying on the playback crate. But that's certainly just one of many possible ways.

[roderickvd]

The more I'm working on this, the more I'm starting to feel you guys @Johannesd3 and @JasonLG1979.

I spent two nights to get PortAudio to work on #660 when I thought: is this working on dev at all? Guess what: it didn't, at least not on Alsa or macOS.

Also I miss the point of having PortAudio in the first place. The purpose of which is cross-platform, portable audio. Isn't that what Rodio (or actually: cpal) is intended for? And offered by GStreamer as well?

[JasonLG1979]

I think that a call should be made to users and authors of the backends in disrepair to either maintain/fix them or they'll be removed because they're unmaintained. Unmaintained code is harmful to the sustainably of a project. From a contributor's perspective it's off-putting because as you've (@roderickvd) figured out you end up fixing a bunch of other people's garbage to make sure your code actually works, and from a novice user's perspective it just looks horrible to advertise broken functionality.

[allquixotic]

It's reasonable enough to call for help for broken backends, so in that case, if the gst backend ever breaks, please ping me so I can jump in and help. I'd even fix the observed design issue with using a static buffer in it if I had the background to do it, but as it stands it might take me weeks or months of trial and error to learn the correct approach to really improve the state of the gst backend.

That said, compare and contrast this to the Linux kernel. People throw drivers into the kernel and then stop working on them all the time, and unless the hardware has no identifiable users, by and large the maintainers keep the drivers working as kernel ABIs and APIs change. I know we don't have the resources of the kernel project here in librespot, so that's a valid reason not to follow that model, but some effort should be made towards community stewardship.

In practice, if a backend has been broken in the dev branch (or in an important PR) to the point that it doesn't work at all for a long time and nobody noticed, that probably means no one is using it. I use the gstreamer backend often, and keep updated with dev, so if it breaks I will notice and intervene to help fix it. I just can't promise to make the code elegant, minimal, optimal and ideally designed when I'm not quite sure how to do what's being proposed. I'm generally much better at fixing broken things than improving a design. :P

[sashahilton00]

@allquixotic indeed the gstreamer backend works well. From a usability perspective, does it make a difference for you if you were to use Spotifyd over librespot (assuming gstreamer was implemented in the former)? I'm thinking that in the context of librespot being a library, it doesn't necessarily make sense to keep gstreamer as part of the core featureset.

[allquixotic]

@sashahilton00 For my use case, I'm using librespot as a program rather than a library, so if spotifyd were collecting PCM samples from librespot (using _lib_respot as a proper library) and then had its own gstreamer backend, that would be totally fine for me.

[roderickvd]

That said, compare and contrast this to the Linux kernel. People throw drivers into the kernel and then stop working on them all the time, and unless the hardware has no identifiable users, by and large the maintainers keep the drivers working as kernel ABIs and APIs change. I know we don't have the resources of the kernel project here in librespot, so that's a valid reason not to follow that model, but some effort should be made towards community stewardship.

That's what I did for #660: I worked on all backends so they compiled with the new API and added newly supported formats to boot. At least, from a code and "hey it compiles!" standpoint. Case in point being GStreamer, which I broke due to an incorrect string literal, even though it compiled just fine.

In practice, if a backend has been broken in the dev branch (or in an important PR) to the point that it doesn't work at all for a long time and nobody noticed, that probably means no one is using it. I use the gstreamer backend often, and keep updated with dev, so if it breaks I will notice and intervene to help fix it. I just can't promise to make the code elegant, minimal, optimal and ideally designed when I'm not quite sure how to do what's being proposed. I'm generally much better at fixing broken things than improving a design. :P

Chalk me up for Alsa.

[Johannesd3]

vorbis/tremor is done now. What about dns-sd? As mentioned on gitter, I found a way to use Windows 10s native service discovery api. Would it be good enough to throws libmdns out?

[roderickvd]

There weren't any further replies on Gitter, but like I said I think your approach is very nice. I say we should move towards your solution if you'd be willing to maintain it. I suspect there would be more applications interested in such a crate than just librespot!

Which brings me to the topic of release planning. I'm thinking along the following lines:

First, release 0.2.1 any day now based on 6881875 to fix that nagging default threshold bug. It doesn't concern us when launched from the command line, but is a bit of a pain for applications that use librespot as a lib.

Release 0.3.0 maybe end of June with as common theme: "breaking improvements, to librespot API in general and playback specifically", excluding the new spotify API. So this is everything in dev right now, could also include your mDNS proposal, and have room to squeeze a few extra's in.

On the playback side we could substitute Rodio with cpal fairly quickly. Finally I am toying around an idea to keep samples in f64 instead of converting from and into f32 all the time, perhaps counter-intuitively this should improve performance on all platforms. This is all the playback breakage I can foresee at this point.

Never say never but I don't know what. Other than adding multi-format support (#651) I can't think of doing other Big Things. I haven't investigated thoroughly yet but I don't think it would be breaking. It's not high priority until Spotify HiFi lands and the new API is up and running. We could do it under the flag of new-api to lay down the infrastructure. Or do it on dev to fix non-playing tracks. Anyway, we can see how it goes and then decide when and where to get it in.

Realistically all those changes in 0.3.0 will trigger some issue reports, so we should maintain a 0.3.x branch to fix bugs and limit ourselves to non-breaking enhancements. Meanwhile we continue work on the new Spotify API and target it for a 0.4.0 release.

Alternative is to take some more steps in between, release more often and each time with breaking changes. I don't know what the packages downstream would be happier with. I guess I'd rather swallow all the breakage at once then be bothered with it time and time again.

What do you think?

P.S. are we really following Semantic Versioning? If so then shouldn't we increase major instead of minor versions?

[Johannesd3]

On the playback side we could substitute Rodio with cpal fairly quickly.

Did you see my branch? But there were problems on Windows. Probably the output format wasn't supported, so rodio's resampling was necessary.

Major version zero (0.y.z) is for initial development. Anything MAY change at any time. The public API SHOULD NOT be considered stable.

... but it's common not to introduce breaking changes in patch versions.

[roderickvd]

On the playback side we could substitute Rodio with cpal fairly quickly.

Did you see my branch? But there were problems on Windows. Probably the output format wasn't supported, so rodio's resampling was necessary.

Yes I did! That really should be 99% of the work.

Windows issues are a bit of a thing, I don't have a native Windows machine I can compile on. My work laptop doesn't have SDD capacity to spare for Visual Studio and such.

I see you implemented both f32 (scarcely supported) and s16 (universally supported). From cursory reading however I think you are writing f32 samples regardless? (that's what packet.samples() returns)

Here you should probably call the converter. On your pre-dithering fork:

        let samples = packet.samples();
        match self.format {
            AudioFormat::F32 => {
                // work with samples in f32
            }
            AudioFormat::S16 => {
                let samples_s16: &[i16] = &convert::to_s16(samples);
                // work with samples_s16 in i16
            }
            _ => unreachable!(),
        };

Because f32 samples are normalized to [-1.0..1.0], just casting them into i16 will yield silence.

Major version zero (0.y.z) is for initial development. Anything MAY change at any time. The public API SHOULD NOT be considered stable.

... but it's common not to introduce breaking changes in patch versions.

Ah didn't know that.

[roderickvd]

Want me to fork your cpal branches and continue work?

[roderickvd]

@Johannesd3 I'll take the thumbs up on cpal as a yes do I'll get to it 😆

Anyone have an opinion on the release map I proposed #648 (reply in thread)? (also releasing 0.2.1 this weekend based on 6881875)