Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs: Create app review guide #3391

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions docs/developing/app-review-guide.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
Apps submitted to the [Sandstorm App Market](https://apps.sandstorm.io/) go
through an app review process. This page serves to both inform developers
and the community about the process, and serve as a guide to app reviewers.

## New app approval process

When an app is submitted to the app market, it becomes immediately available
on the [experimental market](https://apps.sandstorm.io/?experimental=true) for
testing and evaluation. App reviewers are notified that a new package is in
the queue.

During app review, an app reviewer will email the developer at the contact
email listed in the package metadata, introducing themselves and covering the
review process. The reviewer will provide feedback and advice on the submitted
app, which may included both approval-blocking issues and suggested changes or
fixes. The reviewer will also make sure the metadata displays correctly in the
market and appears to accurately depict the submitted app, and if applicable,
may look at the source repository with an eye for Sandstorm-specific changes
to the code.

The app developer can then incorporate that feedback and resubmit, or if none
of the feedback was approval-blocking, request the app be approved as already
submitted. When the app reviewer approves the package, they will notify you as
the App Market does not contact developers directly.

## App update approval process

Generally speaking, app review for app updates moves significantly faster. An
app reviewer should test both new grains and grain updates of the app, with a
focus on ensuring users will not lose data in existing grains when they get the
update. If the app is open source, the app reviewer should verify that the update
was pushed to the source repository as well prior to approval.

The reviewer may look at the changelog and commit history to narrow down
what new features or changes should be tested, and may reach out to the
developer with suggestions or feedback. However, in many cases, the app reviewer
will simply approve the update after verifying its functionality and notify the
developer that it has been approved.

## Approval-blocking issues

The Sandstorm App Market is a fairly permissive environment. While you may
receive feedback about performance, functionality, or usability, we largely
require only that your app works and that it does not lose data.

We will reject an app which crashes immediately and does not function as
advertised, for example. This can often occur when the package is missing files
that were present in the dev environment and wasn't adequately tested prior to
submission.

The latter concern often occurs when a pure client app is submitted up that
implements localStorage as its storage medium, which Sandstorm does not effectively
support. This can often be tested by storing some information in a grain, allowing
Sandstorm to shut down that grain, and then opening it again and seeing if things
persisted that an ordinary user would expect to persist between sessions.

Finally, the App Market may reject apps which present security or privacy issues
that fall outside of what a Sandstorm user would expect when using the app. This
can mean it requests overly-broad permissions when making Powerbox requests or
uses client-side loading to load scripts which share unexpected information outside
of the Sandstorm sandbox. For example, we may block approval on an app which attempts
to load an analytics script from an outside server, or does not make it suitably
clear to a user why it needs certain outside network access prior to requesting it.

**Note:** Client-side loading is a known gap in Sandstorm's security model that we
intend to close. Not only is misuse of it a security or privacy issue, it's use may
ocdtrekkie marked this conversation as resolved.
Show resolved Hide resolved
lead to apps that break when Sandstorm closes this gap.

## Commercial or proprietary apps
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think what to put here is going to be a larger discussion. some thoughts below, but maybe for a first pass we should cut this and merge the uncontroversial bits.


I think there are a couple things we need to sort out before we're really ready to accept proprietary apps:

  • Copyleft compliance. These apps are almost always going to be bundled with GPL and LGPL binaries, which require both (1) distribution of source code and (2) the ability to swap in a modified version of the code.

    We're probably not really compliant with (1) right now even; technically we ought to be shipping the source for all of the distro packages included in the .spk. I'd like to address that at some point, but the docker world is similarly sloppy about this, and as long as we're complying with the "spirit" (it's easy enough for users to get the source, even if we're not mirroring it), I don't think we need to stop the presses. But we should work towards doing that by the book.

    But when we start throwing proprietary code in the mix that makes me nervous, because then if we're not doing this right we're really running afoul of the spirit of the license and are more likely to run into trouble. Furthermore the tooling doesn't really make (2) easy except by re-building the app from source, so we need a clear story re: how a user is supposed to go about doing that if they don't have the source.

  • Licensing policy. If we're allowing "proprietary apps" we probably still need to set some bounds on what kinds of licenses are acceptable. At a minimum the license has to be consistent with the way the app market works and our other policies. I'd really want some lawyers' eyes on this...

Additionally, I really don't like the idea of apps that you can download from the app market that then immediately say "enter license key;" I think if something is not going to work out of the box (for whatever reason) it really needs to be clearly delineated. I also don't want to spend precious developer resources making it easier to build apps with DRM.
I'd prefer most apps (regardless of license) don't do that in the first place, and I think if we're going to spend brain cycles on it at all someone who actually wants it should be paying us. My inclination in the meantime is to just
make it clear that we don't really have a great story there, feel free to come talk to us if you want help figuring stuff out.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We accepted draw.io when it was proprietary, so while we don't have one now, I would say "we already started" accepting proprietary apps.

Copyleft is interesting because in addition to your point, it's very plausible a given Sandstorm app package might have dependencies carrying a scattering of different licenses. I think we've had one or two questions about how to mark a given app's license if it might depend on code with other licenses. I think we (and other containerized platforms) have to look a layer deeper for compliance: If you open up the SPK, the proprietary app is using the open source bits that you can already easily get elsewhere. As long as the proprietary app isn't making closed source modifications to those open source parts, they should be okay.

With licensing policy, I feel like our solution of "recognize acceptable open source licenses, you're on your own outside of that" is fine: We're not responsible for enforcing someone else's license. Things like site licenses should have no problem with allowing "anyone on this Sandstorm server" even to use an app. But most sophisticated licensing schemes don't actually have strong enforcement in the code: They rely on the legal issue your business would be in if found outside of compliance. For instance, Microsoft CALs (required to cover all users of the software) aren't even entered into the software, you just have to have them.

Regarding the "enter license key", the recent Apple App Store/Hey debacle is a good example, but many apps on app stores require login to work, such as Netflix and the like. I think a good expectation here might be that your paid app should ideally offer some minimal/demo functionality if possible, with the ability to unlock full functionality.

I would agree that we probably shouldn't be heavily investing in supporting DRM, in-app purchases, etc., which is why I feel it's probably worthwhile to just go ahead and say "you're able to do this, but you have to code it yourself". And that last sentence or so was definitely suggest people ask us before working on this to see if it'd even fly with the community.

I am alright with cutting this section and/or cutting it back to just the part about talking with the community before trying it, but this PR was in part to provide a good place to have some of this conversation more formally than the occasional chats, so I don't want to do so immediately.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(As an aside, sandstorm-io/vagrant-spk#249 may be particularly helpful in letting people identify the open source components and the exact source of such included in a given SPK.)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding the ability to swap in a modified version of the code, there's nothing in an SPK preventing you from unpacking it, swapping out some files, and repacking it, other than that you'd need to swap the appId, right? To me, that would mean we're meeting the requirement, as long as the proprietary app isn't modifying the open source components pulled in by packing the SPK.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose a process of unpacking the spk, pulling out the relevant bits, and repacking might be acceptable, but ideally someone shouldn't have to basically reverse-engineer the contents of the package to figure out how to swap something like that out. Lack of instructions may run afoul of GPLv3's rules around "Installation Information," though I'm uncertain.

Maybe what we could recommend is to basically publish a repo with build scripts (vagrant-spk and such) but that may pull in binaries of the app proper (or ask you to supply them) and bundle them with stuff from the distro.

My feeling is that the broad-strokes version of this section should be something like "We're not opposed to proprietary apps, but bear in mind they haven't been our focus, you'll probably have to roll some stuff yourself, and here some issues to think about (including some of the stuff around license compliance). Come talk to us if you're unsure about something." I'd be in favor of cutting the specific recommendations about licensing servers; my preference would be to wait for folks to come to us with questions about how that should work and see what the common concerns actually are. We may end up having to put a thing in there at some point that says something to the effect of "ultimately Sandstorm favors user autonomy over developer control. We recognize this limits technical options for enforcing licensing, but you'll have to live with that."

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it help if we added vagrant-spk unpack? (spk unpack is already a thing, but we run into that whole packaging tool confusion nightmare.) I think having a proprietary app publish a "source" repository that doesn't really include the app's source is just going to add confusion.

I suppose I should look at the GPLv3's rules around installation information though, so I can have an informed thought on it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, I am reading https://en.wikipedia.org/wiki/GNU_General_Public_License#Communicating_and_bundling_with_non-GPL_programs and I would definitely say that an SPK is an "aggregate" rather than a single program. It is an unpackable partial operating system, and contains many various programs which use things such as pipes and arguments to communicate. (I think adding a vagrant-spk unpack would be good to ensure someone can unpack/pack with the same tool.)

How would you feel about a line/section that specified that developers are responsible for ensuring their packages do not run afoul of copyleft or any other open source licensing terms and that we will de-list a package reported as in violation of open source licenses?


The Sandstorm App Market supports both commercial and proprietary apps, however
official support for app purchase and licensing is not currently available. For an
app that is merely closed source, but free, this may not pose an issue. For apps
which require some form of licensing to function, we expect at minimum that the
app description clearly and prominently specifies the licensing requirements and
activation method. Sandstorm apps should be as self-sufficient as possible, so a
license key method using some sort of signing that does not need to call home is ideal,
however, a check with a licensing server may be acceptable if it is narrow in scope,
and utilizes the Powerbox to get the user's consent for their server to contact
yours. We would also encourage the app to have at least a minimal level of functionality
ocdtrekkie marked this conversation as resolved.
Show resolved Hide resolved
such as a trial or limited version, such that users can experience the app before
deciding on a purchase. As this is a territory the Sandstorm app review team hasn't
spent a lot of time on, interested parties may wish to reach out to the Sandstorm
community for feedback prior to committing significant work here.