From fa59803b7643789ee7e705216c5c370ed6810a5e Mon Sep 17 00:00:00 2001 From: Jacob Weisz Date: Tue, 21 Jul 2020 14:08:12 -0500 Subject: [PATCH 1/3] Docs: Create app review guide Kinda a preliminary take on this. --- docs/developing/app-review-guide.md | 76 +++++++++++++++++++++++++++++ 1 file changed, 76 insertions(+) create mode 100644 docs/developing/app-review-guide.md diff --git a/docs/developing/app-review-guide.md b/docs/developing/app-review-guide.md new file mode 100644 index 0000000000..44c22d188d --- /dev/null +++ b/docs/developing/app-review-guide.md @@ -0,0 +1,76 @@ +Apps submitted to the [Sandstorm App Market](https://apps.sandstorm.io/) go +through an app review process. This page serves to both inform developers +and the community about the process, and serve as a guide to app reviewers. + +## New app approval process + +When an app is submitted to the app market, it becomes immediately available +on the [experimental market](https://apps.sandstorm.io/?experimental=true) for +testing and evaluation. App reviewers are notified that a new package is in +the queue. + +During app review, an app reviewer will email the developer at the contact +email listed in the package metadata, introducing themselves and covering the +review process. The reviewer will provide feedback and advice on the submitted +app, which may included both approval-blocking issues and suggested changes or +fixes. The reviewer will also make sure the metadata displays correctly in the +market and appears to accurately depict the submitted app, and if applicable, +may look at the source repository with an eye for Sandstorm-specific changes +to the code. + +The app developer can then incorporate that feedback and resubmit, or if none +of the feedback was approval-blocking, request the app be approved as already +submitted. When the app reviewer approves the package, they will notify you as +the App Market does not contact developers directly. + +## App update approval process + +Generally speaking, app review for app updates moves significantly faster. An +app reviewer should test both new grains and grain updates of the app, with a +focus on ensuring users will not lose data in existing grains when they get the +update. The reviewer may look at the changelog and commit history to narrow down +what new features or changes should be tested, and may reach out to the +developer with suggestions or feedback. However, in many cases, the app reviewer +will simply approve the update after verifying it's functionality and notify the +developer that it has been approved. + +## Approval-blocking issues + +The Sandstorm App Market is a fairly permissive environment. While you may +receive feedback about performance, functionality, or usability, we largely +require only that your app works and that it does not lose data. + +We will reject an app which crashes immediately and does not function as +advertised, for example. This can often occur when the package is missing files +that were present in the dev environment and wasn't adequately tested prior to +submission. + +The latter concern often occurs when a pure client app is submitted up that +implements localStorage as its storage medium, which Sandstorm does not effectively +support. This can often be tested by utilizing a grain, allowing Sandstorm to shut +down that grain, and then opening it again and seeing if things persisted that +an ordinary user would expect to persist between sessions. + +Finally, the App Market may reject apps which present security or privacy issues +that fall outside of what a Sandstorm user would expect when using the app. This +can mean it requests overly-broad permissions when making Powerbox requests or +utilizes client-side loading to load scripts which share unexpected information +outside of the Sandstorm sandbox. For example, we may block approval on an app +which attempts to load an analytics script from an outside server, or does not +make it suitably clear to a user why it needs certain outside network access +prior to requesting it. + +## Commercial or proprietary apps + +The Sandstorm App Market supports both commercial and proprietary apps, however +official support for app purchase and licensing is not currently available. For an +app that is merely closed source, but free, this may not pose an issue. For apps +which require some form of licensing to function, we expect at minimum that the +app description clearly and prominently specifies this and what method this can +take. Sandstorm apps should be as self-sufficient as possible, so a license key +method utilizing some sort of signing that does not need to call home is ideal, +however, a check with a licensing server may be acceptable if it is narrow in scope, +and utilizes the Powerbox to get the user's consent for their server to contact +yours. As this is a territory the Sandstorm app review team hasn't spent a lot of +time on, interested parties may wish to reach out to the Sandstorm community for +feedback prior to committing significant work here. From b06f49152dc6e3dc7e76eb226a5aff86af187384 Mon Sep 17 00:00:00 2001 From: Jacob Weisz Date: Wed, 22 Jul 2020 10:07:45 -0500 Subject: [PATCH 2/3] Docs: Second pass on app review guide --- docs/developing/app-review-guide.md | 40 +++++++++++++++++------------ 1 file changed, 24 insertions(+), 16 deletions(-) diff --git a/docs/developing/app-review-guide.md b/docs/developing/app-review-guide.md index 44c22d188d..92c24cbbef 100644 --- a/docs/developing/app-review-guide.md +++ b/docs/developing/app-review-guide.md @@ -28,10 +28,13 @@ the App Market does not contact developers directly. Generally speaking, app review for app updates moves significantly faster. An app reviewer should test both new grains and grain updates of the app, with a focus on ensuring users will not lose data in existing grains when they get the -update. The reviewer may look at the changelog and commit history to narrow down +update. If the app is open source, the app reviewer should verify that the update +was pushed to the source repository as well prior to approval. + +The reviewer may look at the changelog and commit history to narrow down what new features or changes should be tested, and may reach out to the developer with suggestions or feedback. However, in many cases, the app reviewer -will simply approve the update after verifying it's functionality and notify the +will simply approve the update after verifying its functionality and notify the developer that it has been approved. ## Approval-blocking issues @@ -47,18 +50,21 @@ submission. The latter concern often occurs when a pure client app is submitted up that implements localStorage as its storage medium, which Sandstorm does not effectively -support. This can often be tested by utilizing a grain, allowing Sandstorm to shut -down that grain, and then opening it again and seeing if things persisted that -an ordinary user would expect to persist between sessions. +support. This can often be tested by storing some information in a grain, allowing +Sandstorm to shut down that grain, and then opening it again and seeing if things +persisted that an ordinary user would expect to persist between sessions. Finally, the App Market may reject apps which present security or privacy issues that fall outside of what a Sandstorm user would expect when using the app. This can mean it requests overly-broad permissions when making Powerbox requests or -utilizes client-side loading to load scripts which share unexpected information -outside of the Sandstorm sandbox. For example, we may block approval on an app -which attempts to load an analytics script from an outside server, or does not -make it suitably clear to a user why it needs certain outside network access -prior to requesting it. +uses client-side loading to load scripts which share unexpected information outside +of the Sandstorm sandbox. For example, we may block approval on an app which attempts +to load an analytics script from an outside server, or does not make it suitably +clear to a user why it needs certain outside network access prior to requesting it. + +**Note:** Client-side loading is a known gap in Sandstorm's security model that we +intend to close. Not only is misuse of it a security or privacy issue, it's use may +lead to apps that break when Sandstorm closes this gap. ## Commercial or proprietary apps @@ -66,11 +72,13 @@ The Sandstorm App Market supports both commercial and proprietary apps, however official support for app purchase and licensing is not currently available. For an app that is merely closed source, but free, this may not pose an issue. For apps which require some form of licensing to function, we expect at minimum that the -app description clearly and prominently specifies this and what method this can -take. Sandstorm apps should be as self-sufficient as possible, so a license key -method utilizing some sort of signing that does not need to call home is ideal, +app description clearly and prominently specifies the licensing requirements and +activation method. Sandstorm apps should be as self-sufficient as possible, so a +license key method using some sort of signing that does not need to call home is ideal, however, a check with a licensing server may be acceptable if it is narrow in scope, and utilizes the Powerbox to get the user's consent for their server to contact -yours. As this is a territory the Sandstorm app review team hasn't spent a lot of -time on, interested parties may wish to reach out to the Sandstorm community for -feedback prior to committing significant work here. +yours. We would also encourage the app to have at least a minimal level of functionality +such as a trial or limited version, such that users can experience the app before +deciding on a purchase. As this is a territory the Sandstorm app review team hasn't +spent a lot of time on, interested parties may wish to reach out to the Sandstorm +community for feedback prior to committing significant work here. From b594d1ca095728fae12021dba6a663b070ade731 Mon Sep 17 00:00:00 2001 From: Jacob Weisz Date: Thu, 23 Jul 2020 10:13:16 -0500 Subject: [PATCH 3/3] Docs: Third pass on app review guide --- docs/developing/app-review-guide.md | 34 +++++++++++++++++++---------- 1 file changed, 23 insertions(+), 11 deletions(-) diff --git a/docs/developing/app-review-guide.md b/docs/developing/app-review-guide.md index 92c24cbbef..22ca17d5b9 100644 --- a/docs/developing/app-review-guide.md +++ b/docs/developing/app-review-guide.md @@ -63,22 +63,34 @@ to load an analytics script from an outside server, or does not make it suitably clear to a user why it needs certain outside network access prior to requesting it. **Note:** Client-side loading is a known gap in Sandstorm's security model that we -intend to close. Not only is misuse of it a security or privacy issue, it's use may +intend to close. Not only is misuse of it a security or privacy issue, its use may lead to apps that break when Sandstorm closes this gap. ## Commercial or proprietary apps -The Sandstorm App Market supports both commercial and proprietary apps, however +The Sandstorm App Market permits both commercial and proprietary apps, however official support for app purchase and licensing is not currently available. For an app that is merely closed source, but free, this may not pose an issue. For apps which require some form of licensing to function, we expect at minimum that the app description clearly and prominently specifies the licensing requirements and -activation method. Sandstorm apps should be as self-sufficient as possible, so a -license key method using some sort of signing that does not need to call home is ideal, -however, a check with a licensing server may be acceptable if it is narrow in scope, -and utilizes the Powerbox to get the user's consent for their server to contact -yours. We would also encourage the app to have at least a minimal level of functionality -such as a trial or limited version, such that users can experience the app before -deciding on a purchase. As this is a territory the Sandstorm app review team hasn't -spent a lot of time on, interested parties may wish to reach out to the Sandstorm -community for feedback prior to committing significant work here. +activation method. We would also encourage the app to have at least a minimal level of +functionality such as a trial or limited version, such that users can experience the +app before deciding on a purchase. As this is a territory the Sandstorm app review +team hasn't spent a lot of time on, interested parties may wish to reach out to the +Sandstorm community for feedback prior to committing significant work here. + +## Copyright and copyleft + +The developer is responsible for ensuring their packages comply with all relevant +copyright or copyleft limitations. If a package is reported to us as in violation of a +license, it may be pulled from the App Market. + +Sandstorm package files (or SPKs) are an archive format which includes a variety of +components needed to run a given application. Often the process of building an SPK pulls +in components and libraries from open source distributions such as Debian. We believe +these fall comfortably within the GPLv3's [system libraries exception](https://www.gnu.org/licenses/quick-guide-gplv3.html#less-source-to-distribute-new-system-libraries-exception) and falls well within the spirit of most open source licensing models. + +Open source Sandstorm packages generally provide all of the information necessary to modify +and recreate them, including where to get any other components required to build the SPK, +however, proprietary apps may need to seek legal counsel on what requirements are needed +to comply.