Hello World Todo

[marcelklehr]

• Finalize website setup
  • DNS CNAME for api.magpie-experiments.org (reference)
  • DNS A for magpie-experiments.org (docs; I'm happy to do these, but would need to have access to namecheap)
  • Update website-references to new domain
• Bump major version to >= 1 Any low hanging breaking changes already visible?
  • Should we unify measurements case to snake_case? #121
  • Should responseTime be called responseTimeout to avoid confusion? #122
• Naming confusion between "magpie v3" and magpie-base version (currently v0.6) #123
• Get rid of cute Trump references in manual
• Find out dynamically if deploy is on github pages or netlify
• Sort our docker container business (Some fixes for use in docker magpie-backend#131)
• Release magpie-base v0.7.0 v3.0.0 with latest changes
• Add applicable version number to docs
• Update example projects on github to latest version of magpie-base
• Link to repos of magpie-base and magpie-backend from docs
• change allocation order
• pilot iterated guessing
• pilot color guessing

[x-ji]

Looks good! Just to clarify, is the "docker container" only meant to be used in the CI or does it relate to user-facing deployment e.g. on a local machine and online?

[michael-franke]

Not sure if I’m not missing more recent devs, but originally the docker image -I think- was for the user-side to be able to run the backend locally (e.g., when conducting experiments in environments w/o access to the internet).

…

On 23 Mar 2022, at 20:08, Xiang Ji ***@***.***> wrote: Looks good! Just to clarify, is the "docker container" only meant to be used in the CI or does it relate to user-facing deployment e.g. on a local machine and online? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.

[x-ji]

Not sure if I’m not missing more recent devs, but originally the docker image -I think- was for the user-side to be able to run the backend locally (e.g., when conducting experiments in environments w/o access to the internet).

Exactly. This is why I was somewhat confused by magpie-ea/magpie-backend#131 by Marcel. I think now it's clarified after our discussions there, but indeed Marcel is talking about a separate Dockerfile for use in the CI it seems.

[marcelklehr]

Not sure if I’m not missing more recent devs, but originally the docker image -I think- was for the user-side to be able to run the backend locally

Indeed, I created a new docker image for automated testing of the interactive bits in magpie-base. I'm not particular about people using it for normal deployment, though, which would be quite convenient in my opinion, at least when not using heroku.

[marcelklehr]

@x-ji Would you mind creating a new release for magpie-backend, too? Doesn't need to be v3, IMHO (although I'd prefer >v1.0.0 and the semver rules that come with that), but a new release number for every version that is used in production would be nice. Then we can document which versions of magpie-base are compatible with which versions of magpie-backend.

[marcelklehr]

Short update: magpie-base v3.0.0 is now released.

[x-ji]

Short update: magpie-base v3.0.0 is now released.

Great! We haven't actually used release versioning on the backend. The tagged release numbers you may see in the repo are related to the previous attempt to create local releases, and the number hasn't been bumped for more than 3 years I think.

It seems to me that the backend should be compatible with magpie-base for submitting non-complex experiments right? And among the complex experiments, I'm not sure if only interactive experiments would encounter a break of behavior (i.e. iterative experiments would also be fine with all backend versions), or maybe iterative experiments would also break (AFAIK the behavior of the waiting lobby might have also changed).

I think one way to make it happen is to simply use git commit hashes. We can add an instruction for people to check out a particular commit on the master branch of magpie-backend. Seems to me the commit 043f34a012562b283b3efbc788aa12284fbc8551 is the one before the refactoring started. WDYT.

[marcelklehr]

I think it would behoove us to have a clear versioning with CHANGELOG (as well as an update mechanism) in place, as that would allow to make changes with and without breaking compatibility, while being transparent about it. Git commit hashes seem not ideal for this.

The target for such a versioning scheme should probably be the backend HTTP and socket API. Note, that we could still support multiple major magpie-backend versions with one magpie-base version.

I'd be in favor of tagging the commit before the refactoring started as v1.0.0 and tagging the version pushed to production now as v2.0.0.

[x-ji]

Sure, I totally see your perspective. IMO one thing to think about though is that versioning is usually useful for something packaged as a release, i.e. something for the end user to install (either as a desktop application, an NPM package, or as a one-click runnable binary (which was what the previous desktop release of magpie-backend tried to achieve. You can see downloadable releases at https://github.com/magpie-ea/magpie-backend/releases)). At the moment, magpie-backend is still to be deployed by manually cloning the git repo and performing the deployment, which IMO makes the labeling less useful. Sure, they can probably check out a commit by a tag instead of by the hash value, but the steps after that would be the same.

I think it's definitely a good idea to have a simpler mechanism to update the server deployment. Maybe using the deployment script for Heroku can already help.

I can create the v1.0.0 and v2.0.0 tag, as you mentioned, though still I think it's a bit fuzzy when should a version number bump should occur, given that of course we wouldn't tag each new commit with a new version number, and there will probably not be many major feature changes planned. We can for sure further refine this as we see how people really use and upgrade the backend and what their feedback is. WDYT.

[x-ji]

By the way, another thing that occurred to me is that, in backend design, one usual way to denote breaking API changes is by different prefixes, e.g. api/v1/... vs. api/v2/..., while they would actually co-exist. Since here we're dealing with socket connections though, and since it would be hard to keep the old socket behaviors without changing underlying stuff such as DB tables, this might not be so easily applicable.

[x-ji]

The tags v1.0.0 and v2.0.0 are created here: https://github.com/magpie-ea/magpie-backend/tags :) They should be quite flexible to changes if we so wish, different from e.g. releases on a public registry, since they are just labels applied to specific commits. Would this be good enough? For the documentation that explains this, it would go in our main documentation repo/site, right?

[marcelklehr]

At the moment, magpie-backend is still to be deployed by manually cloning the git repo and performing the deployment, which IMO makes the labeling less useful. Sure, they can probably check out a commit by a tag instead of by the hash value, but the steps after that would be the same.

I think it's a bit fuzzy when should a version number bump should occur, given that of course we wouldn't tag each new commit with a new version number, and there will probably not be many major feature changes planned.

I agree. We could use the master branch for releases only and the actual development on a develop branch. This way, we can test stuff out on the develop branch and merge into master only when we want to release. I'd still think having semver versions would be good even in this case. What do you think?

I think it's definitely a good idea to have a simpler mechanism to update the server deployment. Maybe using the deployment script for Heroku can already help.

That's also partly why I was looking into docker, but there's no simple service for deploying docker images that I can see. Does heroku allow deploying docker images directly, perhaps, or docker-compose files?

The tags v1.0.0 and v2.0.0 are created here: magpie-ea/magpie-backend/tags :)

Cool! As I said above, I think we should not change tags once they are published and rather use a new tag, bumping the version number.

For the documentation that explains this, it would go in our main documentation repo/site, right?

Yep, I'm happy to add this to the docs.

[marcelklehr]

Aside from these specifics, I think we're good to go on with a Hello World. What do you think @x-ji ?

[x-ji]

Let me get back to you before the end of tomorrow 😅 .

[x-ji]

I agree. We could use the master branch for releases only and the actual development on a develop branch. This way, we can test stuff out on the develop branch and merge into master only when we want to release. I'd still think having semver versions would be good even in this case. What do you think?

Yes, I see that's probably what you're doing on magpie-base. I think it might also make sense to open separate branches for specific features, and merge them back to master when the feature is complete, which is the normal flow for developing bigger features. I think for smaller features and fixes we may still directly merge to master. In general I still think that server versioning is somewhat different from client versioning, since we'd also expect normal users to not upgrade the server app that much unless there's a need to do so. I can for sure still tag commits using semver when there are enough changes and see how it goes.

Does heroku allow deploying docker images directly?

Yes, https://hexdocs.pm/phoenix/heroku.html has a guide on how to do it. I can also give it a try to see how it goes. Note that there's already a Dockerfile in the project which was intended to help local releases, so I'll also probably have to clean that up and flesh out the local release mechanism as mentioned in a previous email at the same time.

I think we're good to go on with a Hello World

Do you mean we're ready to create a Hello World app and test it, or do you mean we're ready for a release announcement?

[x-ji]

I will also update magpie-demo and the instance on the Osnabrueck server with the newest version today with the the feature about accumulating JSON keys from different submissions.

[marcelklehr]

Do you mean we're ready to create a Hello World app and test it, or do you mean we're ready for a release announcement?

I meant the release announcement.

Yes, hexdocs.pm/phoenix/heroku.html has a guide on how to do it. I can also give it a try to see how it goes.

Nice, that would be cool.

Note that there's already a Dockerfile in the project which was intended to help local releases

I saw the Dockerfile but noticed that it wasn't working anymore and seems to be more geared towards creating a development instance, which is why I created https://github.com/magpie-ea/magpie-docker as a production build that I could use in CI and could be used for deploying to production. If you like we can merge the two.

In general I still think that server versioning is somewhat different from client versioning, since we'd also expect normal users to not upgrade the server app that much unless there's a need to do so.

I agree. My main concern was a) being explicit with which version is compatible to which, which I would solve by using semantic versioning in order to avoid using commit hashes directly and b) having some kind of release channel to a void people updating to a version that is not yet stable, which I would solve by using a stable branch (which only has the latest release version; whatever that branch is called) that people checkout and install from.

[x-ji]

having some kind of release channel to avoid people updating to a version that is not yet stable, which I would solve by using a stable branch (which only has the latest release version; whatever that branch is called) that people checkout and install from.

Yes, that's totally reasonable. Have you written the documentation about version compatibility (I guess at magpie-site)? Let me finish testing one-click deployment on Heroku today and add to the documentation how to deploy and upgrade the backend from the latest releases on the master branch.

why I created https://github.com/magpie-ea/magpie-docker as a production build that I could use in CI and could be used for deploying to production. If you like we can merge the two.

Nice. I see in the README this seems to be intended to be run locally as well? We can add a docker-compose.yml to this repo so that it can be run locally directly, though I guess Michael's wish is that if researchers can run it without having to install Docker, it would be optimal, which I will also explore as mentioned in a recent email.

For CI I think it should be possible to point to magpie-backend directly, so it would indeed be great if we could merge the repos. I guess for today I'll finish the deployment guide first and we can make the announcement, after which we can refine that part :)

[x-ji]

I spent some time trying to set up a Docker + Heroku workflow for deploying the app, though I actually end up seeing limited benefits, especially since we still have to set some environment variables for the release, which is what most of the current steps are doing anyways.

There's a new service https://fly.io (The creator of Phoenix framework now works there) which basically supports true one-command deployment from the command line, but its registration requires inputting a credit card up front, which I'm not sure if most researchers will be happy to do compared to Heroku.

In addition to the heroku-deploy.sh script, I added another script heroku-update.sh. The way to use them is explained in the README. I hope these will be good enough at the moment for the users to easily deploy and update the app. Let me know if the scripts work well for you and if you have any suggestions :)

[x-ji]

I think everything should be ready, if we agree that we can live with the deploy and update scripts for the backend for now.

[marcelklehr]

Have you written the documentation about version compatibility?

I added it here: https://magpie-experiments.org/02_using_the_server_app/01_installation/

I see in the README this seems to be intended to be run locally as well?

Not really, I guess for people that want to test their experiments, other solutions are better. The rationale was to have a production docker image available that can be run anywhere, without too much hassle if one knows docker as well as using it as a backend in the CI tests on github.

For CI I think it should be possible to point to magpie-backend directly, so it would indeed be great if we could merge the repos.

Roger that. Ideally we'd also figure out setting username and password at runtime for the docker image, then 🙏 so it still works in the CI tests... :)

[x-ji]

I just pushed v2.0.5 which uses the BasicAuth plug at runtime, which should solve the issue.

As we don't have a Dockerfile in this project itself yet, I think keeping the separate project at the moment for running integration tests is totally fine. If we come up with a Dockerfile for this project itself, we can then think about that. Great work on coming up with the integration tests between magpie-base and magpie-backend 🎉 !

[marcelklehr]

Great work on coming up with the integration tests between magpie-base and magpie-backend tada !

Thanks :) I'm quite proud of it ✨ 😀