-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Design #1
Comments
Here's what I envision so far: Core has the idea of groups, users, projects, environments, providers, drones, plugins, and workflows.
Here's the workflow I imagine. You create a project (under group or user) and you assign a name, Drones should be clustered so that they don't die and you have to connect to those machines every time. Workflow setup will also install the necessary plugins on the drones. I can see drones as having one endpoint that takes an action and data; the actions could be like setup and trigger. I think plugins will need to be on both, since workflows need to be created and we need to know what steps are part of a plugin. Workflows are necessary since we don't want the user touching the drones, since ideally you should ever touch them once, when you create them (or update). Having multiple drones in an environment will allow parallel builds. Also versioning plugins and drones to a specific core API version will save a lot of frustration I think. Also workflows should be exportable/importable and shareable as JSON via a gist or something. |
Having it such that it is the user responsibility to manage the drones (ssh in to installs things like node dependencies, heroku toolbelt, ssh keys for being able to pull, strider plugins, etc) drastically reduces core's responsibilities. if a [core] plugin writer wishes to automate drone management he or she may do so and simply ask for, in a UI, for an IP address, SSH port, username, password, or private key. but I think it's asking too much to do this in core |
I think you are right about half of it, but plugin management on drones should totally be in core. By that I mean plugins get installed automatically via some hook triggered by core. I think this is fundamental, because no one wants to manage drones individually, it's error prone and a waste of time. Sure let the user install the drone, install node/npm, or python, etc. themselves, but the plugins (npm based) should be managed from core. |
If we're strictly speaking about a plugin from npm or git getting remotely installed, configured, maintained by core, then I think that is fine. I was talking about machine provisioning, not so much drone runtime. I agree w/ you with that regard, although I am curious what such plugins would be like. |
Yeah, I would love to know what they look like as well ;) |
This might be wildly out of spec/whack but wouldn't using something like rabbitmq, eventd, kafka, kestrel, etc as the communication technology between servers and 'core' be the parent node that manages drones. It would be nice because one server can manage many which makes adding/removing servers easy as well as distributed without some arbitrary Node code or any downtime. It's a good model that we can see working all over the internet as a solution to exactly the problem you're talking about @keyvanfatehi it's also language independent, a thin REST layer on top would make it very manageable by sysadmins and developers. |
I would strongly recommend not building a heavy weight dependency like These systems are nice but not trivial to deploy and manage. My suggestion would be have something simple built into core (I am a big Admittedly this doesn't support node auto discovery but that can be layered Other transports could be supported for discovery and/or persistent I really don't think bundling a JVM or RabbitMQ binary with Strider would Also there are many ways to do discovery - for example new nodes can ping On Tuesday, July 14, 2015, Dave Mackintosh [email protected] wrote:
Niall O'Higgins |
When you setup a node with the token (which has the core url) it will ping the core and authenticate making it ready for additional configuration. This could be done with http. If the node doesn't have access to core, then you could ping the node from core or use a proxy. If we want status checking, we could use long-polling it's fast enough for pinging. For stdout data, we can just stream the http request. |
Websockets would work well for that sort of thing. PrimusJS and the WS module work well together for a persistent client / server architecture (have implemented something similar)
|
IronMQ provides an http API for messaging. POST messages Of course this was not great for my application that needed to be realtime, I agree with not bundling this stuff but I also acknowledge the existence That said, we can easily do a simple MQ concept with http ourselves simply On Tuesday, July 14, 2015, niallo [email protected] wrote:
|
@microadam this is server-to-server. |
Id rather avoid websockets to be honest. My early vote is simple messaging over rest API with pluggable backend so if folks wanna use rabbit they can. Tough to achieve though because you cannot ack a packet if you close the channel. Rabbit works way better when it's not wrapped and hidden. Maybe this kind of plugin is exemplary of one that would exist in core and drone side |
s/exemplary/an example/ can't edit comments on iPhone for some reason |
@knownasilya I like your idea so far. However, I think we shouldn't focus on the clustered architecture just yet. For now I think we should mainly focus on the interaction between plugins Therefore, I propose that we should split each component (by component I don't only mean plugins I also mean functional units in plugins and in core) of strider into microservices. In case some of you are not familiar with microservices: think of them as very lightweight REST API services which only implement some basic functionality (e.g. one possible microservice could, given a Github pull request sha-string check whether this pull request is already being processed by strider).
|
Also: maybe it wasn't completely clear from my ramblings in my previous post but I think we should proceed as follows:
BTW: are there any plans when we will start with the implementation for this thing? I'd be more |
Can start as soon as you want. I just pushed up some basic structural code/tests with hapi (I'm really liking it). |
Technology stack discussion should happen in #2 |
Primus and WS work server to server
|
I have a design question: shouldn't we use a version prefix in the HTTP API urls and shouldn't we implement HATEOAS ? |
Yes on the prefix, I've already added that. No on HATEOAS, since that's not a standard practice for node rest apis (more of a java/soap practice), I'd rather implement http://jsonapi.org |
+1 for JSON API. |
Picking up from Strider-CD/strider#667
Idea
Thin API that communicates with drones and supports plugins. No UI in core, since it can be implemented as a plugin.
Goal
Lay-down a basic design and non-negotiables that should be in core.
TODOs
The text was updated successfully, but these errors were encountered: