Skip to content

Latest commit

 

History

History
112 lines (92 loc) · 5.57 KB

README.md

File metadata and controls

112 lines (92 loc) · 5.57 KB
Raw

codecov Go

ColonyOSLogo

Tutorials

This repo contains several tutorials to help you get started with ColonyOS.

What is Colonies?

ColonyOS is an open-source framework designed to facilitate seamless execution of computational workloads across platforms, e.g. cloud, edge, devices, or High-Performance Computing (HPC), creating so-called Compute Continuums. Colonies is key component of ColonyOS and provides a Meta-Orchestrator, delivering functionalities like a traditional operating system, but functions as an overlay on top of existing platforms.

At its core, Colonies provides a distributed runtime environment, called a Colony, consisting of a network of loosely connected Executors. Executors are distributed microservices that can easily integrate with any third-party application or system. They can be implemented in any language and run anywhere on the Internet, e.g. on supercomputers or in a web browser on a smart phone.

Colonies has been developed with a strong emphasis on security and robustness to ensure scalability and reliability.

How does it work?

  • Users submit function specifications to a Colonies server specifying computations they wish to run. These functions will then be assigned and executed by the Executors.
  • Developers can focus on implementing Executors as independent microservices. The system can then easily scale just by deploying more Executors, enabling parallel execution. Failed processes are automatically re-assigned to other Executors.
  • Developers can define and submit workflows describing a sequence of computations carried out by several Executors.
  • Colonies makes it possible to develop a loosely-decoupled architecture that spans multiple platforms and infrastructures, with all coordination managed by Colonies.
  • Colonies also functions as a ledger, containing full execution history. This enables developers to keep track of the system and more easily debug it.

Design

  • Colonies serves as an intermediary layer to indirectly control and integrate with other computer systems. This enables Executors (or users) to publish instructions and subsequently assign these instructions to other Executors.
  • By chaining instructions together, it becomes possible to execute workloads that can operate seamlessly across different platforms
  • A build-in zero-trust protocol makes it possible to organize remote Exectors as a single unit (a Colony). This feature empowers users to maintain control, even when distributed are deployed on multiple platforms.

MetaOS

Example

Start a Colonier server

source devenv
colonies dev

Submit a meta-process

{
    "conditions": {
        "executortype": "cli"
    },
    "funcname": "echo sayhello"
}
colonies function submit --spec sayhello.json

Start a Unix executor (executes functions as Unix commands)

See Unix executor repo how to install an Unix executor.

./bin/unix_executor start 

INFO[0000] Lauching process                              Args="[]" Func="echo sayhello"
sayhello

See this guide how to implement executors in Python, Julia, Go, and JavaScript.

Dashboard screenshots

Below are some screenshots from the Colonies Dashboard: Dashboard Dashboard Dashboard

More information

Installation

Presentations

Guides

Design

Executors

SDKs

Deployment

More information can also be found here.

Current users

  • ColonyOS is currently being used by RockSigma AB to build a compute engine for automatic seismic processing in underground mines.

Running the tests

Follow the instructions at Installation Guide then type:

make test