This repo contains several tutorials to help you get started with ColonyOS.
ColonyOS is an open-source framework designed to facilitate seamless execution of computational workloads across platforms, e.g. cloud, edge, devices, or High-Performance Computing (HPC), creating so-called Compute Continuums. Colonies is key component of ColonyOS and provides a Meta-Orchestrator, delivering functionalities like a traditional operating system, but functions as an overlay on top of existing platforms.
At its core, Colonies provides a distributed runtime environment, called a Colony, consisting of a network of loosely connected Executors. Executors are distributed microservices that can easily integrate with any third-party application or system. They can be implemented in any language and run anywhere on the Internet, e.g. on supercomputers or in a web browser on a smart phone.
Colonies has been developed with a strong emphasis on security and robustness to ensure scalability and reliability.
- Users submit function specifications to a Colonies server specifying computations they wish to run. These functions will then be assigned and executed by the Executors.
- Developers can focus on implementing Executors as independent microservices. The system can then easily scale just by deploying more Executors, enabling parallel execution. Failed processes are automatically re-assigned to other Executors.
- Developers can define and submit workflows describing a sequence of computations carried out by several Executors.
- Colonies makes it possible to develop a loosely-decoupled architecture that spans multiple platforms and infrastructures, with all coordination managed by Colonies.
- Colonies also functions as a ledger, containing full execution history. This enables developers to keep track of the system and more easily debug it.
- Colonies serves as an intermediary layer to indirectly control and integrate with other computer systems. This enables Executors (or users) to publish instructions and subsequently assign these instructions to other Executors.
- By chaining instructions together, it becomes possible to execute workloads that can operate seamlessly across different platforms
- A build-in zero-trust protocol makes it possible to organize remote Exectors as a single unit (a Colony). This feature empowers users to maintain control, even when distributed are deployed on multiple platforms.
source devenv
colonies dev
{
"conditions": {
"executortype": "cli"
},
"funcname": "echo sayhello"
}
colonies function submit --spec sayhello.json
See Unix executor repo how to install an Unix executor.
./bin/unix_executor start
INFO[0000] Lauching process Args="[]" Func="echo sayhello"
sayhello
See this guide how to implement executors in Python, Julia, Go, and JavaScript.
Below are some screenshots from the Colonies Dashboard:
- Introduction
- Getting started
- How to implement a Colonies executor
- How to implement a Fibonacci executor in Go
- How to create workflows DAGs
- How to use generators
- How to use crons
- How to use the Colonies CLI
- Logging
- Golang Colonies SDK
- Rust Colonies SDK
- Julia Colonies SDK
- JavaScript Colonies SDK
- Python Colonies SDK
- Haskell Colonies SDK
More information can also be found here.
- ColonyOS is currently being used by RockSigma AB to build a compute engine for automatic seismic processing in underground mines.
Follow the instructions at Installation Guide then type:
make test