Skip to content

Commit

Permalink
Merge pull request #80 from 0chain/sunilmhta/disable-multipart-client
Browse files Browse the repository at this point in the history
disable multi part in client-api layer.
  • Loading branch information
dabasov authored Nov 23, 2023
2 parents 8f1ada1 + 1f61cf3 commit 23372af
Show file tree
Hide file tree
Showing 3 changed files with 45 additions and 25 deletions.
60 changes: 39 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,47 @@
# Introduction
This module provides s3-compatible API to 0chain allocation using minio-gateway Interface.
User can set their access-key and secret-key before running zs3Server. So basically, zs3server is an http server that provides s3 compatible api so that clients that are already s3 compatible can easily communicates with 0chain allocationcan. Its just a plug and play.
# zs3server - a no-code decentralized storage server

# Architecture
zs3server provides a no-code s3-compatible decentralized storage server on Züs allocation using a minio-gateway interface.

- [Züs Overview](#züs-overview)
- [Architecture](#architecture)
- [Building zs3 server](#building-zs3-server)
- [Running zs3 server](#running-zs3-server)
- [Test MiniO client](#test-minio-client)
- [Configure Minio client](#configure-minio-client)

## Züs Overview
[Züs](https://zus.network/) is a high-performance cloud on a fast blockchain offering privacy and configurable uptime. It is an alternative to traditional cloud S3 and has shown better performance on a test network due to its parallel data architecture. The technology uses erasure code to distribute the data between data and parity servers. Züs storage is configurable to provide flexibility for IT managers to design for desired security and uptime, and can design a hybrid or a multi-cloud architecture with a few clicks using [Blimp's](https://blimp.software/) workflow, and can change redundancy and providers on the fly.

For instance, the user can start with 10 data and 5 parity providers and select where they are located globally, and later decide to add a provider on-the-fly to increase resilience, performance, or switch to a lower cost provider.

Users can also add their own servers to the network to operate in a hybrid cloud architecture. Such flexibility allows the user to improve their regulatory, content distribution, and security requirements with a true multi-cloud architecture. Users can also construct a private cloud with all of their own servers rented across the globe to have a better content distribution, highly available network, higher performance, and lower cost.

[The QoS protocol](https://medium.com/0chain/qos-protocol-weekly-debrief-april-12-2023-44524924381f) is time-based where the blockchain challenges a provider on a file that the provider must respond within a certain time based on its size to pass. This forces the provider to have a good server and data center performance to earn rewards and income.

The [privacy protocol](https://zus.network/build) from Züs is unique where a user can easily share their encrypted data with their business partners, friends, and family through a proxy key sharing protocol, where the key is given to the providers, and they re-encrypt the data using the proxy key so that only the recipient can decrypt it with their private key.

Züs has ecosystem apps to encourage traditional storage consumption such as [Blimp](https://blimp.software/), a S3 server and cloud migration platform, and [Vult](https://vult.network/), a personal cloud app to store encrypted data and share privately with friends and family, and [Chalk](https://chalk.software/), a zero upfront cost permanent storage solution for NFT artists.

Other apps are [Bolt](https://bolt.holdings/), a wallet that is very secure with air-gapped 2FA split-key protocol to prevent hacks from compromising your digital assets, and it enables you to stake and earn from the storage providers; [Atlus](https://atlus.cloud/), a blockchain explorer and [Chimney](https://demo.chimney.software/), which allows anyone to join the network and earn using their server or by just renting one, with no prior knowledge required.

## Architecture

![Main-architecture](./assets/main-struture.png)

There are three main components that will be installed in the customer server.

1. ZS3Server is the main component which will communicate directly with 0chain allocation
1. ZS3Server is the main component that will communicate directly with Züs storage.

2. [LogSereach](/logsearchapi/README.md) API is the log component which will store the audit log from S3 server and It will be consumed using ZS3 API
2. [LogSearch](/logsearchapi/README.md) API is the log component that will store the audit log from the S3 server and it will be consumed using ZS3 API

3. [MinioClient](/client-api/README.md) is the component that will communicate directly to the zs3server and it is protected using access and secret key.

## Building zs3-server
Prerequisites to run MinIO ZCN gateway:
- [A wallet.json created using zwalletcli](https://github.com/0chain/zwalletcli#creating-and-restoring-wallets)
- [Config.yaml](https://github.com/0chain/zboxcli/blob/staging/network/config.yaml)
- [An allocation.txt created using zboxcli](https://github.com/0chain/zboxcli/tree/staging#create-new-allocation).

# Run zs3-server
As a prerequisite to run MinIO ZCN gateway, you need 0chain credentials; wallet.json, config.yaml and allocation.txt.

## Build Binary
```
git clone [email protected]:0chain/zs3server.git
cd zs3server
Expand All @@ -29,22 +52,17 @@ export MINIO_ROOT_PASSWORD=someminiopassword
./minio gateway zcn --configDir /path/to/config/dir
Note: allocation and configDir both are optional. By default configDir takes ~/.zcn as configDir and if allocation is not provided in command then it will look for allocation.txt file in configDir directory.
```

> If you want to debug on local you might want to build with `-gcflags="all=-N -l"` flag to view all the objects during debugging.
## Run using docker

To build and run minio sevrer component in any machine you will need first to install docker and docker-compose

1. Make sure docker and docker-compose in your machine

2. Make sure you have the allocation ready in the default folder ``~/.zcn``
## Running zs3-server

3. Run docker-compose command like the following
1. To build and run the minio server component you need to install [docker](https://www.docker.com/products/docker-desktop/).

2. Run the docker-compose command inside the zs3server directory./
```
docker-compose up -d
docker-compose -f environment/docker-compose.yaml up -d
```
3. Make sure allocation.txt file exist in the default folder ``~/.zcn``

4. Now you can interact with the clint API follow this [doc](/client-api/README.md)

Expand Down Expand Up @@ -121,7 +139,7 @@ and Amazon S3 compatible cloud storage services.
### Installation
Install from here for your os: https://min.io/docs/minio/macos/index.html

### Configure `mc`
## Configure MinIO Client
```
mc config host add zcn http://localhost:9000 miniouser miniopassword
```
Expand Down
4 changes: 3 additions & 1 deletion client-api/s3api/put_object.go
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,9 @@ func putObject(bucketName string, objectName string, minioCredentials MinioCrede
return putObjectResponse, err
}

uploadInfo, err := minioClient.PutObject(ctx, bucketName, objectName, file, fileStat.Size(), minio.PutObjectOptions{ContentType: "application/octet-stream"})
// TODO: Enable multi part once it is supported.
// Disabling multi part as of now because multipart APIs are not implemented for zcn gateway.
uploadInfo, err := minioClient.PutObject(ctx, bucketName, objectName, file, fileStat.Size(), minio.PutObjectOptions{DisableMultipart: true,ContentType: "application/octet-stream"})
if err != nil {
fmt.Println("error from PutObject", err)
return putObjectResponse, err
Expand Down
6 changes: 3 additions & 3 deletions environment/docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ services:
- db:/var/lib/postgresql/data

logsearchapi:
image: 0chaindev/blimp-logsearchapi:v0.0.3
image: 0chaindev/blimp-logsearchapi:staging
depends_on:
- db
ports:
Expand All @@ -27,7 +27,7 @@ services:
links:
- db
minioserver:
image: 0chaindev/blimp-minioserver:v0.0.2
image: 0chaindev/blimp-minioserver:staging
container_name: minioserver
ports:
- 9000:9000
Expand All @@ -44,7 +44,7 @@ services:
volumes:
- ~/.zcn:/root/.zcn
minioclinet:
image: 0chaindev/blimp-clientapi:v0.0.4
image: 0chaindev/blimp-clientapi:staging
container_name: minioclient
depends_on:
- minioserver
Expand Down

0 comments on commit 23372af

Please sign in to comment.