How to Run Containerized OpenSDS for Testing Work

Here is a tutorial guiding users and new contributors to get familiar with OpenSDS by installing a simple containerized local cluster. You can also use the ansible script to install automatically, see detail in OpenSDS Local Cluster Installation through ansible.

Pre-configuration

Before you start, some configurations are required:

export BackendType="sample" # 'sample' is the default option, currently also support 'lvm'

mkdir -p /etc/opensds && sudo cat > /etc/opensds/opensds.conf <<OPENSDS_GLOABL_CONFIG_DOC
[osdsapiserver]
api_endpoint = 0.0.0.0:50040
auth_strategy = noauth
# If https is enabled, the default value of cert file
# is /opt/opensds-security/opensds/opensds-cert.pem,
# and key file is /opt/opensds-security/opensds/opensds-key.pem
https_enabled = False
beego_https_cert_file =
beego_https_key_file =
# Encryption and decryption tool. Default value is aes.
password_decrypt_tool = aes

[osdslet]
api_endpoint = 0.0.0.0:50049

[osdsdock]
api_endpoint = 0.0.0.0:50050
# Choose the type of dock resource, only support 'provisioner' and 'attacher'.
dock_type = provisioner
# Specify which backends should be enabled, sample,ceph,cinder,lvm and so on.
enabled_backends = sample

[sample]
name = sample
description = Sample Test
driver_name = sample

[ceph]
name = ceph
description = Ceph Test
driver_name = ceph
config_path = /etc/opensds/driver/ceph.yaml

[cinder]
name = cinder
description = Cinder Test
driver_name = cinder
config_path = /etc/opensds/driver/cinder.yaml

[lvm]
name = lvm
description = LVM Test
driver_name = lvm
config_path = /etc/opensds/driver/lvm.yaml
host_based_replication_driver = DRBD

[huawei_dorado]
name = dorado
description = dorado Test
driver_name = dorado
config_path = /etc/opensds/driver/dorado.yaml
replication_type = array_based

[huawei_fusionstorage]
name = fusionstorage backend
description = This is a fusionstorage backend service
driver_name = huawei_fusionstorage
config_path = /etc/opensds/driver/fusionstorage.yaml

[database]
credential = opensds:[email protected]:3306/dbname
endpoint = localhost:2379,localhost:2380
driver = etcd
OPENSDS_GLOABL_CONFIG_DOC

If you choose lvm as backend, you need to make sure physical volume and volume group existed. Besides, you need to configure lvm driver.

sudo pvdisplay # Check if physical volume existed
sudo vgdisplay # Check if volume group existed

mkdir -p /etc/opensds/driver && sudo cat > /etc/opensds/driver/lvm.yaml <<OPENSDS_DRIVER_CONFIG_DOC
tgtBindIp: 127.0.0.1
tgtConfDir: /etc/tgt/conf.d
pool:
  vg001:
    storageType: block
    availabilityZone: default
    extras:
      dataStorage:
        provisioningPolicy: Thin
        isSpaceEfficient: false
      ioConnectivity:
        accessProtocol: iscsi
        maxIOPS: 7000000
        maxBWS: 600
      advanced:
        diskType: SSD
        latency: 5ms
OPENSDS_DRIVER_CONFIG_DOC

OpenSDS Service Installation

If you are a lazy one, just like me, you probably want to do this:(docker-compose required)

wget https://raw.githubusercontent.com/opensds/opensds/master/docker-compose.yml

docker-compose up -d

Or you can do this:

docker run -d --net=host -v /usr/share/ca-certificates/:/etc/ssl/certs quay.io/coreos/etcd:latest

docker run -d --net=host -v /etc/opensds:/etc/opensds opensdsio/opensds-apiserver:latest

docker run -d --net=host -v /etc/opensds:/etc/opensds opensdsio/opensds-controller:latest

docker run -d --net=host --privileged=true -v /etc/opensds:/etc/opensds -v /dev/:/dev/ -v /run/:/run/:shared -v /etc/localtime:/etc/localtime:ro -v /lib/modules:/lib/modules:ro opensdsio/opensds-dock:latest

If you are a smart guy, you probably need to configure your service ip and database endpoint:

export HostIP="your_real_ip"
docker run -d --net=host -v /usr/share/ca-certificates/:/etc/ssl/certs quay.io/coreos/etcd:latest \
 -name etcd0 \
 -advertise-client-urls http://${HostIP}:2379,http://${HostIP}:4001 \
 -listen-client-urls http://0.0.0.0:2379,http://0.0.0.0:4001 \
 -initial-advertise-peer-urls http://${HostIP}:2380 \
 -listen-peer-urls http://0.0.0.0:2380 \
 -initial-cluster-token etcd-cluster-1 \
 -initial-cluster etcd0=http://${HostIP}:2380 \
 -initial-cluster-state new

docker run -d --net=host -v /etc/opensds:/etc/opensds opensdsio/opensds-apiserver:latest /usr/bin/osdsapiserver --api-endpoint=0.0.0.0:50040 --db-endpoint=${HostIP}:2379,${HostIP}:2380

docker run -d --net=host -v /etc/opensds:/etc/opensds opensdsio/opensds-controller:latest /usr/bin/osdslet --api-endpoint=0.0.0.0:50040 --db-endpoint=${HostIP}:2379,${HostIP}:2380

docker run -d --net=host --privileged=true -v /etc/opensds:/etc/opensds -v /dev/:/dev/ -v /run/:/run/:shared -v /etc/localtime:/etc/localtime:ro -v /lib/modules:/lib/modules:ro opensdsio/opensds-dock:latest /usr/bin/osdsdock -logtostderr --api-endpoint=0.0.0.0:50050 --db-endpoint=${HostIP}:2379,${HostIP}:2380

Test

Download cli tool.

wget https://github.com/opensds/opensds/releases/download/v0.5.0/opensds-hotpot-v0.5.0-linux-amd64.tar.gz 
tar xvf opensds-hotpot-v0.5.0-linux-amd64.tar.gz 
cp opensds-hotpot-v0.5.0-linux-amd64/bin/* /usr/local/bin
chmod 755 /usr/local/bin/osdsctl

export OPENSDS_ENDPOINT=http://127.0.0.1:50040
export OPENSDS_AUTH_STRATEGY=noauth
osdsctl pool list

Create a default profile firstly.

osdsctl profile create '{"name": "default", "description": "default policy"}'

Create a volume.

osdsctl volume create 1 --name=test-001

List all volumes.

osdsctl volume list

Delete the volume.

osdsctl volume delete <your_volume_id>

After this is done, just enjoy it!