CEPHSTORA-222 Use Ansible 2.5.7.0 and ceph-ansible 3.1

This PR changes to use Ansible 2.5.7.0 (the latest Ansible 2.5 version)
along with ceph-ansible 3.1

To get this working we clean up some unrequired references in
ansible.cfg, adjust the bootstrap-ansible.sh file and adjust the plugins
and ceph-ansible role versions.

The master branch of OSA-plugins includes a connection module that will
work with ansible 2.5+, whereas stable/queens does not. Additionally,
the config_template module required for ceph-ansible has been updated in
ceph-ansible stable-3.1 so we no longer need the plugins directory for
prod and should move it to the testing roles to clone.

The openstack_hosts role also has some changes that are required for
Ansible 2.5 to run in our keystone_rgw scenario test. This is a testing
only role so we can bump this to master safely.
8f7114732df71249a907fa738da3c6b40492b50d is required for
openstack_hosts to work with Ansible 2.5+

Finally, this PR adjusts the update_dependency job so that v3.1 of
ceph-ansible is used as the base, and skips the
openstack-ansible-plugins repo.

README.md

@@ -8,9 +8,9 @@ versions of ``ceph-ansible`` used in RPC deployments.
 
 ## Current versions of ceph-ansible & Ansible
 
-### **ceph-ansible version:** v3.0.39
+### **ceph-ansible version:** 3.1.0
 
-### **Ansible version:** 2.4.4.0
+### **Ansible version:** 2.5.7.0
 
 ## What is rpc-ceph?
 

ansible-role-requirements.yml

@@ -1,7 +1,7 @@
 - name: ceph-ansible
   scm: git
   src: https://github.com/ceph/ceph-ansible
-  version: v3.0.39
+  version: stable-3.1
 - name: rsyslog_client
   scm: git
   src: https://git.openstack.org/openstack/openstack-ansible-rsyslog_client
@@ -30,7 +30,3 @@
   scm: git
   src: https://github.com/cloudalchemy/ansible-alertmanager
   version: 0.13.1
-- name: ../ceph_plugins
-  scm: git
-  src: https://git.openstack.org/openstack/openstack-ansible-plugins
-  version: 17.0.4

ansible.cfg

@@ -2,15 +2,8 @@
 
 library            = /etc/ansible/ceph_roles/ceph-ansible/library:./library:/etc/ansible/ceph_plugins/library
 # set plugin path directories here, separate with colons
-action_plugins     = /etc/ansible/ceph_plugins/action:/etc/ansible/ceph_roles/ceph-ansible/plugins/actions
-cache_plugins      = /etc/ansible/ceph_plugins/cache
+action_plugins     = /etc/ansible/ceph_roles/ceph-ansible/plugins/actions:/etc/ansible/ceph_plugins/action
 callback_plugins   = /etc/ansible/ceph_roles/ceph-ansible/plugins/callback:/etc/ansible/ceph_plugins/callback
 connection_plugins = /etc/ansible/ceph_plugins/connection
-lookup_plugins     = /etc/ansible/ceph_plugins/lookup
-inventory_plugins  = /etc/ansible/ceph_plugins/inventory
-vars_plugins       = /etc/ansible/ceph_plguins/vars_plugins
 filter_plugins     = /etc/ansible/ceph_plugins/filter
-test_plugins       = /etc/ansible/ceph_plugins/test
-terminal_plugins   = /etc/ansible/ceph_plugins/terminal
-strategy_plugins   = /etc/ansible/ceph_plugins/strategy
 roles_path         = /etc/ansible/ceph_roles/ceph-ansible/roles:/etc/ansible/ceph_roles:/etc/ansible/roles

gating/update_dependencies/run

@@ -4,7 +4,7 @@ set -x
 ## For use when updating the versions of upstream projects inside rpc-ceph.
 ## NB This will NOT upgrade a deployed environment, and should not be used in
 ## production
-CEPH_MAJOR_VERSION="v3\.0"
+CEPH_MAJOR_VERSION="v3\.1"
 OSA_MAJOR_VERSION="17\."
 
 function update_ceph_ansible {
@@ -37,7 +37,8 @@ function update_upstream_osa_roles {
   ## Update both the roles and test-roles
   for role_reqs_file in ansible-role-requirements.yml tests/ansible-role-test-requirements.yml; do
     ## For each of the openstack-ansible repos - exclude openstack-ansible-tests which has no branches
-    for repo in $(sed -n -e 's/^.*src: //p' ${role_reqs_file} | grep openstack-ansible | grep -v openstack-ansible-tests); do
+    ## Additionally exclude openstack-ansible-plugins and openstack-ansible-hosts due to no support for Ansible 2.5 on a stable branch (until Rocky)
+    for repo in $(sed -n -e 's/^.*src: //p' ${role_reqs_file} | grep openstack-ansible | grep -v openstack-ansible-tests | grep -v openstack-ansible-plugins | grep -v openstack_hosts); do
       ## Get the latest tag and save that
       LATEST_TAG=$(git ls-remote --tags $repo | grep "${OSA_MAJOR_VERSION}" | grep -v '{}' | cut -d/ -f 3 | sort --version-sort | tail -n 1)
       ## Update repos

phobos/README.md

@@ -125,8 +125,8 @@ From the root of the rpc-ceph repo clone:
 * Optionally set `rpc_ceph_version` to a specific version of rpc-ceph or a pull request ref you would like to test
 
 ```bash
-$ pip install -r requirements.txt ansible==2.4.4.0
-$ ansible-playbook -e cluster_deploy_version=example-v01 \
+$ pip install -r requirements.txt ansible==2.5.7.0
+$ ansible-playbook -e cluster_deploy_version=perf-v01 \
                    -e ssh_keyname=mykey \
                    -e stor_count=3 \
                    -e client_count=1 \

scripts/bootstrap-ansible.sh

@@ -1,6 +1,6 @@
 set -e -u -x
 
-export ANSIBLE_PACKAGE=${ANSIBLE_PACKAGE:-"ansible==2.4.4.0"}
+export ANSIBLE_PACKAGE=${ANSIBLE_PACKAGE:-"ansible==2.5.7.0"}
 export SSH_DIR=${SSH_DIR:-"/root/.ssh"}
 export ANSIBLE_ROLE_FILE=${ANSIBLE_ROLE_FILE:-"ansible-role-requirements.yml"}
 # Set the role fetch mode to any option [git-clone]

tests/ansible-role-test-requirements.yml

@@ -41,8 +41,12 @@
 - name: openstack_hosts
   scm: git
   src: https://git.openstack.org/openstack/openstack-ansible-openstack_hosts
-  version: 17.0.4
+  version: master
 - name: memcached_server
   scm: git
   src: https://git.openstack.org/openstack/openstack-ansible-memcached_server
   version: 17.0.4
+- name: ../ceph_plugins
+  scm: git
+  src: https://git.openstack.org/openstack/openstack-ansible-plugins
+  version: master