Kolla-ansible项目提供一个完整的Ansible Playbook&＃xff0c;来部署Docker的镜像&＃xff0c;再完成openstack组件的自动化部署。并提供all-in-one和multihost的环境。
源码地址&＃xff1a;https://github.com/openstack/kolla-ansible.git
Paste_Image.png
[metadata]
name &＃61; kolla-ansible // 项目名称
summary &＃61; Ansible Deployment of Kolla containers
description-file &＃61; README.rst
author &＃61; OpenStack
author-email &＃61; openstack-dev&＃64;lists.openstack.org
home-page &＃61; http://docs.openstack.org/developer/kolla-ansible/
license &＃61; Apache License, Version 2.0
classifier &＃61;Environment :: OpenStackIntended Audience :: Information TechnologyIntended Audience :: System AdministratorsLicense :: OSI Approved :: Apache Software LicenseOperating System :: POSIX :: LinuxProgramming Language :: PythonProgramming Language :: Python :: 2Programming Language :: Python :: 2.7Programming Language :: Python :: 3Programming Language :: Python :: 3.5[files]
packages &＃61; kolla_ansible //包名
data_files &＃61; //pbr方式打包对应的文件映射share/kolla-ansible/ansible &＃61; ansible/*share/kolla-ansible/tools &＃61; tools/validate-docker-execute.shshare/kolla-ansible/tools &＃61; tools/cleanup-containersshare/kolla-ansible/tools &＃61; tools/cleanup-hostshare/kolla-ansible/tools &＃61; tools/cleanup-imagesshare/kolla-ansible/tools &＃61; tools/stop-containers
share/kolla-ansible/doc &＃61; doc/*
share/kolla-ansible/etc_examples &＃61; etc/*
share/kolla-ansible &＃61; tools/init-runonce
share/kolla-ansible &＃61; tools/init-vpn
share/kolla-ansible &＃61; tools/openrc-example
share/kolla-ansible &＃61; setup.cfgscripts &＃61; //可执行脚本tools/kolla-ansible[entry_points]
console_scripts &＃61; //控制台可执行脚本&＃xff0c;执行两个Python文件的main函数
kolla-genpwd &＃61; kolla_ansible.cmd.genpwd:main
kolla-mergepwd &＃61; kolla_ansible.cmd.mergepwd:main[global]
setup-hooks &＃61;
pbr.hooks.setup_hook[pbr] //打包方式[build_sphinx]
all_files &＃61; 1
build-dir &＃61; doc/build
source-dir &＃61; doc[build_releasenotes]
all_files &＃61; 1
build-dir &＃61; releasenotes/build
source-dir &＃61; releasenotes/source
安装执行脚本&＃xff0c;通过pbr打包&＃xff0c;执行过程会读取setup.cfg配置&＃xff0c;还会安装同父目录下requirements.txt中的依赖。更多参考https://julien.danjou.info/blog/2017/packaging-python-with-pbr
import setuptools# In python <2.7.4, a lazy loading of package &＃96;pbr&＃96; will break
# setuptools if some other modules registered functions in &＃96;atexit&＃96;.
# solution from: http://bugs.python.org/issue15881#msg170215
try:import multiprocessing # noqa
except ImportError:passsetuptools.setup(setup_requires&＃61;[&＃39;pbr>&＃61;2.0.0&＃39;],pbr&＃61;True)
该脚本是封装了ansible-playbook&＃xff0c;对kolla进行了ansible的定制。主要根据action的类型&＃xff0c;传递不同的配置文件。
中间基础变量定义&＃xff1a;
find_base_dir
INVENTORY&＃61;"${BASEDIR}/ansible/inventory/all-in-one"
PLAYBOOK&＃61;"${BASEDIR}/ansible/site.yml"
VERBOSITY&＃61;
EXTRA_OPTS&＃61;${EXTRA_OPTS}
CONFIG_DIR&＃61;"/etc/kolla"
PASSWORDS_FILE&＃61;"${CONFIG_DIR}/passwords.yml"
DANGER_CONFIRM&＃61;
INCLUDE_IMAGES&＃61;
Find_base_dir是一个脚本开始时候的一个函数&＃xff08;不展开解释&＃xff09;&＃xff0c;用于找到kolla-ansible脚本所在的路径。
脚本传参解释&＃xff1a;
while [ "$#" -gt 0 ]; do
case "$1" in
(--inventory|-i)INVENTORY&＃61;"$2"shift 2;;(--playbook|-p)PLAYBOOK&＃61;"$2"shift 2;;(--tags|-t)EXTRA_OPTS&＃61;"$EXTRA_OPTS --tags $2"shift 2;;(--verbose|-v)VERBOSITY&＃61;"$VERBOSITY --verbose"shift 1;;(--configdir)CONFIG_DIR&＃61;"$2"shift 2;;(--yes-i-really-really-mean-it)DANGER_CONFIRM&＃61;"$1"shift 1;;(--include-images)INCLUDE_IMAGES&＃61;"$1"shift 1;;(--key|-k)VAULT_PASS_FILE&＃61;"$2"EXTRA_OPTS&＃61;"$EXTRA_OPTS --vault-password-file&＃61;$VAULT_PASS_FILE"shift 2;;(--extra|-e)EXTRA_OPTS&＃61;"$EXTRA_OPTS -e $2"shift 2;;(--passwords)PASSWORDS_FILE&＃61;"$2"shift 2;;(--help|-h)usageshiftexit 0;;(--)shiftbreak;;(*)echo "error"exit 3;;
esac
donecase "$1" in(prechecks)ACTION&＃61;"Pre-deployment checking"EXTRA_OPTS&＃61;"$EXTRA_OPTS -e action&＃61;precheck";;
(check)ACTION&＃61;"Post-deployment checking"EXTRA_OPTS&＃61;"$EXTRA_OPTS -e action&＃61;check";;
(mariadb_recovery)ACTION&＃61;"Attempting to restart mariadb cluster"EXTRA_OPTS&＃61;"$EXTRA_OPTS -e action&＃61;deploy -e common_run&＃61;true"PLAYBOOK&＃61;"${BASEDIR}/ansible/mariadb_recovery.yml";;
(destroy)ACTION&＃61;"Destroy Kolla containers, volumes and host configuration"PLAYBOOK&＃61;"${BASEDIR}/ansible/destroy.yml"if [[ "${INCLUDE_IMAGES}" &＃61;&＃61; "--include-images" ]]; thenEXTRA_OPTS&＃61;"$EXTRA_OPTS -e destroy_include_images&＃61;yes"fiif [[ "${DANGER_CONFIRM}" !&＃61; "--yes-i-really-really-mean-it" ]]; thencat <
EOFexit 1fi;;
(bootstrap-servers)ACTION&＃61;"Bootstraping servers"PLAYBOOK&＃61;"${BASEDIR}/ansible/kolla-host.yml"EXTRA_OPTS&＃61;"$EXTRA_OPTS -e action&＃61;bootstrap-servers";;
(deploy)ACTION&＃61;"Deploying Playbooks"EXTRA_OPTS&＃61;"$EXTRA_OPTS -e action&＃61;deploy";;
(deploy-bifrost)ACTION&＃61;"Deploying Bifrost"PLAYBOOK&＃61;"${BASEDIR}/ansible/bifrost.yml"EXTRA_OPTS&＃61;"$EXTRA_OPTS -e action&＃61;deploy";;
(deploy-servers)ACTION&＃61;"Deploying servers with bifrost"PLAYBOOK&＃61;"${BASEDIR}/ansible/bifrost.yml"EXTRA_OPTS&＃61;"$EXTRA_OPTS -e action&＃61;deploy-servers";;
(post-deploy)ACTION&＃61;"Post-Deploying Playbooks"PLAYBOOK&＃61;"${BASEDIR}/ansible/post-deploy.yml";;
(pull)ACTION&＃61;"Pulling Docker images"EXTRA_OPTS&＃61;"$EXTRA_OPTS -e action&＃61;pull";;
(upgrade)ACTION&＃61;"Upgrading OpenStack Environment"EXTRA_OPTS&＃61;"$EXTRA_OPTS -e action&＃61;upgrade -e serial&＃61;${ANSIBLE_SERIAL}";;
(reconfigure)ACTION&＃61;"Reconfigure OpenStack service"EXTRA_OPTS&＃61;"$EXTRA_OPTS -e action&＃61;reconfigure -e serial&＃61;${ANSIBLE_SERIAL}";;
(stop)ACTION&＃61;"Stop Kolla containers"PLAYBOOK&＃61;"${BASEDIR}/ansible/stop.yml";;
(certificates)ACTION&＃61;"Generate TLS Certificates"PLAYBOOK&＃61;"${BASEDIR}/ansible/certificates.yml";;
(genconfig)ACTION&＃61;"Generate configuration files for enabled OpenStack services"EXTRA_OPTS&＃61;"$EXTRA_OPTS -e action&＃61;config";;
(*) usageexit 0;;
esac
这段是根据传递的参数&＃xff0c;不同的参数针对不同的配置文件等额外属性。这里第一个参数有好多action&＃xff0c;如deploy&＃xff0c;post-deploy&＃xff0c;stop等等。
最后三行&＃xff0c;组合命令&＃xff0c;执行&＃xff1a;
CONFIG_OPTS&＃61;"-e &＃64;${CONFIG_DIR}/globals.yml -e &＃64;${PASSWORDS_FILE} -e CONFIG_DIR&＃61;${CONFIG_DIR}"
CMD&＃61;"ansible-playbook -i $INVENTORY $CONFIG_OPTS $EXTRA_OPTS $PLAYBOOK $VERBOSITY"
process_cmd
传递进来的参数组合成ansible-playbook的CMD命令后&＃xff0c;调用process_cmd函数执行。
由于openstack的组件比较多&＃xff0c;且大多数处于并列关系。这里不再一一展开&＃xff0c;以neutron为例进行解读。其他没有涉及的有重要的点&＃xff0c;会进行内容穿插。
该目录下是一些自定义的一些模块&＃xff0c;这些module在目标节点上运行&＃xff0c;包括bslurp.py&＃xff0c;kolla_container_facts.py&＃xff0c;kolla_docker.py&＃xff0c;kolla-toolbox.py&＃xff0c;merge_configs.py&＃xff0c;merge_yaml.py&＃xff0c;前四个我会一一介绍&＃xff0c;后两个是空文件&＃xff0c;代码其实在action_plugins目录中&＃xff08;那两个空文件是映射同名的action,我们可以向module一样使用action&＃xff09;。
Bslurp.py&＃xff0c;好像做了文件分发的事情&＃xff0c;通过copy_from_host&＃xff08;我没查到调用它的地方&＃xff09;和copy_to_host&＃xff08;在ceph角色部署的时候&＃xff0c;下发keyring到osd节点是通过这个模块函数下发的&＃xff09;两个函数实现&＃xff0c;。判断依据是模块参数dest。见代码中文注释。
def copy_from_host(module):#此处省略不少代码module.exit_json(content&＃61;base &＃xff21;64.b64encode(data), sha1&＃61;sha1, mode&＃61;mode,source&＃61;src)def copy_to_host(module):compress &＃61; module.params.get(&＃39;compress&＃39;)dest &＃61; module.params.get(&＃39;dest&＃39;)mode &＃61; int(module.params.get(&＃39;mode&＃39;), 0)sha1 &＃61; module.params.get(&＃39;sha1&＃39;)# src是加密后的数据src &＃61; module.params.get(&＃39;src&＃39;)# decode已经加密的数据data &＃61; base64.b64decode(src)
#解压数据raw_data &＃61; zlib.decompress(data) if compress else data#sha1安全算法数据校验if sha1:if os.path.exists(dest):if os.access(dest, os.R_OK):with open(dest, &＃39;rb&＃39;) as f:if hashlib.sha1(f.read()).hexdigest() &＃61;&＃61; sha1:module.exit_json(changed&＃61;False)else:module.exit_json(failed&＃61;True, changed&＃61;False,msg&＃61;&＃39;file is not accessible: {}&＃39;.format(dest))if sha1 !&＃61; hashlib.sha1(raw_data).hexdigest():module.exit_json(failed&＃61;True, changed&＃61;False,msg&＃61;&＃39;sha1 sum does not match data&＃39;)# 保存数据到dest值。这段代码有健壮性问题&＃xff0c;没有考虑到磁盘写满的场景&＃xff0c;保险的做法创建一个tmp文件&＃xff0c;把数据拷贝到tmp文件&＃xff0c;再把tmp文件重命名为dest值。否则容易把文件写空。with os.fdopen(os.open(dest, os.O_WRONLY | os.O_CREAT, mode), &＃39;wb&＃39;) as f:f.write(raw_data)#调用module要求的exit_json接口退出。module.exit_json(changed&＃61;True)def main():
# 定义dict类型的参数&＃xff0c;ansible module的接口要求argument_spec &＃61; dict(compress&＃61;dict(default&＃61;True, type&＃61;&＃39;bool&＃39;),dest&＃61;dict(type&＃61;&＃39;str&＃39;),mode&＃61;dict(default&＃61;&＃39;0644&＃39;, type&＃61;&＃39;str&＃39;),sha1&＃61;dict(default&＃61;None, type&＃61;&＃39;str&＃39;),src&＃61;dict(required&＃61;True, type&＃61;&＃39;str&＃39;))
# 创建ansible模块对象module &＃61; AnsibleModule(argument_spec)# 获取模块dest参数值dest &＃61; module.params.get(&＃39;dest&＃39;)try:if dest: # 如果dest参数存在&＃xff0c;则推送操作&＃xff0c;push下发到相应的hostcopy_to_host(module)else:# 如果dest参数不存在&＃xff0c;则进行pull操作。copy_from_host(module)except Exception:# 异常场景下退出&＃xff0c;ansible自定义模块语法规范。 module.exit_json(failed&＃61;True, changed&＃61;True,msg&＃61;repr(traceback.format_exc()))# import module snippets
from ansible.module_utils.basic import * # noqa
if __name__ &＃61;&＃61; &＃39;__main__&＃39;:main()
kolla_docker.py&＃xff0c;容器相关的操作&＃xff0c;openstack的组件都通过容器部署&＃xff0c;每个组件role的部署都会用到&＃xff0c;非常重要。
...
#创建docker的client函数
def get_docker_client():try:return docker.Clientexcept AttributeError:return docker.APIClient
Aclass DockerWorker(object):def __init__(self, module):# 构造函数&＃xff0c;传入参数是AnsibleModule类型的对象self.module &＃61; module# params参数续传self.params &＃61; self.module.paramsself.changed &＃61; False# TLS not fully implemented# tls_config &＃61; self.generate_tls()# 创建一个docker.client对象options &＃61; {&＃39;version&＃39;: self.params.get(&＃39;api_version&＃39;)}self.dc &＃61; get_docker_client()(**options)
# ....
# ....# 启动容器的函数&＃xff0c;是AnsibleModule的其中一种action
def start_container(self):#检查镜像是否存在&＃xff0c;不存在pullif not self.check_image():self.pull_image()#检查容器container &＃61; self.check_container()#容器异样&＃xff0c;则删除&＃xff0c;再回调if container and self.check_container_differs():self.stop_container()self.remove_container()container &＃61; self.check_container()#容器不存在&＃xff0c;创建&＃xff0c;再回调if not container:self.create_container()container &＃61; self.check_container()#容器状态非启动&＃xff0c;则启动if not container[&＃39;Status&＃39;].startswith(&＃39;Up &＃39;):self.changed &＃61; Trueself.dc.start(container&＃61;self.params.get(&＃39;name&＃39;))# We do not want to detach so we wait around for container to exit#如果container没有detach断开&＃xff0c;那么进入wait状态,调用fail_json方法&＃xff0c;传递fail的参数if not self.params.get(&＃39;detach&＃39;):rc &＃61; self.dc.wait(self.params.get(&＃39;name&＃39;))if rc !&＃61; 0:self.module.fail_json(failed&＃61;True,changed&＃61;True,msg&＃61;"Container exited with non-zero return code")#如果返回参数remove_on_exit&＃xff0c;那么删除该containerif self.params.get(&＃39;remove_on_exit&＃39;):self.stop_container()self.remove_container()def generate_module():# NOTE(jeffrey4l): add empty string &＃39;&＃39; to choices let us use# pid_mode: "{{ service.pid_mode | default (&＃39;&＃39;) }}" in yaml
#定义参数字典&＃xff0c;ansible module的api规范argument_spec &＃61; dict(common_options&＃61;dict(required&＃61;False, type&＃61;&＃39;dict&＃39;, default&＃61;dict()),#action参数&＃xff0c;必须传递&＃xff0c;类型为str,value值必须在choices的列表action&＃61;dict(required&＃61;True, type&＃61;&＃39;str&＃39;,choices&＃61;[&＃39;compare_container&＃39;, &＃39;compare_image&＃39;,&＃39;create_volume&＃39;, &＃39;get_container_env&＃39;,&＃39;get_container_state&＃39;, &＃39;pull_image&＃39;,&＃39;recreate_or_restart_container&＃39;,&＃39;remove_container&＃39;, &＃39;remove_volume&＃39;,&＃39;restart_container&＃39;, &＃39;start_container&＃39;,&＃39;stop_container&＃39;]),api_version&＃61;dict(required&＃61;False, type&＃61;&＃39;str&＃39;, default&＃61;&＃39;auto&＃39;),auth_email&＃61;dict(required&＃61;False, type&＃61;&＃39;str&＃39;),auth_password&＃61;dict(required&＃61;False, type&＃61;&＃39;str&＃39;),auth_registry&＃61;dict(required&＃61;False, type&＃61;&＃39;str&＃39;),auth_username&＃61;dict(required&＃61;False, type&＃61;&＃39;str&＃39;),detach&＃61;dict(required&＃61;False, type&＃61;&＃39;bool&＃39;, default&＃61;True),labels&＃61;dict(required&＃61;False, type&＃61;&＃39;dict&＃39;, default&＃61;dict()),name&＃61;dict(required&＃61;False, type&＃61;&＃39;str&＃39;),environment&＃61;dict(required&＃61;False, type&＃61;&＃39;dict&＃39;),image&＃61;dict(required&＃61;False, type&＃61;&＃39;str&＃39;),ipc_mode&＃61;dict(required&＃61;False, type&＃61;&＃39;str&＃39;, choices&＃61;[&＃39;host&＃39;, &＃39;&＃39;]),cap_add&＃61;dict(required&＃61;False, type&＃61;&＃39;list&＃39;, default&＃61;list()),security_opt&＃61;dict(required&＃61;False, type&＃61;&＃39;list&＃39;, default&＃61;list()),pid_mode&＃61;dict(required&＃61;False, type&＃61;&＃39;str&＃39;, choices&＃61;[&＃39;host&＃39;, &＃39;&＃39;]),privileged&＃61;dict(required&＃61;False, type&＃61;&＃39;bool&＃39;, default&＃61;False),graceful_timeout&＃61;dict(required&＃61;False, type&＃61;&＃39;int&＃39;, default&＃61;10),remove_on_exit&＃61;dict(required&＃61;False, type&＃61;&＃39;bool&＃39;, default&＃61;True),restart_policy&＃61;dict(required&＃61;False, type&＃61;&＃39;str&＃39;, choices&＃61;[&＃39;no&＃39;,&＃39;never&＃39;,&＃39;on-failure&＃39;,&＃39;always&＃39;,&＃39;unless-stopped&＃39;]),restart_retries&＃61;dict(required&＃61;False, type&＃61;&＃39;int&＃39;, default&＃61;10),tls_verify&＃61;dict(required&＃61;False, type&＃61;&＃39;bool&＃39;, default&＃61;False),tls_cert&＃61;dict(required&＃61;False, type&＃61;&＃39;str&＃39;),tls_key&＃61;dict(required&＃61;False, type&＃61;&＃39;str&＃39;),tls_cacert&＃61;dict(required&＃61;False, type&＃61;&＃39;str&＃39;),volumes&＃61;dict(required&＃61;False, type&＃61;&＃39;list&＃39;),volumes_from&＃61;dict(required&＃61;False, type&＃61;&＃39;list&＃39;))# 属性依赖ansible module的api规范&＃xff0c;如start_container这个action, #必须要image和name这个两个属性。required_if &＃61; [[&＃39;action&＃39;, &＃39;pull_image&＃39;, [&＃39;image&＃39;]],[&＃39;action&＃39;, &＃39;start_container&＃39;, [&＃39;image&＃39;, &＃39;name&＃39;]],[&＃39;action&＃39;, &＃39;compare_container&＃39;, [&＃39;name&＃39;]],[&＃39;action&＃39;, &＃39;compare_image&＃39;, [&＃39;name&＃39;]],[&＃39;action&＃39;, &＃39;create_volume&＃39;, [&＃39;name&＃39;]],[&＃39;action&＃39;, &＃39;get_container_env&＃39;, [&＃39;name&＃39;]],[&＃39;action&＃39;, &＃39;get_container_state&＃39;, [&＃39;name&＃39;]],[&＃39;action&＃39;, &＃39;recreate_or_restart_container&＃39;, [&＃39;name&＃39;]],[&＃39;action&＃39;, &＃39;remove_container&＃39;, [&＃39;name&＃39;]],[&＃39;action&＃39;, &＃39;remove_volume&＃39;, [&＃39;name&＃39;]],[&＃39;action&＃39;, &＃39;restart_container&＃39;, [&＃39;name&＃39;]],[&＃39;action&＃39;, &＃39;stop_container&＃39;, [&＃39;name&＃39;]]]#实例化
module &＃61; AnsibleModule(argument_spec&＃61;argument_spec,required_if&＃61;required_if,bypass_checks&＃61;False)#以下部分主要做环境变量和通用参数以及特殊参数的更新。new_args &＃61; module.params.pop(&＃39;common_options&＃39;, dict())# NOTE(jeffrey4l): merge the environmentenv &＃61; module.params.pop(&＃39;environment&＃39;, dict())if env:new_args[&＃39;environment&＃39;].update(env)for key, value in module.params.items():if key in new_args and value is None:continuenew_args[key] &＃61; value# if pid_mode &＃61; ""/None/False, remove itif not new_args.get(&＃39;pid_mode&＃39;, False):new_args.pop(&＃39;pid_mode&＃39;, None)# if ipc_mode &＃61; ""/None/False, remove itif not new_args.get(&＃39;ipc_mode&＃39;, False):new_args.pop(&＃39;ipc_mode&＃39;, None)module.params &＃61; new_args
# 返回为AnsibleModule实例return moduledef main():module &＃61; generate_module()try:dw &＃61; DockerWorker(module)# TODO(inc0): We keep it bool to have ansible deal with consistent# types. If we ever add method that will have to return some# meaningful data, we need to refactor all methods to return dicts.#返回值 result是action传递的函数名的运行成功与否的结果&＃xff0c;意义#不大result &＃61; bool(getattr(dw, module.params.get(&＃39;action&＃39;))())module.exit_json(changed&＃61;dw.changed, result&＃61;result)except Exception:module.exit_json(failed&＃61;True, changed&＃61;True,msg&＃61;repr(traceback.format_exc()))# import module snippets
from ansible.module_utils.basic import * # noqa
if __name__ &＃61;&＃61; &＃39;__main__&＃39;:main()
Kolla_toolbox.py&＃xff0c;在toolbox容器中运行ansible命令。这个我就不展开了&＃xff0c;只贴关键代码。
#生成commandline函数&＃xff0c;只包含ansible的命令
def gen_commandline(params):command &＃61; [&＃39;ansible&＃39;, &＃39;localhost&＃39;]........return command#主函数
def main():....
....client &＃61; get_docker_client()(version&＃61;module.params.get(&＃39;api_version&＃39;))
#调用函数&＃xff0c;获取命令&＃xff08;dict类型&＃xff09;command_line &＃61; gen_commandline(module.params)
#过滤名称为kolla_toolbox的容器列表kolla_toolbox &＃61; client.containers(filters&＃61;dict(name&＃61;&＃39;kolla_toolbox&＃39;,status&＃61;&＃39;running&＃39;))if not kolla_toolbox:module.fail_json(msg&＃61;&＃39;kolla_toolbox container is not running.&＃39;)#默认只有一个&＃xff0c;所有选了数组的第一个。kolla_toolbox变量名不建议重复使用&＃xff0c;开源代码就是坑。kolla_toolbox &＃61; kolla_toolbox[0]
#在容器中执行命令job &＃61; client.exec_create(kolla_toolbox, command_line)output &＃61; client.exec_start(job)
....
....module.exit_json(**ret)
****Kolla_container_facts.py**** 调用dockerclient的python接口获取指定容器的facts信息&＃xff0c;只传递一个name值即可。result类型是dict(changed&＃61;xxx, _containers&＃61;[]),代码不展开了。
该目录下记录了自定义的action的plugins&＃xff0c;这些plugins在master上运行。但也可以在library目录下定义同名空文件&＃xff0c;可以当作module使用。这里有两个代码文件&＃xff0c;merge_configs.py和merge_yaml.py&＃xff0c;用于conf和yml配置文件的合并。这里就分下下merge_yaml.py这个action plugin.
Merge_yaml.py&＃xff0c;在task的参数sources中传递多个yml文件&＃xff0c;合并之后输出到目标节点的dest中。期间在合并的同时&＃xff0c;进行了参数模拟变量的渲染工作&＃xff0c;最后调用copy模块把渲染后的数据文件复制过去。分析代码如下。
from ansible.plugins import action
#继承父类action.ActionBase
class ActionModule(action.ActionBase):TRANSFERS_FILES &＃61; Truedef read_config(self, source):result &＃61; None# Only use config if presentif os.access(source, os.R_OK):with open(source, &＃39;r&＃39;) as f:template_data &＃61; f.read()# 渲染template模板数据&＃xff0c;因为最终执行copy模块的时候&＃xff0c;# 变量被重新还原了&＃xff0c;所以这里要先template变量先渲染&＃xff0c;# 因为有些变量对可能在copy模块中会消失template_data &＃61; self._templar.template(template_data)#把YAML数据&＃xff0c;转化为dict对象result &＃61; safe_load(template_data)return result or {}# 自定义action plugin必须实现的方法def run(self, tmp&＃61;None, task_vars&＃61;None):#task_vars是这个task的一些外传入的变量,# 如host vars, group vars, config vars,etcif task_vars is None:task_vars &＃61; dict()#自定义action plugin必须调用父类的run方法result &＃61; super(ActionModule, self).run(tmp, task_vars)# NOTE(jeffrey4l): Ansible 2.1 add a remote_user param to the# _make_tmp_path function. inspect the number of the args here. In# this way, ansible 2.0 and ansible 2.1 are both supported#创建tmp临时目录&＃xff0c;兼容2.0以后的版本make_tmp_path_args &＃61; inspect.getargspec(self._make_tmp_path)[0]if not tmp and len(make_tmp_path_args) &＃61;&＃61; 1:tmp &＃61; self._make_tmp_path()if not tmp and len(make_tmp_path_args) &＃61;&＃61; 2:remote_user &＃61; (task_vars.get(&＃39;ansible_user&＃39;)or self._play_context.remote_user)tmp &＃61; self._make_tmp_path(remote_user)# save template args.# _task.args是这个task的参数&＃xff0c;这里把参数中key为vars的对应值# 保存到extra_vars变量中extra_vars &＃61; self._task.args.get(&＃39;vars&＃39;, list())# 备份template的可用变量old_vars &＃61; self._templar._available_variables# 将task_vars和extra_vars的所有变量merge到一起&＃xff0c;赋值到temp_varstemp_vars &＃61; task_vars.copy()temp_vars.update(extra_vars)#把最新的变量数据设置到templar对象&＃xff08;模板对象&＃xff09;self._templar.set_available_variables(temp_vars)output &＃61; {}# 获取task的参数为sources的values值,可能是单个文件&＃xff0c;# 也有可能是多个文件组成的listsources &＃61; self._task.args.get(&＃39;sources&＃39;, None)#非数组&＃xff0c;转化为只有一个item的数组if not isinstance(sources, list):sources &＃61; [sources]#便历sources数组&＃xff0c;读取文件中内容&＃xff0c;并合并更新#dict.update方式有去重效果&＃xff0c;相当于mergefor source in sources:output.update(self.read_config(source))# restore original vars#还原templar对象的变量self._templar.set_available_variables(old_vars)#把最新的合并好的数据output传递到远端的target host。复制给xfered变量remote_path &＃61; self._connection._shell.join_path(tmp, &＃39;src&＃39;)xfered &＃61; self._transfer_data(remote_path,dump(output,default_flow_style&＃61;False))#把本task的参数拷贝&＃xff0c;作为新模块的参数new_module_argsnew_module_args &＃61; self._task.args.copy()#更新new_module_args的src的值,后面copy模块的参数要求new_module_args.update(dict(src&＃61;xfered))#删除new_module_args的sources参数&＃xff0c;后面copy模块的参数要求del new_module_args[&＃39;sources&＃39;]#传入最新的参数new_module_args&＃xff0c;task_vars执行copy的moduleresult.update(self._execute_module(module_name&＃61;&＃39;copy&＃39;,module_args&＃61;new_module_args,task_vars&＃61;task_vars,tmp&＃61;tmp))#返回result&＃xff0c; action plugin 接口的要求return result
#control主机组包含本地localhost节点&＃xff0c;连接方式为local
[control]
localhost ansible_connection&＃61;local[network]
localhost ansible_connection&＃61;local#neutron主机组包含network组下的所有节点
[neutron:children]
network# Neutron
#neutron-server主机组包含control组下的所有节点
[neutron-server:children]
control#neutron-dhcp-agent主机组包含neutron组下的所有节点
[neutron-dhcp-agent:children]
neutron[neutron-l3-agent:children]
neutron[neutron-lbaas-agent:children]
neutron[neutron-metadata-agent:children]
neutron[neutron-vpnaas-agent:children]
neutron[neutron-bgp-dragent:children]
neutron
#调用ansible的setup获取节点的facts。gather_facts被设置为false是为了避免ansible再次去gathering facts.
- name: Gather facts for all hostshosts: allserial: &＃39;{{ serial|default("0") }}&＃39;gather_facts: falsetasks:- setup:tags: always# NOTE(pbourke): This case covers deploying subsets of hosts using --limit. The
# limit arg will cause the first play to gather facts only about that node,
# meaning facts such as IP addresses for rabbitmq nodes etc. will be undefined
# in the case of adding a single compute node.
# We don&＃39;t want to add the delegate parameters to the above play as it will
# result in ((num_nodes-1)^2) number of SSHs when running for all nodes
# which can be very inefficient.- name: Gather facts for all hosts (if using --limit)hosts: allserial: &＃39;{{ serial|default("0") }}&＃39;gather_facts: falsetasks:- setup:delegate_facts: Truedelegate_to: "{{ item }}"with_items: "{{ groups[&＃39;all&＃39;] }}"when:- (play_hosts | length) !&＃61; (groups[&＃39;all&＃39;] | length)#检测openstack_release全局变量信息&＃xff0c;默认在globals.yml是不配置的&＃xff0c;而
#在ansible/group_vars/all.yml中配置的默认值是auto。这里的两个tasks
#就是如果在auto的场景下&＃xff0c;通过python的pbr包去检测安装好的#kolla-ansible版本&＃xff0c;再将该版本号赋值给openstack_release变量。
#这里用到了ansible自带的local_action模块和register中间信息存储模块。
- name: Detect openstack_release variablehosts: allgather_facts: falsetasks:- name: Get current kolla-ansible version numberlocal_action: command python -c "import pbr.version; print(pbr.version.VersionInfo(&＃39;kolla-ansible&＃39;))"register: kolla_ansible_versionchanged_when: falsewhen: openstack_release &＃61;&＃61; "auto"- name: Set openstack_release variableset_fact:openstack_release: "{{ kolla_ansible_version.stdout }}"when: openstack_release &＃61;&＃61; "auto"tags: always#对所有节点进行recheck检查&＃xff0c;前提条件是ansible-playbook命令传递的#action的值是precheck。
- name: Apply role prechecksgather_facts: falsehosts:- allroles:- role: precheckswhen: action &＃61;&＃61; "precheck"# 基于ntp时间同步角色的部署&＃xff0c;hosts组为chrony-server和chrony
#前提条件是enable_chrony变量是否是yes&＃xff0c;该值可在etc/kolla/globals.yml
#中配置&＃xff0c;默认是no。
- name: Apply role chronygather_facts: falsehosts:- chrony-server- chronyserial: &＃39;{{ serial|default("0") }}&＃39;roles:- { role: chrony,tags: chrony,when: enable_chrony | bool }#部署neutron角色&＃xff0c;这里部署的节点除了neutron相关的host组之外&＃xff0c;还包括#compute和manila-share&＃xff08;openstack的一个文件共享组件&＃xff09;组。
- name: Apply role neutrongather_facts: falsehosts:- neutron-server- neutron-dhcp-agent- neutron-l3-agent- neutron-lbaas-agent- neutron-metadata-agent- neutron-vpnaas-agent- compute- manila-shareserial: &＃39;{{ serial|default("0") }}&＃39;roles:- { role: neutron,tags: neutron,when: enable_neutron | bool }
该场景的action是precheck。由tasks/main.yml引用precheck.yml。
---
# kolla_container_facts是自定义的library&＃xff0c;上文已经分析过代码&＃xff0c;
# 用于获取容器名为neutron_server的一些容器属性数据&＃xff0c;注册到中间变量container_facts
- name: Get container factskolla_container_facts:name:- neutron_serverregister: container_facts# 中间变量container_facts没有找到neutron_server关键字且该主机在neutron-server主机组中&＃xff0c;
# 判断neutron_server_port 端口是否已经stopped
- name: Checking free port for Neutron Serverwait_for:host: "{{ hostvars[inventory_hostname][&＃39;ansible_&＃39; &＃43; api_interface][&＃39;ipv4&＃39;][&＃39;address&＃39;] }}"port: "{{ neutron_server_port }}"connect_timeout: 1timeout: 1state: stoppedwhen:- container_facts[&＃39;neutron_server&＃39;] is not defined- inventory_hostname in groups[&＃39;neutron-server&＃39;]# enable_neutron_agent_ha是true,且只规划了多个一个dhcp和l3服务节点&＃xff0c;给出fail提示
- name: Checking number of network agentslocal_action: fail msg&＃61;"Number of network agents are less than two when enabling agent ha"changed_when: falsewhen:- enable_neutron_agent_ha | bool- groups[&＃39;neutron-dhcp-agent&＃39;] | length <2or groups[&＃39;neutron-l3-agent&＃39;] | length <2# When MountFlags is set to shared, a signal bit configured on 20th bit of a number
# We need to check the 20th bit. 2^20 &＃61; 1048576. So we are validating against it.
# 检查docker服务的MountFlags是否设置为了shared
- name: Checking if &＃39;MountFlags&＃39; for docker service is set to &＃39;shared&＃39;command: systemctl show dockerregister: resultchanged_when: falsefailed_when: result.stdout.find(&＃39;MountFlags&＃61;1048576&＃39;) &＃61;&＃61; -1when:- (inventory_hostname in groups[&＃39;neutron-dhcp-agent&＃39;]or inventory_hostname in groups[&＃39;neutron-l3-agent&＃39;]or inventory_hostname in groups[&＃39;neutron-metadata-agent&＃39;])- ansible_os_family &＃61;&＃61; &＃39;RedHat&＃39; or ansible_distribution &＃61;&＃61; &＃39;Ubuntu&＃39;
该场景的action是deploy。由tasks/main.yml引用deploy.yml
# enforce ironic usage only with openvswitch
# 裸机部署检查&＃xff0c;检查ironic服务必须启动&＃xff0c;neutron的plugin必须使用OpenvSwitch- include: ironic-check.yml#在neutron-server的节点执行注册
- include: register.ymlwhen: inventory_hostname in groups[&＃39;neutron-server&＃39;]#执行配置&＃xff0c;拷贝配置文件&＃xff0c;启动组件容器主要都在这里实现
- include: config.yml#在nova fake driver模拟场景下&＃xff0c;计算节点执行config-neutron-fake.yml&＃xff0c;不详细分析
#nova fake driver可以在单个计算节点中创建多个docker容器运行novc-compute&＃xff0c;
#Nova fake driver can not work with all-in-one deployment. This is because the fake
#neutron-openvswitch-agent for the fake nova-compute container conflicts with
#neutron-openvswitch-agent on the compute nodes. Therefore, in the inventory
#the network node must be different than the compute node.
- include: config-neutron-fake.ymlwhen:- enable_nova_fake | bool- inventory_hostname in groups[&＃39;compute&＃39;]#在neutron-server的节点执行创建数据库&＃xff0c;创建容器
#bootstrap.yml会去创建数据库相关信息&＃xff0c;结束后会去调用#bootstrap_servcie.yml该文件是用于在server节点上创建容器。
- include: bootstrap.ymlwhen: inventory_hostname in groups[&＃39;neutron-server&＃39;]#执行handlers目录下的task任务
- name: Flush Handlersmeta: flush_handlers
Register.yml, 往keystone中注册neutron服务的鉴权相关信息。
---
# 在keystone创建neutron的service 和endpoint
# kolla_toolbox见library分析&＃xff0c;用于在toolbox容器中执行ansible命令
# kolla_keystone_service模块是kolla-ansible的父项目kolla中的代码&＃xff0c;已经是一个可调用的ansible模块
#service名称为neutron,对应的endpoint分为内部&＃xff0c;管理员&＃xff0c;公共三个。
# 变量主要在ansible/role/neutron/defauts/main.yml和ansible/group_vars/all.yml中
- name: Creating the Neutron service and endpointkolla_toolbox:module_name: "kolla_keystone_service"module_args:service_name: "neutron"service_type: "network"description: "Openstack Networking"endpoint_region: "{{ openstack_region_name }}"url: "{{ item.url }}"interface: "{{ item.interface }}"region_name: "{{ openstack_region_name }}"auth: "{{ &＃39;{{ openstack_neutron_auth }}&＃39; }}"module_extra_vars:openstack_neutron_auth: "{{ openstack_neutron_auth }}"run_once: Truewith_items:- {&＃39;interface&＃39;: &＃39;admin&＃39;, &＃39;url&＃39;: &＃39;{{ neutron_admin_endpoint }}&＃39;}- {&＃39;interface&＃39;: &＃39;internal&＃39;, &＃39;url&＃39;: &＃39;{{ neutron_internal_endpoint }}&＃39;}- {&＃39;interface&＃39;: &＃39;public&＃39;, &＃39;url&＃39;: &＃39;{{ neutron_public_endpoint }}&＃39;}# 同上&＃xff0c;创建项目&＃xff0c;用户&＃xff0c;角色。经分析openstack_neutron_auth变量实际为#openstack的admin的auth。
- name: Creating the Neutron project, user, and rolekolla_toolbox:module_name: "kolla_keystone_user"module_args:project: "service"user: "{{ neutron_keystone_user }}"password: "{{ neutron_keystone_password }}"role: "admin"region_name: "{{ openstack_region_name }}"auth: "{{ &＃39;{{ openstack_neutron_auth }}&＃39; }}"module_extra_vars:openstack_neutron_auth: "{{ openstack_neutron_auth }}"run_once: True
Config.yml&＃xff0c;配置文件合并下发&＃xff0c;创建或重启容器。
#调用sysctl模块&＃xff0c;配置ip转发相关配置
- name: Setting sysctl valuesvars:neutron_l3_agent: "{{ neutron_services[&＃39;neutron-l3-agent&＃39;] }}"neutron_vpnaas_agent: "{{ neutron_services[&＃39;neutron-vpnaas-agent&＃39;] }}"sysctl: name&＃61;{{ item.name }} value&＃61;{{ item.value }} sysctl_set&＃61;yeswith_items:- { name: "net.ipv4.ip_forward", value: 1}- { name: "net.ipv4.conf.all.rp_filter", value: 0}- { name: "net.ipv4.conf.default.rp_filter", value: 0}when:- set_sysctl | bool- (neutron_l3_agent.enabled | bool and neutron_l3_agent.host_in_groups | bool)or (neutron_vpnaas_agent.enabled | bool and neutron_vpnaas_agent.host_in_groups | bool)# 创建neutron各服务的配置文件目录&＃xff0c;前提条件主要看host_in_groups变量&＃xff0c;这个在
# ansible/role/neutron/defauts/main.yml文件中进行了详细的定义
- name: Ensuring config directories existfile:path: "{{ node_config_directory }}/{{ item.key }}"state: "directory"recurse: yeswhen:- item.value.enabled | bool- item.value.host_in_groups | boolwith_dict: "{{ neutron_services }}"....
....#下发配置文件到指定目录&＃xff0c;三份陪配置文件合一&＃xff0c;merge_conifgs模块在action plugin中已经分析过了
#文件下发完了之后&＃xff0c;通知相应组件的容器重启。在handlers目录下。
# 重启容器这个操作会调用recreate_or_restart_container这个action&＃xff0c;第一次会创建容器。
- name: Copying over neutron_lbaas.confvars:service_name: "{{ item.key }}"services_need_neutron_lbaas_conf:- "neutron-server"- "neutron-lbaas-agent"merge_configs:sources:- "{{ role_path }}/templates/neutron_lbaas.conf.j2"- "{{ node_custom_config }}/neutron/neutron_lbaas.conf"- "{{ node_custom_config }}/neutron/{{ inventory_hostname }}/neutron_lbaas.conf"dest: "{{ node_config_directory }}/{{ item.key }}/neutron_lbaas.conf"register: neutron_lbaas_confswhen:- item.value.enabled | bool- item.value.host_in_groups | bool- item.key in services_need_neutron_lbaas_confwith_dict: "{{ neutron_services }}"notify:- "Restart {{ item.key }} container"....
....
#kolla_docker是自定义的模块&＃xff0c;通过调用compare_container查找该节点的所有neutron service容器
- name: Check neutron containerskolla_docker:action: "compare_container"common_options: "{{ docker_common_options }}"name: "{{ item.value.container_name }}"image: "{{ item.value.image }}"privileged: "{{ item.value.privileged | default(False) }}"volumes: "{{ item.value.volumes }}"register: check_neutron_containerswhen:- action !&＃61; "config"- item.value.enabled | bool- item.value.host_in_groups | boolwith_dict: "{{ neutron_services }}"notify:- "Restart {{ item.key }} container"
Bootstrap.yml&＃xff0c;创建neutron数据库对象。
---
# kolla_toolbox自定义模块&＃xff0c;在toolbox容器中调用mysql_db的ansible模块&＃xff0c;创建db
# delegate_to指定在第一个neutron-server上执行&＃xff0c;run_onece只运行一次
- name: Creating Neutron database
kolla_toolbox:
module_name: mysql_db
module_args:
login_host: "{{ database_address }}"
login_port: "{{ database_port }}"
login_user: "{{ database_user }}"
login_password: "{{ database_password }}"
name: "{{ neutron_database_name }}"
register: database
run_once: True
delegate_to: "{{ groups[&＃39;neutron-server&＃39;][0] }}"
# 创建neuron数据库的用户并设置权限
- name: Creating Neutron database user and setting permissionskolla_toolbox:module_name: mysql_usermodule_args:login_host: "{{ database_address }}"login_port: "{{ database_port }}"login_user: "{{ database_user }}"login_password: "{{ database_password }}"name: "{{ neutron_database_name }}"password: "{{ neutron_database_password }}"host: "%"priv: "{{ neutron_database_name }}.*:ALL"append_privs: "yes"run_once: Truedelegate_to: "{{ groups[&＃39;neutron-server&＃39;][0] }}"#数据库改变之后&＃xff0c;调用bootstrap_service.yml
- include: bootstrap_service.ymlwhen: database.changed
Bootstrap_service.yml&＃xff0c;创建bootrsap_neutron&＃xff0c;bootrsap_neutron_lbassd-agent&＃xff0c;bootrsap_neutron_vpnaas_agent容器。
Ansible/role/neutron/handlers/main.yml&＃xff0c;重建或重启neutron相关容器。列出一个分析下。
# 举例分析&＃xff1a;neutron_lbaas_confs变量是之前执行的时候注册的变量&＃xff0c;这里的task变量
# neutron_lbaas_conf从neutron_lbaas_confs的结果中取值分析作为when的条件判断
# kolla_docker模块的volumes参数是从role/neutron/defaults/main.yml中获取&＃xff0c;如下&＃xff0c;
#volumes:
# - "{{ node_config_directory }}/neutron-lbaas-agent/:{{
# container_config_directory }}/:ro"
# - "/etc/localtime:/etc/localtime:ro"
# - "/run:/run:shared"
# - "kolla_logs:/var/log/kolla/"
#且在config.yml已经把neutron_lbaas的配置文件下发{{ node_config_directory }}/
# neutron-lbaas-agent/这个目录系了。 容器中的路径container_config_directory变量
# 可以在groups_var中找到,值为 /var/lib/kolla/config_files。
- name: Restart neutron-server containervars:service_name: "neutron-server"service: "{{ neutron_services[service_name] }}"config_json: "{{ neutron_config_jsons.results|selectattr(&＃39;item.key&＃39;, &＃39;equalto&＃39;, service_name)|first }}"neutron_conf: "{{ neutron_confs.results|selectattr(&＃39;item.key&＃39;, &＃39;equalto&＃39;, service_name)|first }}"neutron_lbaas_conf: "{{ neutron_lbaas_confs.results|selectattr(&＃39;item.key&＃39;, &＃39;equalto&＃39;, service_name)|first }}"neutron_ml2_conf: "{{ neutron_ml2_confs.results|selectattr(&＃39;item.key&＃39;, &＃39;equalto&＃39;, service_name)|first }}"policy_json: "{{ policy_jsons.results|selectattr(&＃39;item.key&＃39;, &＃39;equalto&＃39;, service_name)|first }}"neutron_server_container: "{{ check_neutron_containers.results|selectattr(&＃39;item.key&＃39;, &＃39;equalto&＃39;, service_name)|first }}"kolla_docker:action: "recreate_or_restart_container"common_options: "{{ docker_common_options }}"name: "{{ service.container_name }}"image: "{{ service.image }}"volumes: "{{ service.volumes }}"privileged: "{{ service.privileged | default(False) }}"when:- action !&＃61; "config"- service.enabled | bool- service.host_in_groups | bool- config_json | changedor neutron_conf | changedor neutron_lbaas_conf | changedor neutron_vpnaas_conf | changedor neutron_ml2_conf | changedor policy_json | changedor neutron_server_container | changed
Pull.yml 下拉镜像
#调用kolla_docker的pull_image 下拉镜像
- name: Pulling neutron images
kolla_docker:
action: "pull_image"
common_options: "{{ docker_common_options }}"
image: "{{ item.value.image }}"
when:
- item.value.enabled | bool
- item.value.host_in_groups | bool
with_dict: "{{ neutron_services }}"
kolla-ansible\ansible\roles\chrony\defaults\main.yml文件中的一段&＃xff0c;如下:
#docker_registry是本地注册的registry地址&＃xff0c;在global.yml中会配置。Registry虚拟机为会从定向5000端口带宿主机的4000端口。
docker_namespace也在global.yml定义&＃xff0c;如果从官网下载源码镜像的话&＃xff0c;配置成lokolla。
kolla_base_distro默认centos&＃xff0c;也可以配置成ubuntu
kolla_install_type 默认为binary, 我们配置在global.yml配置成了source
例子&＃xff1a;
docker_registry&＃xff1a;192.168.102.15:4000
docker_namespace&＃xff1a;lokolla
kolla_base_distro&＃xff1a;“centos”
kolla_install_type: source
openstack_release&＃xff1a; auto (自发现&＃xff0c;前文有讲到)
所以最后的chrony_image_full为192.168.102.15:4000/lokolla/centos-source-chrony:4.0.2chrony_image: "{{ docker_registry ~ &＃39;/&＃39; if docker_registry else &＃39;&＃39; }}{{ docker_namespace }}/{{ kolla_base_distro }}-{{ kolla_install_type }}-chrony"
chrony_tag: "{{ openstack_release }}"
chrony_image_full: "{{ chrony_image }}:{{ chrony_tag }}"
作者&＃xff1a;JohnLee1100
链接&＃xff1a;https://www.jianshu.com/p/d66d0f61ad34
京公网安备 11010802041100号 | 京ICP备19059560号-4 | PHP1.CN 第一PHP社区 版权所有 