第三十章Centos7下Docker安装Airflow

一、拉取Airflow镜像文件

[root@prd-airflow001 ~]$ docker pull puckel/docker-airflow


二、创建挂载目录并授权

[root@prd-airflow001 ~]$ mkdir -p /data/airflow_data
[root@prd-airflow001 ~]$ chmod 777 /data/airflow_data -R


三、创建并启动Airflow容器

[root@prd-airflow001 ~]$ docker run \
-itd \
-p 8080:8080 \
-v /data/airflow_data:/usr/local/airflow \
-v /etc/localtime:/etc/localtime \
--restart always \
--privileged=true \
--name prd-airflow \
puckel/docker-airflow


四、查看Airflow是否启动成功

[root@m1 ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d52953b79600 puckel/docker-airflow "/entrypoint.sh webs…" 7 second ago Up 7 second 5555/tcp, 8793/tcp, 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp prd-airflow


五、配置airflow

#1.进入容器
[root@prd-airflow001 ~]$ docker exec -it d52953b79600 bash
#2.初始化数据库
airflow@d52953b79600:~$ airflow initdb
出现错误：
airflow.exceptions.AirflowException: Could not create Fernet object: Incorrect padding
解决办法：
python -c "from cryptography.fernet import Fernet;print(Fernet.generate_key().decode())"
oNu9XwewQNyx9mAJT2vZvtm3qzPRZIWRqwk9hSVch4A=
export AIRFLOW__CORE__FERNET_KEY=oNu9XwewQNyx9mAJT2vZvtm3qzPRZIWRqwk9hSVch4A=
#3.安装相关使用插件
airflow@d52953b79600:~$ pip install pymysql
airflow@d52953b79600:~$ pip install clickhouse_driver
#4.启动调度服务
airflow@d52953b79600:~$ airflow scheduler
airflow@d52953b79600:~$ nohup airflow scheduler >>airflow-scheduler.log 2>& 1 &
airflow@d52953b79600:~$ exit


六、验证连接

#1.登录ip+port 发现调度可以正常启动。