使用Gearman+Calabash并行测试手机APP

摘要&＃xff1a;使用任务分发系统Gearman分布式执行Calabash的自动化测试用例&＃xff0c;可以达到并行测试手机APP的目的。

Gearman

Gearman是一个分发任务的程序框架&＃xff0c;可以用在各种场合&＃xff0c;与Hadoop相比&＃xff0c;Gearman更偏向于任务分发功能。它的 任务分布非常简单&＃xff0c;简单得可以只需要用脚本即可完成。

Calabash

Calabash-android是支持android的UI自动化测试框架&＃xff0c;PC端使用了cucumber框架&＃xff0c;通过http和json与模拟器和真机上安装的测试apk通信&＃xff0c;测试apk调用robotium的方法来进行UI自动化测试&＃xff0c;支持webview操作。

calabash脚本以 使用calabash测试开源中国Android客户端 为例。

Ubuntu上安装Gearman

$ sudo apt-get install gearman-job-server
$ gearmand -Vgearmand 1.0.6 - https://bugs.launchpad.net/gearmand $ sudo apt-get install gearman-tools
$ gearman -H


运行Gearman并发执行Calabash测试用例

启动gearman job server&＃xff0c;作为后台服务运行:

$ gearmand -d


启动两个worker&＃xff0c;每个woker代表一个手机测试设备(tester)&＃xff0c;woker始终不退出&＃xff0c;且没有任何打印:

$ gearman -w -f tester -- ./test-case.sh NEXUS-7 testdata-aa.bashrc
$ gearman -w -f tester -- ./test-case.sh HUAHEI-8860 testdata-bb.bashrc


分发测试用例到各个worker&＃xff0c;收集到所有测试打印后&＃xff0c;即完成测试&＃xff1a;

./run-suite.sh &＃64;BVT


实时查看测试结果&＃xff0c;任何一个worker测试完一个用例&＃xff0c;测试结果会立即打印出来&＃xff1a;

tail -f /tmp/test-result


并发测试的效果&＃xff1a; 理论上如果顺序执行测试所有用例要10个小时&＃xff0c;那么只要准备10个手机&＃xff0c;每个手机对应启动一个worker:tester&＃xff0c;测试时间将缩短到最少1个小时。

下面的脚本是在同一台Ubuntu电脑上&＃xff0c;利用多个USB口连接多个手机来并发测试&＃xff0c; 如果要在多台电脑上执行并发测试&＃xff0c;那要考虑这多台电脑如何获取同一份apk和calabash脚本&＃xff0c;可以考虑从一个公共的url去wget。

test-case.sh

从stdin接收到的calabash测试脚本文件路径&＃xff0c;调用calabash-android完成测试&＃xff0c;再作为client调用printer

#!/bin/bash
if [ x$2 &＃61;&＃61; x ]
thenecho "usage: $0 device-id testdata-xx.bashrc"echo " device-id get from: adb devices"exit 1
fi
read line
echo $0 $1 $2
uuid&＃61;&＃96;uuidgen&＃96;
d1&＃61;&＃96;date &＃43;%T&＃96;
suite_path&＃61;$HOME/git/oschina/android-app/calabash
apk_path&＃61;$HOME/git/oschina/android-app/bin/oschina-android-app.apk
echo "test result in /tmp/${uuid}"
echo $d1 at &＃96;hostname&＃96;:$1 >> /tmp/${uuid}
mkdir -p $HOME/$1
cd $HOME/$1
export ADB_DEVICE_ARG&＃61;$1
. $src/$2
calabash-android run $apk_path -r $suite_path/features/ $line 2>&1 | tee -a /tmp/${uuid}
d2&＃61;&＃96;date &＃43;%T&＃96;
echo $d2 at &＃96;hostname&＃96;:$1 >> /tmp/${uuid}
failed&＃61;&＃96;grep "Failing Scenarios" /tmp/${uuid} | wc -l&＃96;
if [ $failed &＃61;&＃61; 0 ]
thenecho "&＃61;&＃61;&＃61;PASS&＃61;&＃61;&＃61; : $line" >> /tmp/${uuid}
elseecho "&＃61;&＃61;&＃61;FAIL&＃61;&＃61;&＃61; : $line" >> /tmp/${uuid}
fi
gearman -f printer rm -f /tmp/${uuid}


print-result.sh

从stdin接收测试结果&＃xff0c;并添加到文件/tmp/test-result

#!/bin/bash
while read line
doecho "$line" >> /tmp/test-result
done
case_count&＃61;&＃96;cat /tmp/case_count&＃96;
finished&＃61;&＃96;egrep "&＃61;&＃61;&＃61;(PASS|FAIL)&＃61;&＃61;&＃61;" /tmp/test-result | wc -l&＃96;
echo -n "Progress:" $finished "/ $case_count [" >> /tmp/test-resultfor ((i&＃61;0; i<$finished; i&＃43;&＃43; ))
doecho -n ">" >> /tmp/test-result
done
let unfinished&＃61;$case_count-$finished
for ((i&＃61;0; i<$unfinished; i&＃43;&＃43; ))
doecho -n "." >> /tmp/test-result
done
echo -en "]\n" >> /tmp/test-result


run-suite.sh

作为client&＃xff0c;后台一次分发所有calabash脚本到tester&＃xff0c;并启动woker: printer&＃xff0c;直到收到所有测试结果打印才退出

#!/bin/bash
if [ x$1 &＃61;&＃61; x ]
thenecho "usage:" $0 "&＃64;BVT|&＃64;nightly|all|failed"exit 1
fisuite_path&＃61;$HOME/git/oschina/android-app/calabash
uuid&＃61;&＃96;uuidgen&＃96;if [ "$1" &＃61;&＃61; "all" ]
thenfind $suite_path -name "*.feature" > /tmp/${uuid}
elif [ "$1" &＃61;&＃61; "failed" ]
thenif [ -f /tmp/failed ]thencat /tmp/failed > /tmp/${uuid}elsetouch /tmp/${uuid}fi
elif [[ "$1" &＃61;&＃61; &＃64;* ]]
thengrep $1 $suite_path -rl | grep ".feature$" > /tmp/${uuid}
ficase_count&＃61;&＃96;cat /tmp/${uuid} | wc -l&＃96;
echo $case_count > /tmp/case_count
if [ $case_count &＃61;&＃61; 0 ]
thenecho "NO case to run, exit." | tee /tmp/test-resultrm -f /tmp/${uuid}exit 1
fiecho "total $case_count cases." | tee /tmp/test-resulti&＃61;0
while read line
dolet i&＃43;&＃61;1echo "send No.$i case --- $line"echo $line | gearman -b -f runcase
done rm -f /tmp/${uuid}gearman -w -c $case_count -f printer -- ./print-result.sh $case_count
cat /tmp/test-result
grep "&＃61;&＃61;&＃61;FAIL&＃61;&＃61;&＃61;" /tmp/test-result | awk &＃39;{print $3}&＃39; > /tmp/failed
echo "" > /tmp/summary
echo "&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;test result summary&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;" 2>&1 | tee -a /tmp/summary
grep "&＃61;&＃61;&＃61;PASS&＃61;&＃61;&＃61;" /tmp/test-result 2>&1 | tee -a /tmp/summary
grep "&＃61;&＃61;&＃61;FAIL&＃61;&＃61;&＃61;" /tmp/test-result 2>&1 | tee -a /tmp/summary
fails&＃61;&＃96;grep "&＃61;&＃61;&＃61;FAIL&＃61;&＃61;&＃61;" /tmp/test-result | wc -l&＃96;
if [ $fails &＃61;&＃61; 0 ]
thenecho -e "\033[32;1;7mPASS\033[0m :" &＃96;grep "&＃61;&＃61;&＃61;PASS&＃61;&＃61;&＃61;" /tmp/test-result | wc -l&＃96; 2>&1 | tee -a /tmp/summary
elseecho -e "\033[31;1;7mFAIL\033[0m :" &＃96;grep "&＃61;&＃61;&＃61;FAIL&＃61;&＃61;&＃61;" /tmp/test-result | wc -l&＃96; 2>&1 | tee -a /tmp/summary
fi
echo "TOTAL: $case_count" 2>&1 | tee -a /tmp/summary
cat /tmp/summary >> /tmp/test-result