Erlang 集群中Mnesia Table 如何在不影响正常业务的情况下增加/减少字段?
如何使用 mnesia:transform_table/4 在Mnesia Table增加(或减少)字段？
假如我们有一个用于监控记录节点各种情况的app，命名这counter, 每秒要证明一下自己还活着，把计数 + 1。
-record(node_info, {name, count}).
{atomic,ok} = mnesia:create_table(node_info,
[{attributes, record_info(node_info)},
{ram_copies, [node()]}]);
使用一个gen_server每秒计数 + 1.
update_count() ->
Fun = fun() ->
[Client] = mnesia:read(?TABLE, node()),
mnesia:write(Client#node_info{count = Client#node_info.count + 1})
end,
{atomic, ok} = mnesia:sync_transaction(Fun).
mnesia node_info table 记录逻辑完整代码可见
这样，我们先release一下0.1.0版本的包，并把它跑起来
我们把release并启动的流程写在脚本 ./script/0.1.0-alice-start.sh 里。
#!/bin/bash rm -rf _build git checkout mnesia-upgrade-0.1.0-alice export RELX_REPLACE_OS_VARS=true export NODENAME=alice@127.0.0.1 export COOKIE=zhongwen rebar3 release _build/default/rel/counter/bin/counter-0.1.0-alice console
我们把release的包叫做counter-0.1.0-alice并使用git在这些代码上打上mnesia-upgrade-0.1.0-alice的Tag(方便后面切换版本)
关于 RELX_REPLACE_OS_VARS 的说明请看 rebar3 dynamic configuration
release完成后以console的方式启动。
启动后可以看到在console里面每5秒打印一次node_info表里面的内容。
=PROGRESS REPORT==== 13-Nov-2016::13:53:36 ===
application: sasl
started_at: 'alice@127.0.0.1'
Eshell V7.3 (abort with ^G)
(alice@127.0.0.1)1> [[{node_info,'alice@127.0.0.1',2}]]
[[{node_info,'alice@127.0.0.1',3}]]
[[{node_info,'alice@127.0.0.1',4}]]
...
现在需要增加再加一个字段叫smp_support来记录节点是否支持smp
-record(node_info, {name, count, smp_support}).
-module(counter_db_transform).
-export([add_smp_support_to_node_info_table/0]).
-export([del_smp_support_to_node_info_table/0]).
add_smp_support_to_node_info_table() ->
Fun =
fun({node_info, Node, Count}) ->
{node_info, Node, Count, undefined};
(Record) -> Record
end,
NewAttrList = [node, count, smp_support],
{atomic, ok} = mnesia:transform_table(node_info, Fun, NewAttrList),
ok.
del_smp_support_to_node_info_table() ->
Fun =
fun({node_info, Node, Count, _}) ->
{node_info, Node, Count};
(Record) -> Record
end,
NewAttrList = [node, count],
{atomic, ok} = mnesia:transform_table(node_info, Fun, NewAttrList),
ok.
http://erlang.org/doc/man/mnesia.html#transform_table-4
Argument Fun can also be the atom ignore, which indicates that only the metadata about the table is updated. Use of ignore is not recommended, but included as a possibility for the user do to an own transformation.
这里的ignore就是只改变表的结构，但是不会对返回ignore的值作任何修改，也就是还是原来的record,我们并没有用到。
{"0.2.0",
[
{"0.1.0", [
{load_module, counter_db_schema},
{add_module, counter_db_transform},
{apply, {counter_db_transform, add_smp_support_to_node_info_table, []}},
{update, counter_collect_server, {advanced, delete_id}}
]}
],
[
{"0.1.0", [
{apply, {counter_db_transform, del_smp_support_to_node_info_table, []}},
{delete_module, counter_db_transform},
{load_module, counter_db_schema},
{update, counter_collect_server, {advanced, add_id}}
]}
]
}.
具体的appup.src规则(load_module, add_module, delete_module, update, apply)可以参照 cookbook 。
全部的代码都已完成，把代码打上mnesia-upgrade-0.2.0-alice的Tag,
接下来操作一下如何升级counter-0.1.0-alice到counter-0.2.0-alice。
升级脚本写在 ./script/upgrade-0.1.0-alice-to-0.2.0-alice.sh 中
#!/bin/bash git checkout mnesia-upgrade-0.2.0-alice export RELX_REPLACE_OS_VARS=true export NODENAME=alice@127.0.0.1 export COOKIE=zhongwen rebar3 release rebar3 relup rebar3 tar mv _build/default/rel/counter/counter-0.2.0-alice.tar.gz _build/default/rel/counter/releases/0.2.0-alice/counter.tar.gz _build/default/rel/counter/bin/counter-0.1.0-alice upgrade 0.2.0-alice
相比于0.1.0的release完整包的操作，我们在打0.2.0包是使用的升级包的方式，
release => relup => tar => 使用bin/counter-0.1.0-alice upgrade 0.2.0-alice升级。
执行此脚本后结果如下
===> tarball /Users/zhongwen/github/erlang-rock/mneisa-cluster-hot-upgrade-record-structure/counter/_build/default/rel/counter/counter-0.2.0-alice.tar.gz successfully created! Release 0.2.0-alice not found, attempting to unpack releases/0.2.0-alice/counter.tar.gz Unpacked successfully: "0.2.0-alice" Installed Release: 0.2.0-alice Made release permanent: "0.2.0-alice"
说明升级成功，我们再来去上面的console里面看一下每5秒钟打印的输出有木有了smp_support字段。
[[{node_info,'alice@127.0.0.1',124}]]
[[{node_info,'alice@127.0.0.1',125}]]
code change {169265188944133434916933618072298650122,
{state,undefined,5000},
delete_id}
[[{node_info,'alice@127.0.0.1',126,true}]]
[[{node_info,'alice@127.0.0.1',127,true}]]
[[{node_info,'alice@127.0.0.1',128,true}]]
(alice@127.0.0.1)6> mnesia:table_info(node_info, attributes).
[node,count,smp_support]
(alice@127.0.0.1)7> application:which_applications().
[{sasl,"SASL CXC 138 11","2.7"},
{counter,"counter","0.2.0"},
{mnesia,"MNESIA CXC 138 12","4.13.3"},
{stdlib,"ERTS CXC 138 10","2.8"},
{kernel,"ERTS CXC 138 10","4.2"}]
升级成功！
PS: code change 发生的原因是因为我们使用了appup.src里面使用update规则把state里面的id删掉了，有兴趣可以细看一下0.1.0升级和0.2.0的代码区别(在release如何运用code_change/3 热升级gen_server里面的state)。
counter applications的原来的目的就是集群中每个节点都把自己的情况以node()为key存到node_info表里面，然后可以在任意节点上看到集群上所有节点的node_info了，所以我们接下来，先建立一个集群环境，然后再在集群环境下尝试做升级。
我们从头开始把counter-0.3.0作为base版本
把counter-0.2.0中的appup.src删掉，并把app.src rebar.config里面的版本号升级一下为0.3.0后打上mnesia-upgrade-0.3.0-alice Tag。
启动alice
脚本文件 ./script/0.3.0-alice-start.sh
#!/bin/bash git checkout mnesia-upgrade-0.3.0-alice export RELX_REPLACE_OS_VARS=true export NODENAME=alice@127.0.0.1 export CLUSTER_NODENAME=bob@127.0.0.1 export COOKIE=zhongwen rebar3 release _build/default/rel/counter/bin/counter-0.3.0-alice console
正常启动alice时时结果如下
=PROGRESS REPORT==== 13-Nov-2016::14:32:24 ===
application: sasl
started_at: 'alice@127.0.0.1'
Eshell V7.3 (abort with ^G)
(alice@127.0.0.1)1> [[{node_info,'alice@127.0.0.1',2,true}]]
[[{node_info,'alice@127.0.0.1',3,true}]]
(alice@127.0.0.1)1> application:which_applications().
[{sasl,"SASL CXC 138 11","2.7"},
{counter,"counter","0.3.0"},
{mnesia,"MNESIA CXC 138 12","4.13.3"},
{stdlib,"ERTS CXC 138 10","2.8"},
{kernel,"ERTS CXC 138 10","4.2"}]
(alice@127.0.0.1)2> [[{node_info,'alice@127.0.0.1',4,true}]]
[[{node_info,'alice@127.0.0.1',5,true}]]
基本功能和0.2.0一样，不过会多一个节点互连功能，
把counter-0.3.0-alice的包名改成counter-0.3.0-bob后mnesia-upgrade-0.3.0-bob Tag。
脚本文件 ./script/0.3.0-bob-start.sh
#!/bin/bash git checkout mnesia-upgrade-0.3.0-bob export RELX_REPLACE_OS_VARS=true export NODENAME=bob@127.0.0.1 export CLUSTER_NODENAME=alice@127.0.0.1 export COOKIE=zhongwen rebar3 release _build/default/rel/counter/bin/counter-0.3.0-bob console
正常启动bob里输出为
=PROGRESS REPORT==== 13-Nov-2016::14:38:34 ===
application: sasl
started_at: 'bob@127.0.0.1'
Eshell V7.3 (abort with ^G)
(bob@127.0.0.1)1> [[{node_info,'alice@127.0.0.1',113,true}],[{node_info,'bob@127.0.0.1',2,true}]]
可以看到在bob console里面已看到了alice节点的情况
我们再来确认一下alice是否能看到bob
(alice@127.0.0.1)1> [[{node_info,'alice@127.0.0.1',112,true}]]
[[{node_info,'alice@127.0.0.1',113,true}],[{node_info,'bob@127.0.0.1',1,true}]]
集群和单节点不一样的是，mnesia table是在alice bob之前 共享表结构和数据 的，如果我们先升级alice,那么没有升级的bob就会报错，因为bob上跑的是旧代码，他的node_info 没有binary字段，但alice升级就把node_info table升级成带binary字段的table了。这样bob就惨了。
所以我们一定要保证bob在alice升级完成后也不报错，所以在做真正升级时做一个兼容代码的小版本，让旧代码遇到新表数据时也不报错！
我们来做这个兼容小版本叫0.3.1
主要变动就是：
update_count() ->
Fun = fun() ->
- [Client] = mnesia:read(?TABLE, node()),
- mnesia:write(Client#node_info{count = Client#node_info.count + 1,
- smp_support = erlang:system_info(smp_support)})
- end,
+ case mnesia:read(?TABLE, node()) of
+ [Client] when is_record(Client, node_info) ->
+ mnesia:write(Client#node_info{count = Client#node_info.count + 1,
+ smp_support = erlang:system_info(smp_support)
+ });
+ [{node_info, Name, Count, _SmpSupport, _Binary}] ->
+ mnesia:write({node_info, Name, Count + 1, erlang:system_info(smp_support), erlang:memory(binary)})
+ end
+ end,
{atomic, ok} = mnesia:sync_transaction(Fun),
io:format("~p~n", [[mnesia:dirty_read(?TABLE, Key) ||Key <- mnesia:dirty_all_keys(?TABLE)]]).
让这代码可以跑新旧表结果的数据。
所有改动可以使用以下命令看到
git diff mnesia-upgrade-0.3.0-alice mnesia-upgrade-0.3.1-alice
升级alice 0.3.0 到 0.3.1
./script/upgrade-0.3.0-alice-to-0.3.1-alice.sh
#!/bin/bash git checkout mnesia-upgrade-0.3.1-alice export RELX_REPLACE_OS_VARS=true export NODENAME=alice@127.0.0.1 export CLUSTER_NODENAME=bob@127.0.0.1 export COOKIE=zhongwen rebar3 release rebar3 relup --upfrom 0.3.0-alice rebar3 tar mv _build/default/rel/counter/counter-0.3.1-alice.tar.gz _build/default/rel/counter/releases/0.3.1-alice/counter.tar.gz _build/default/rel/counter/bin/counter-0.3.0-alice upgrade 0.3.1-alice
成功提示
===> Resolved counter-0.3.1-alice ===> tarball /Users/zhongwen/github/erlang-rock/mneisa-cluster-hot-upgrade-record-structure/counter/_build/default/rel/counter/counter-0.3.1-alice.tar.gz successfully created! Release 0.3.1-alice not found, attempting to unpack releases/0.3.1-alice/counter.tar.gz Unpacked successfully: "0.3.1-alice" Installed Release: 0.3.1-alice Made release permanent: "0.3.1-alice"
同样升级bob 0.3.0 到 0.3.1
./script/upgrade-0.3.0-bob-to-0.3.1-bob.sh
成功提示
===> Resolved counter-0.3.1-bob ===> tarball /Users/zhongwen/github/erlang-rock/mneisa-cluster-hot-upgrade-record-structure/counter/_build/default/rel/counter/counter-0.3.1-bob.tar.gz successfully created! Release 0.3.1-bob not found, attempting to unpack releases/0.3.1-bob/counter.tar.gz Unpacked successfully: "0.3.1-bob" Installed Release: 0.3.1-bob Made release permanent: "0.3.1-bob"
这里我们从0.3.1升级到0.4.1(加了binary字段)上就和单节点上升级一样操作了，先升级alice,然后再升级bob即可。
主要就是跑以下2个脚本
./script/upgrade-0.3.1-alice-to-0.4.1-alice.sh
./script/upgrade-0.3.1-bob-to-0.4.1-bob.sh
因为步骤一样，所以就不在演示啦。看效果就是
bob alice都在线升级到了0.4.1。可以看到每个节点binary统计。
[[{node_info,'alice@127.0.0.1',90,true,41456}],
[{node_info,'bob@127.0.0.1',87,true,29608}]]
以上所述就是小编给大家介绍的《Mnesia Cluster Table 在线增字段》，希望对大家有所帮助，如果大家有任何疑问请给我留言，小编会及时回复大家的。在此也非常感谢大家对 我们 的支持！
京公网安备 11010802041100号 | 京ICP备19059560号-4 | PHP1.CN 第一PHP社区 版权所有