当Samza ApplicationMaster启动时&＃xff0c;它做以下的事情&＃xff1a;
- 通过STREAMING_CONFIG环境变量从YARN获取配置信息(configuration)
- 在随机端口上 启动一个JMX server
- 实例化一个metrics registry和reporter来追踪计量信息
- 将AM向YARN的RM注册
- 使用每个stream的PartitionManager来获取总共的partition数量
- 从Samza的job configuration里获取总的container数量
- 将partition分给container(在Samza AM的dashboard里&＃xff0c;称为Task Group)
- 为每个container向YARN发送一个ResourceRequest
- 每秒向YARN RM poll一次&＃xff0c;检查allocated and released containers
AMRMClientAsync handles communication with the ResourceManager and provides asynchronous updates on events such as container allocations and completions. It contains a thread that sends periodic heartbeats to the ResourceManager. It should be used by implementing a CallbackHandler:
class MyCallbackHandler implements AMRMClientAsync.CallbackHandler {public void onContainersAllocated(List The client&＃39;s lifecycle should be managed similarly to the following:
AMRMClientAsync asyncClient &＃61; createAMRMClientAsync(appAttId, 1000, new MyCallbackhandler());asyncClient.init(conf);asyncClient.start();RegisterApplicationMasterResponse response &＃61; asyncClient.registerApplicationMaster(appMasterHostname, appMasterRpcPort,appMasterTrackingUrl);asyncClient.addContainerRequest(containerRequest);[... wait for application to complete]asyncClient.unregisterApplicationMaster(status, appMsg, trackingUrl);asyncClient.stop();
这个类是用来做为一个Client和RM进行通信&＃xff0c;并且注册一个用于回调的对象来处理container 的allocation和completion事件。它启动一个线程&＃xff0c;周期性地发送hearbeat至ResourceManager
京公网安备 11010802041100号