• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

Java RMState类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.RMState的典型用法代码示例。如果您正苦于以下问题:Java RMState类的具体用法?Java RMState怎么用?Java RMState使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



RMState类属于org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore包,在下文中一共展示了RMState类的20个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: recover

import org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.RMState; //导入依赖的package包/类
public void recover(RMState state) {
  if (state.getAMRMTokenSecretManagerState() != null) {
    // recover the current master key
    MasterKey currentKey =
        state.getAMRMTokenSecretManagerState().getCurrentMasterKey();
    this.currentMasterKey =
        new MasterKeyData(currentKey, createSecretKey(currentKey.getBytes()
          .array()));

    // recover the next master key if not null
    MasterKey nextKey =
        state.getAMRMTokenSecretManagerState().getNextMasterKey();
    if (nextKey != null) {
      this.nextMasterKey =
          new MasterKeyData(nextKey, createSecretKey(nextKey.getBytes()
            .array()));
      this.timer.schedule(new NextKeyActivator(), this.activationDelay);
    }
  }
}
 
开发者ID:naver,项目名称:hadoop,代码行数:21,代码来源:AMRMTokenSecretManager.java


示例2: recover

import org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.RMState; //导入依赖的package包/类
@Override
public void recover(RMState rmState) throws Exception {

  LOG.info("recovering RMDelegationTokenSecretManager.");
  // recover RMDTMasterKeys
  for (DelegationKey dtKey : rmState.getRMDTSecretManagerState()
    .getMasterKeyState()) {
    addKey(dtKey);
  }

  // recover RMDelegationTokens
  Map<RMDelegationTokenIdentifier, Long> rmDelegationTokens =
      rmState.getRMDTSecretManagerState().getTokenState();
  this.delegationTokenSequenceNumber =
      rmState.getRMDTSecretManagerState().getDTSequenceNumber();
  for (Map.Entry<RMDelegationTokenIdentifier, Long> entry : rmDelegationTokens
    .entrySet()) {
    addPersistedDelegationToken(entry.getKey(), entry.getValue());
  }
}
 
开发者ID:naver,项目名称:hadoop,代码行数:21,代码来源:RMDelegationTokenSecretManager.java


示例3: recover

import org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.RMState; //导入依赖的package包/类
@Override
public void recover(RMState state) {
  ApplicationStateData appState =
      state.getApplicationState().get(getApplicationId());
  this.recoveredFinalState = appState.getState();
  LOG.info("Recovering app: " + getApplicationId() + " with " + 
      + appState.getAttemptCount() + " attempts and final state = "
      + this.recoveredFinalState );
  this.diagnostics.append(appState.getDiagnostics());
  this.storedFinishTime = appState.getFinishTime();
  this.startTime = appState.getStartTime();

  for(int i=0; i<appState.getAttemptCount(); ++i) {
    // create attempt
    createNewAttempt();
    ((RMAppAttemptImpl)this.currentAttempt).recover(state);
  }
}
 
开发者ID:naver,项目名称:hadoop,代码行数:19,代码来源:RMAppImpl.java


示例4: finishApplicationMaster

import org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.RMState; //导入依赖的package包/类
private void finishApplicationMaster(RMApp rmApp, MockRM rm, MockNM nm,
    MockAM am, FinishApplicationMasterRequest req) throws Exception {
  RMState rmState =
      ((MemoryRMStateStore) rm.getRMContext().getStateStore()).getState();
  Map<ApplicationId, ApplicationStateData> rmAppState =
      rmState.getApplicationState();
  am.unregisterAppAttempt(req,true);
  am.waitForState(RMAppAttemptState.FINISHING);
  nm.nodeHeartbeat(am.getApplicationAttemptId(), 1, ContainerState.COMPLETE);
  am.waitForState(RMAppAttemptState.FINISHED);
  rm.waitForState(rmApp.getApplicationId(), RMAppState.FINISHED);
  // check that app/attempt is saved with the final state
  ApplicationStateData appState = rmAppState.get(rmApp.getApplicationId());
  Assert
    .assertEquals(RMAppState.FINISHED, appState.getState());
  Assert.assertEquals(RMAppAttemptState.FINISHED,
    appState.getAttempt(am.getApplicationAttemptId()).getState());
}
 
开发者ID:naver,项目名称:hadoop,代码行数:19,代码来源:TestRMRestart.java


示例5: testCreateAppSubmittedRecovery

import org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.RMState; //导入依赖的package包/类
protected RMApp testCreateAppSubmittedRecovery(
    ApplicationSubmissionContext submissionContext) throws IOException {
  RMApp application = createNewTestApp(submissionContext);
  // NEW => SUBMITTED event RMAppEventType.RECOVER
  RMState state = new RMState();
  ApplicationStateData appState =
      ApplicationStateData.newInstance(123, 123, null, "user");
  state.getApplicationState().put(application.getApplicationId(), appState);
  RMAppEvent event =
      new RMAppRecoverEvent(application.getApplicationId(), state);


  application.handle(event);
  assertStartTimeSet(application);
  assertAppState(RMAppState.SUBMITTED, application);
  return application;
}
 
开发者ID:naver,项目名称:hadoop,代码行数:18,代码来源:TestRMAppTransitions.java


示例6: recover

import org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.RMState; //导入依赖的package包/类
@Override
public void recover(RMState state) throws Exception {
  LOG.info("Recovering Reservation system");
  writeLock.lock();
  try {
    Map<String, Map<ReservationId, ReservationAllocationStateProto>> reservationSystemState =
        state.getReservationState();
    if (planFollower != null) {
      for (String plan : plans.keySet()) {
        // recover reservations if any from state store
        if (reservationSystemState.containsKey(plan)) {
          loadPlan(plan, reservationSystemState.get(plan));
        }
        synchronizePlan(plan, false);
      }
      startPlanFollower(conf.getLong(
          YarnConfiguration.RM_WORK_PRESERVING_RECOVERY_SCHEDULING_WAIT_MS,
          YarnConfiguration.DEFAULT_RM_WORK_PRESERVING_RECOVERY_SCHEDULING_WAIT_MS));
    }
  } finally {
    writeLock.unlock();
  }
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:24,代码来源:AbstractReservationSystem.java


示例7: recover

import org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.RMState; //导入依赖的package包/类
@Override
public void recover(RMState state) {
  ApplicationStateData appState =
      state.getApplicationState().get(getApplicationId());
  this.recoveredFinalState = appState.getState();
  LOG.info("Recovering app: " + getApplicationId() + " with " + 
      + appState.getAttemptCount() + " attempts and final state = "
      + this.recoveredFinalState );
  this.diagnostics.append(null == appState.getDiagnostics() ? "" : appState
      .getDiagnostics());
  this.storedFinishTime = appState.getFinishTime();
  this.startTime = appState.getStartTime();
  this.callerContext = appState.getCallerContext();

  // send the ATS create Event
  sendATSCreateEvent(this, this.startTime);

  for(int i=0; i<appState.getAttemptCount(); ++i) {
    // create attempt
    createNewAttempt();
    ((RMAppAttemptImpl)this.currentAttempt).recover(state);
  }
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:24,代码来源:RMAppImpl.java


示例8: recover

import org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.RMState; //导入依赖的package包/类
@Override
public void recover(RMState state) throws Exception {
  // recover RMdelegationTokenSecretManager
  rmContext.getRMDelegationTokenSecretManager().recover(state);

  // recover AMRMTokenSecretManager
  rmContext.getAMRMTokenSecretManager().recover(state);

  // recover reservations
  if (reservationSystem != null) {
    reservationSystem.recover(state);
  }
  // recover applications
  rmAppManager.recover(state);

  setSchedulerRecoveryStartAndWaitTime(state, conf);
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:18,代码来源:ResourceManager.java


示例9: testCreateAppSubmittedRecovery

import org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.RMState; //导入依赖的package包/类
protected RMApp testCreateAppSubmittedRecovery(
    ApplicationSubmissionContext submissionContext) throws IOException {
  RMApp application = createNewTestApp(submissionContext);
  // NEW => SUBMITTED event RMAppEventType.RECOVER
  RMState state = new RMState();
  ApplicationStateData appState =
      ApplicationStateData.newInstance(123, 123, null, "user", null);
  state.getApplicationState().put(application.getApplicationId(), appState);
  RMAppEvent event =
      new RMAppRecoverEvent(application.getApplicationId(), state);


  application.handle(event);
  assertStartTimeSet(application);
  assertAppState(RMAppState.SUBMITTED, application);
  return application;
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:18,代码来源:TestRMAppTransitions.java


示例10: validateStoredReservation

import org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.RMState; //导入依赖的package包/类
private void validateStoredReservation(
    RMStateStoreHelper stateStoreHelper, TestDispatcher dispatcher,
    RMContext rmContext, ReservationId r1, String planName,
    ReservationAllocation allocation,
    ReservationAllocationStateProto allocationStateProto) throws Exception {
  RMStateStore store = stateStoreHelper.getRMStateStore();
  when(rmContext.getStateStore()).thenReturn(store);
  store.setRMDispatcher(dispatcher);
  RMState state = store.loadState();
  Map<String, Map<ReservationId, ReservationAllocationStateProto>>
      reservationState = state.getReservationState();
  Assert.assertNotNull(reservationState);
  Map<ReservationId, ReservationAllocationStateProto> reservations =
      reservationState.get(planName);
  Assert.assertNotNull(reservations);
  ReservationAllocationStateProto storedReservationAllocation =
      reservations.get(r1);
  Assert.assertNotNull(storedReservationAllocation);

  assertAllocationStateEqual(
      allocationStateProto, storedReservationAllocation);
  assertAllocationStateEqual(allocation, storedReservationAllocation);
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:24,代码来源:RMStateStoreTestBase.java


示例11: recover

import org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.RMState; //导入依赖的package包/类
@Override
public void recover(RMState state) {
  ApplicationState appState = state.getApplicationState().get(getApplicationId());
  this.recoveredFinalState = appState.getState();
  LOG.info("Recovering app: " + getApplicationId() + " with " + 
      + appState.getAttemptCount() + " attempts and final state = " + this.recoveredFinalState );
  this.diagnostics.append(appState.getDiagnostics());
  this.storedFinishTime = appState.getFinishTime();
  this.startTime = appState.getStartTime();

  for(int i=0; i<appState.getAttemptCount(); ++i) {
    // create attempt
    createNewAttempt();
    ((RMAppAttemptImpl)this.currentAttempt).recover(state);
  }
}
 
开发者ID:Nextzero,项目名称:hadoop-2.6.0-cdh5.4.3,代码行数:17,代码来源:RMAppImpl.java


示例12: serviceStart

import org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.RMState; //导入依赖的package包/类
@Override
protected void serviceStart() throws Exception {
  RMStateStore rmStore = rmContext.getStateStore();
  // The state store needs to start irrespective of recoveryEnabled as apps
  // need events to move to further states.
  rmStore.start();

  if(recoveryEnabled) {
    try {
      rmStore.checkVersion();
      if (rmContext.isWorkPreservingRecoveryEnabled()) {
        rmContext.setEpoch(rmStore.getAndIncrementEpoch());
      }
      RMState state = rmStore.loadState();
      recover(state);
    } catch (Exception e) {
      // the Exception from loadState() needs to be handled for
      // HA and we need to give up master status if we got fenced
      LOG.error("Failed to load/recover state", e);
      throw e;
    }
  }

  super.serviceStart();
}
 
开发者ID:Nextzero,项目名称:hadoop-2.6.0-cdh5.4.3,代码行数:26,代码来源:ResourceManager.java


示例13: finishApplicationMaster

import org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.RMState; //导入依赖的package包/类
private void finishApplicationMaster(RMApp rmApp, MockRM rm, MockNM nm,
    MockAM am, FinishApplicationMasterRequest req) throws Exception {
  RMState rmState =
      ((MemoryRMStateStore) rm.getRMContext().getStateStore()).getState();
  Map<ApplicationId, ApplicationState> rmAppState =
      rmState.getApplicationState();
  am.unregisterAppAttempt(req,true);
  am.waitForState(RMAppAttemptState.FINISHING);
  nm.nodeHeartbeat(am.getApplicationAttemptId(), 1, ContainerState.COMPLETE);
  am.waitForState(RMAppAttemptState.FINISHED);
  rm.waitForState(rmApp.getApplicationId(), RMAppState.FINISHED);
  // check that app/attempt is saved with the final state
  ApplicationState appState = rmAppState.get(rmApp.getApplicationId());
  Assert
    .assertEquals(RMAppState.FINISHED, appState.getState());
  Assert.assertEquals(RMAppAttemptState.FINISHED,
    appState.getAttempt(am.getApplicationAttemptId()).getState());
}
 
开发者ID:Nextzero,项目名称:hadoop-2.6.0-cdh5.4.3,代码行数:19,代码来源:TestRMRestart.java


示例14: recover

import org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.RMState; //导入依赖的package包/类
@Override
public void recover(RMState state) {
  ApplicationStateData appState =
      state.getApplicationState().get(getApplicationId());
  this.recoveredFinalState = appState.getState();
  LOG.info("Recovering app: " + getApplicationId() + " with " + 
      + appState.getAttemptCount() + " attempts and final state = "
      + this.recoveredFinalState );
  this.diagnostics.append(null == appState.getDiagnostics() ? "" : appState
      .getDiagnostics());
  this.storedFinishTime = appState.getFinishTime();
  this.startTime = appState.getStartTime();
  this.callerContext = appState.getCallerContext();

  // send the ATS create Event during RM recovery.
  // NOTE: it could be duplicated with events sent before RM get restarted.
  sendATSCreateEvent();

  for(int i=0; i<appState.getAttemptCount(); ++i) {
    // create attempt
    createNewAttempt();
    ((RMAppAttemptImpl)this.currentAttempt).recover(state);
  }
}
 
开发者ID:hopshadoop,项目名称:hops,代码行数:25,代码来源:RMAppImpl.java


示例15: recover

import org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.RMState; //导入依赖的package包/类
@Override
public void recover(RMState state) throws Exception {
  // recover RMdelegationTokenSecretManager
  rmContext.getRMDelegationTokenSecretManager().recover(state);

  // recover AMRMTokenSecretManager
  rmContext.getAMRMTokenSecretManager().recover(state);

  // recover reservations
  if (reservationSystem != null) {
    reservationSystem.recover(state);
  }
  // recover applications
  rmAppManager.recover(state);

  recoverSalt();
  
  setSchedulerRecoveryStartAndWaitTime(state, conf);
}
 
开发者ID:hopshadoop,项目名称:hops,代码行数:20,代码来源:ResourceManager.java


示例16: finishApplicationMaster

import org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.RMState; //导入依赖的package包/类
private void finishApplicationMaster(RMApp rmApp, MockRM rm, MockNM nm,
    MockAM am, FinishApplicationMasterRequest req) throws Exception {
  am.unregisterAppAttempt(req,true);
  am.waitForState(RMAppAttemptState.FINISHING);
  nm.nodeHeartbeat(am.getApplicationAttemptId(), 1, ContainerState.COMPLETE);
  am.waitForState(RMAppAttemptState.FINISHED);
  rm.waitForState(rmApp.getApplicationId(), RMAppState.FINISHED);
  // check that app/attempt is saved with the final state
  RMState rmState = rm.getRMContext().getStateStore().loadState();
  Map<ApplicationId, ApplicationStateData> rmAppState =
      rmState.getApplicationState();
  ApplicationStateData appState = rmAppState.get(rmApp.getApplicationId());
  Assert
    .assertEquals(RMAppState.FINISHED, appState.getState());
  Assert.assertEquals(RMAppAttemptState.FINISHED,
    appState.getAttempt(am.getApplicationAttemptId()).getState());
}
 
开发者ID:hopshadoop,项目名称:hops,代码行数:18,代码来源:TestRMRestart.java


示例17: testCreateAppSubmittedRecovery

import org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.RMState; //导入依赖的package包/类
protected RMApp testCreateAppSubmittedRecovery(
    ApplicationSubmissionContext submissionContext) throws IOException {
  RMApp application = createNewTestApp(submissionContext);
  // NEW => SUBMITTED event RMAppEventType.RECOVER
  RMState state = new RMState();
  ApplicationStateData appState =
      ApplicationStateData.newInstance(123, 123, null, "user", null);
  state.getApplicationState().put(application.getApplicationId(), appState);
  RMAppEvent event =
      new RMAppRecoverEvent(application.getApplicationId(), state);

  application.handle(event);
  assertStartTimeSet(application);
  assertAppState(RMAppState.SUBMITTED, application);
  return application;
}
 
开发者ID:hopshadoop,项目名称:hops,代码行数:17,代码来源:TestRMAppTransitions.java


示例18: recover

import org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.RMState; //导入依赖的package包/类
@Override
public void recover(RMState state) throws Exception{
  ApplicationState appState = state.getApplicationState().get(getApplicationId());
  this.recoveredFinalState = appState.getState();
  LOG.info("Recovering app: " + getApplicationId() + " with " + 
      + appState.getAttemptCount() + " attempts and final state = " + this.recoveredFinalState );
  this.diagnostics.append(appState.getDiagnostics());
  this.storedFinishTime = appState.getFinishTime();
  this.startTime = appState.getStartTime();

  for(int i=0; i<appState.getAttemptCount(); ++i) {
    // create attempt
    createNewAttempt();
    ((RMAppAttemptImpl)this.currentAttempt).recover(state);
  }
}
 
开发者ID:Seagate,项目名称:hadoop-on-lustre2,代码行数:17,代码来源:RMAppImpl.java


示例19: recover

import org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.RMState; //导入依赖的package包/类
@Override
public void recover(RMState state) throws Exception {
  ApplicationState appState =
      state.getApplicationState().get(getAppAttemptId().getApplicationId());
  ApplicationAttemptState attemptState =
      appState.getAttempt(getAppAttemptId());
  assert attemptState != null;
  LOG.info("Recovering attempt: " + getAppAttemptId() + " with final state: "
      + attemptState.getState());
  diagnostics.append("Attempt recovered after RM restart");
  diagnostics.append(attemptState.getDiagnostics());
  setMasterContainer(attemptState.getMasterContainer());
  recoverAppAttemptCredentials(attemptState.getAppAttemptCredentials());
  this.recoveredFinalState = attemptState.getState();
  this.originalTrackingUrl = attemptState.getFinalTrackingUrl();
  this.proxiedTrackingUrl = generateProxyUriWithScheme(originalTrackingUrl);
  this.finalStatus = attemptState.getFinalApplicationStatus();
  this.startTime = attemptState.getStartTime();
}
 
开发者ID:Seagate,项目名称:hadoop-on-lustre2,代码行数:20,代码来源:RMAppAttemptImpl.java


示例20: serviceStart

import org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.RMState; //导入依赖的package包/类
@Override
protected void serviceStart() throws Exception {
  RMStateStore rmStore = rmContext.getStateStore();
  // The state store needs to start irrespective of recoveryEnabled as apps
  // need events to move to further states.
  rmStore.start();

  if(recoveryEnabled) {
    try {
      rmStore.checkVersion();
      RMState state = rmStore.loadState();
      recover(state);
    } catch (Exception e) {
      // the Exception from loadState() needs to be handled for
      // HA and we need to give up master status if we got fenced
      LOG.error("Failed to load/recover state", e);
      throw e;
    }
  }

  super.serviceStart();
}
 
开发者ID:Seagate,项目名称:hadoop-on-lustre2,代码行数:23,代码来源:ResourceManager.java



注:本文中的org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.RMState类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
Java Issuable类代码示例发布时间:2022-05-22
下一篇:
Java StopContainersRequestProto类代码示例发布时间:2022-05-22
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap