• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

Java RMContainerState类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerState的典型用法代码示例。如果您正苦于以下问题:Java RMContainerState类的具体用法?Java RMContainerState怎么用?Java RMContainerState使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



RMContainerState类属于org.apache.hadoop.yarn.server.resourcemanager.rmcontainer包,在下文中一共展示了RMContainerState类的20个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: recoverContainer

import org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerState; //导入依赖的package包/类
public synchronized void recoverContainer(SchedulerNode node,
    RMContainer rmContainer) {
  // recover app scheduling info
  appSchedulingInfo.recoverContainer(rmContainer);

  if (rmContainer.getState().equals(RMContainerState.COMPLETED)) {
    return;
  }
  LOG.info("SchedulerAttempt " + getApplicationAttemptId()
    + " is recovering container " + rmContainer.getContainerId());
  liveContainers.put(rmContainer.getContainerId(), rmContainer);
  attemptResourceUsage.incUsed(node.getPartition(), rmContainer
      .getContainer().getResource());
  
  // resourceLimit: updated when LeafQueue#recoverContainer#allocateResource
  // is called.
  // newlyAllocatedContainers.add(rmContainer);
  // schedulingOpportunities
  // lastScheduledContainer
}
 
开发者ID:naver,项目名称:hadoop,代码行数:21,代码来源:SchedulerApplicationAttempt.java


示例2: recoverContainer

import org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerState; //导入依赖的package包/类
public synchronized void recoverContainer(RMContainer rmContainer) {
  QueueMetrics metrics = queue.getMetrics();
  if (pending) {
    // If there was any container to recover, the application was
    // running from scheduler's POV.
    pending = false;
    metrics.runAppAttempt(applicationId, user);
  }

  // Container is completed. Skip recovering resources.
  if (rmContainer.getState().equals(RMContainerState.COMPLETED)) {
    return;
  }

  metrics.allocateResources(user, 1, rmContainer.getAllocatedResource(),
    false);
}
 
开发者ID:naver,项目名称:hadoop,代码行数:18,代码来源:AppSchedulingInfo.java


示例3: recoverContainer

import org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerState; //导入依赖的package包/类
@Override
public void recoverContainer(Resource clusterResource,
    SchedulerApplicationAttempt attempt, RMContainer rmContainer) {
  if (rmContainer.getState().equals(RMContainerState.COMPLETED)) {
    return;
  }
  // Careful! Locking order is important! 
  synchronized (this) {
    FiCaSchedulerNode node =
        scheduler.getNode(rmContainer.getContainer().getNodeId());
    super.allocateResource(clusterResource, rmContainer.getContainer()
        .getResource(), node.getLabels());
  }
  if (parent != null) {
    parent.recoverContainer(clusterResource, attempt, rmContainer);
  }
}
 
开发者ID:naver,项目名称:hadoop,代码行数:18,代码来源:ParentQueue.java


示例4: getLogAggregationContextFromContainerToken

import org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerState; //导入依赖的package包/类
private LogAggregationContext getLogAggregationContextFromContainerToken(
    MockRM rm1, MockNM nm1, LogAggregationContext logAggregationContext)
    throws Exception {
  RMApp app2 = rm1.submitApp(200, logAggregationContext);
  MockAM am2 = MockRM.launchAndRegisterAM(app2, rm1, nm1);
  nm1.nodeHeartbeat(true);
  // request a container.
  am2.allocate("127.0.0.1", 512, 1, new ArrayList<ContainerId>());
  ContainerId containerId =
      ContainerId.newContainerId(am2.getApplicationAttemptId(), 2);
  rm1.waitForState(nm1, containerId, RMContainerState.ALLOCATED);

  // acquire the container.
  List<Container> containers =
      am2.allocate(new ArrayList<ResourceRequest>(),
        new ArrayList<ContainerId>()).getAllocatedContainers();
  Assert.assertEquals(containerId, containers.get(0).getId());
  // container token is generated.
  Assert.assertNotNull(containers.get(0).getContainerToken());
  ContainerTokenIdentifier token =
      BuilderUtils.newContainerTokenIdentifier(containers.get(0)
        .getContainerToken());
  return token.getLogAggregationContext();
}
 
开发者ID:naver,项目名称:hadoop,代码行数:25,代码来源:TestContainerAllocation.java


示例5: recoverContainer

import org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerState; //导入依赖的package包/类
@Override
public void recoverContainer(Resource clusterResource,
    SchedulerApplicationAttempt attempt, RMContainer rmContainer) {
  if (rmContainer.getState().equals(RMContainerState.COMPLETED)) {
    return;
  }
  // Careful! Locking order is important! 
  synchronized (this) {
    FiCaSchedulerNode node =
        scheduler.getNode(rmContainer.getContainer().getNodeId());
    super.allocateResource(clusterResource, rmContainer.getContainer()
        .getResource(), node.getPartition(), false);
  }
  if (parent != null) {
    parent.recoverContainer(clusterResource, attempt, rmContainer);
  }
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:18,代码来源:ParentQueue.java


示例6: allocateAndLaunchContainers

import org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerState; //导入依赖的package包/类
private void allocateAndLaunchContainers(MockAM am, MockNM nm, MockRM rm,
    int nContainer, int mem, int priority, int startContainerId)
        throws Exception {
  am.allocate(Arrays
      .asList(ResourceRequest.newInstance(Priority.newInstance(priority), "*",
          Resources.createResource(mem), nContainer)),
      null);
  ContainerId lastContainerId = ContainerId.newContainerId(
      am.getApplicationAttemptId(), startContainerId + nContainer - 1);
  Assert.assertTrue(rm.waitForState(nm, lastContainerId,
      RMContainerState.ALLOCATED, 10 * 1000));
  // Acquire them, and NM report RUNNING
  am.allocate(null, null);

  for (int cId = startContainerId; cId < startContainerId
      + nContainer; cId++) {
    sentRMContainerLaunched(rm,
        ContainerId.newContainerId(am.getApplicationAttemptId(), cId));
    rm.waitForContainerState(
        ContainerId.newContainerId(am.getApplicationAttemptId(), cId),
        RMContainerState.RUNNING);
  }
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:24,代码来源:TestContainerResizing.java


示例7: recoverContainer

import org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerState; //导入依赖的package包/类
public synchronized void recoverContainer(RMContainer rmContainer) {
  // recover app scheduling info
  appSchedulingInfo.recoverContainer(rmContainer);

  if (rmContainer.getState().equals(RMContainerState.COMPLETED)) {
    return;
  }
  LOG.info("SchedulerAttempt " + getApplicationAttemptId()
    + " is recovering container " + rmContainer.getContainerId());
  liveContainers.put(rmContainer.getContainerId(), rmContainer);
  Resources.addTo(currentConsumption, rmContainer.getContainer()
    .getResource());
  // resourceLimit: updated when LeafQueue#recoverContainer#allocateResource
  // is called.
  // newlyAllocatedContainers.add(rmContainer);
  // schedulingOpportunities
  // lastScheduledContainer
}
 
开发者ID:yncxcw,项目名称:big-c,代码行数:19,代码来源:SchedulerApplicationAttempt.java


示例8: recoverContainer

import org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerState; //导入依赖的package包/类
@Override
public void recoverContainer(Resource clusterResource,
    SchedulerApplicationAttempt attempt, RMContainer rmContainer) {
  if (rmContainer.getState().equals(RMContainerState.COMPLETED)) {
    return;
  }
  // Careful! Locking order is important! 
  synchronized (this) {
    FiCaSchedulerNode node =
        scheduler.getNode(rmContainer.getContainer().getNodeId());
    super.allocateResource(clusterResource, rmContainer.getContainer()
        .getResource(), node.getLabels(),false);
  }
  if (parent != null) {
    parent.recoverContainer(clusterResource, attempt, rmContainer);
  }
}
 
开发者ID:yncxcw,项目名称:big-c,代码行数:18,代码来源:ParentQueue.java


示例9: recoverContainer

import org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerState; //导入依赖的package包/类
@Override
public void recoverContainer(Resource clusterResource,
    SchedulerApplicationAttempt attempt, RMContainer rmContainer) {
  if (rmContainer.getState().equals(RMContainerState.COMPLETED)) {
    return;
  }
  // Careful! Locking order is important! 
  synchronized (this) {
    super.allocateResource(clusterResource, rmContainer.getContainer()
        .getResource(), labelManager.getLabelsOnNode(rmContainer
        .getContainer().getNodeId()));
  }
  if (parent != null) {
    parent.recoverContainer(clusterResource, attempt, rmContainer);
  }
}
 
开发者ID:Nextzero,项目名称:hadoop-2.6.0-cdh5.4.3,代码行数:17,代码来源:ParentQueue.java


示例10: recoverContainer

import org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerState; //导入依赖的package包/类
@Override
public void recoverContainer(Resource clusterResource,
    SchedulerApplicationAttempt attempt, RMContainer rmContainer) {
  if (rmContainer.getState().equals(RMContainerState.COMPLETED)) {
    return;
  }
  // Careful! Locking order is important! 
  synchronized (this) {
    FiCaSchedulerNode node =
        scheduler.getNode(rmContainer.getContainer().getNodeId());
    allocateResource(clusterResource,
        rmContainer.getContainer().getResource(), node.getPartition(), false);
  }
  if (parent != null) {
    parent.recoverContainer(clusterResource, attempt, rmContainer);
  }
}
 
开发者ID:hopshadoop,项目名称:hops,代码行数:18,代码来源:ParentQueue.java


示例11: decreaseContainer

import org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerState; //导入依赖的package包/类
@Override
protected void decreaseContainer(SchedContainerChangeRequest decreaseRequest,
    SchedulerApplicationAttempt attempt) {
  RMContainer rmContainer = decreaseRequest.getRMContainer();
  // Check container status before doing decrease
  if (rmContainer.getState() != RMContainerState.RUNNING) {
    LOG.info("Trying to decrease a container not in RUNNING state, container="
        + rmContainer + " state=" + rmContainer.getState().name());
    return;
  }
  FiCaSchedulerApp app = (FiCaSchedulerApp)attempt;
  LeafQueue queue = (LeafQueue) attempt.getQueue();
  try {
    queue.decreaseContainer(clusterResource, decreaseRequest, app);
    // Notify RMNode that the container can be pulled by NodeManager in the
    // next heartbeat
    this.rmContext.getDispatcher().getEventHandler()
        .handle(new RMNodeDecreaseContainerEvent(
            decreaseRequest.getNodeId(),
            Collections.singletonList(rmContainer.getContainer())));
  } catch (InvalidResourceRequestException e) {
    LOG.warn("Error happens when checking decrease request, Ignoring.."
        + " exception=", e);
  }
}
 
开发者ID:hopshadoop,项目名称:hops,代码行数:26,代码来源:CapacityScheduler.java


示例12: waitForState

import org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerState; //导入依赖的package包/类
public void waitForState(MockNM nm, ContainerId containerId,
    RMContainerState containerState) throws Exception {
  RMContainer container = getResourceScheduler().getRMContainer(containerId);
  int timeoutSecs = 0;
  while(container == null && timeoutSecs++ < 20) {
    nm.nodeHeartbeat(true);
    container = getResourceScheduler().getRMContainer(containerId);
    System.out.println("Waiting for container " + containerId + " to be allocated.");
    Thread.sleep(100);
  }
  Assert.assertNotNull("Container shouldn't be null", container);
  timeoutSecs = 0;
  while (!containerState.equals(container.getState()) && timeoutSecs++ < 40) {
    System.out.println("Container : " + containerId + " State is : "
        + container.getState() + " Waiting for state : " + containerState);
    nm.nodeHeartbeat(true);
    Thread.sleep(300);
  }
  System.out.println("Container State is : " + container.getState());
  Assert.assertEquals("Container state is not correct (timedout)",
    containerState, container.getState());
}
 
开发者ID:Seagate,项目名称:hadoop-on-lustre2,代码行数:23,代码来源:MockRM.java


示例13: recoverContainer

import org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerState; //导入依赖的package包/类
@Override
public void recoverContainer(Resource clusterResource,
    SchedulerApplicationAttempt schedulerAttempt, RMContainer rmContainer) {
  if (rmContainer.getState().equals(RMContainerState.COMPLETED)) {
    return;
  }
  increaseUsedResources(rmContainer);
  updateAppHeadRoom(schedulerAttempt);
  updateAvailableResourcesMetrics();
}
 
开发者ID:naver,项目名称:hadoop,代码行数:11,代码来源:FifoScheduler.java


示例14: doneApplicationAttempt

import org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerState; //导入依赖的package包/类
private synchronized void doneApplicationAttempt(
    ApplicationAttemptId applicationAttemptId,
    RMAppAttemptState rmAppAttemptFinalState, boolean keepContainers)
    throws IOException {
  FiCaSchedulerApp attempt = getApplicationAttempt(applicationAttemptId);
  SchedulerApplication<FiCaSchedulerApp> application =
      applications.get(applicationAttemptId.getApplicationId());
  if (application == null || attempt == null) {
    throw new IOException("Unknown application " + applicationAttemptId + 
    " has completed!");
  }

  // Kill all 'live' containers
  for (RMContainer container : attempt.getLiveContainers()) {
    if (keepContainers
        && container.getState().equals(RMContainerState.RUNNING)) {
      // do not kill the running container in the case of work-preserving AM
      // restart.
      LOG.info("Skip killing " + container.getContainerId());
      continue;
    }
    completedContainer(container,
      SchedulerUtils.createAbnormalContainerStatus(
        container.getContainerId(), SchedulerUtils.COMPLETED_APPLICATION),
      RMContainerEventType.KILL);
  }

  // Clean up pending requests, metrics etc.
  attempt.stop(rmAppAttemptFinalState);
}
 
开发者ID:naver,项目名称:hadoop,代码行数:31,代码来源:FifoScheduler.java


示例15: completedContainer

import org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerState; //导入依赖的package包/类
/**
 * Clean up a completed container.
 */
@Override
protected synchronized void completedContainer(RMContainer rmContainer,
    ContainerStatus containerStatus, RMContainerEventType event) {
  if (rmContainer == null) {
    LOG.info("Null container completed...");
    return;
  }

  Container container = rmContainer.getContainer();

  // Get the application for the finished container
  FSAppAttempt application =
      getCurrentAttemptForContainer(container.getId());
  ApplicationId appId =
      container.getId().getApplicationAttemptId().getApplicationId();
  if (application == null) {
    LOG.info("Container " + container + " of" +
        " unknown application attempt " + appId +
        " completed with event " + event);
    return;
  }

  // Get the node on which the container was allocated
  FSSchedulerNode node = getFSSchedulerNode(container.getNodeId());

  if (rmContainer.getState() == RMContainerState.RESERVED) {
    application.unreserve(rmContainer.getReservedPriority(), node);
  } else {
    application.containerCompleted(rmContainer, containerStatus, event);
    node.releaseContainer(container);
    updateRootQueueMetrics();
  }

  LOG.info("Application attempt " + application.getApplicationAttemptId()
      + " released container " + container.getId() + " on node: " + node
      + " with event: " + event);
}
 
开发者ID:naver,项目名称:hadoop,代码行数:41,代码来源:FairScheduler.java


示例16: recoverContainer

import org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerState; //导入依赖的package包/类
@Override
public void recoverContainer(Resource clusterResource,
    SchedulerApplicationAttempt attempt, RMContainer rmContainer) {
  if (rmContainer.getState().equals(RMContainerState.COMPLETED)) {
    return;
  }
  // Careful! Locking order is important! 
  synchronized (this) {
    FiCaSchedulerNode node =
        scheduler.getNode(rmContainer.getContainer().getNodeId());
    allocateResource(clusterResource, attempt, rmContainer.getContainer()
        .getResource(), node.getLabels());
  }
  getParent().recoverContainer(clusterResource, attempt, rmContainer);
}
 
开发者ID:naver,项目名称:hadoop,代码行数:16,代码来源:LeafQueue.java


示例17: waitForState

import org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerState; //导入依赖的package包/类
public boolean waitForState(MockNM nm, ContainerId containerId,
    RMContainerState containerState, int timeoutMillisecs) throws Exception {
  RMContainer container = getResourceScheduler().getRMContainer(containerId);
  int timeoutSecs = 0;
  while(container == null && timeoutSecs++ < timeoutMillisecs / 100) {
    nm.nodeHeartbeat(true);
    container = getResourceScheduler().getRMContainer(containerId);
    System.out.println("Waiting for container " + containerId + " to be allocated.");
    Thread.sleep(100);
    
    if (timeoutMillisecs <= timeoutSecs * 100) {
      return false;
    }
  }
  Assert.assertNotNull("Container shouldn't be null", container);
  while (!containerState.equals(container.getState())
      && timeoutSecs++ < timeoutMillisecs / 100) {
    System.out.println("Container : " + containerId + " State is : "
        + container.getState() + " Waiting for state : " + containerState);
    nm.nodeHeartbeat(true);
    Thread.sleep(100);
    
    if (timeoutMillisecs <= timeoutSecs * 100) {
      return false;
    }
  }
  
  System.out.println("Container State is : " + container.getState());
  Assert.assertEquals("Container state is not correct (timedout)",
    containerState, container.getState());
  return true;
}
 
开发者ID:naver,项目名称:hadoop,代码行数:33,代码来源:MockRM.java


示例18: waitContainerAllocated

import org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerState; //导入依赖的package包/类
private void waitContainerAllocated(MockAM am, int mem, int nContainer,
    int startContainerId, MockRM rm, MockNM nm) throws Exception {
  for (int cId = startContainerId; cId < startContainerId + nContainer; cId++) {
    am.allocate("*", mem, 1, new ArrayList<ContainerId>());
    ContainerId containerId =
        ContainerId.newContainerId(am.getApplicationAttemptId(), cId);
    Assert.assertTrue(rm.waitForState(nm, containerId,
        RMContainerState.ALLOCATED, 10 * 1000));
  }
}
 
开发者ID:naver,项目名称:hadoop,代码行数:11,代码来源:TestCapacityScheduler.java


示例19: testContainerTokenGeneratedOnPullRequest

import org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerState; //导入依赖的package包/类
@Test
public void testContainerTokenGeneratedOnPullRequest() throws Exception {
  MockRM rm1 = new MockRM(conf);
  rm1.start();
  MockNM nm1 = rm1.registerNode("127.0.0.1:1234", 8000);
  RMApp app1 = rm1.submitApp(200);
  MockAM am1 = MockRM.launchAndRegisterAM(app1, rm1, nm1);
  // request a container.
  am1.allocate("127.0.0.1", 1024, 1, new ArrayList<ContainerId>());
  ContainerId containerId2 =
      ContainerId.newContainerId(am1.getApplicationAttemptId(), 2);
  rm1.waitForState(nm1, containerId2, RMContainerState.ALLOCATED);

  RMContainer container =
      rm1.getResourceScheduler().getRMContainer(containerId2);
  // no container token is generated.
  Assert.assertEquals(containerId2, container.getContainerId());
  Assert.assertNull(container.getContainer().getContainerToken());

  // acquire the container.
  List<Container> containers =
      am1.allocate(new ArrayList<ResourceRequest>(),
        new ArrayList<ContainerId>()).getAllocatedContainers();
  Assert.assertEquals(containerId2, containers.get(0).getId());
  // container token is generated.
  Assert.assertNotNull(containers.get(0).getContainerToken());
  rm1.stop();
}
 
开发者ID:naver,项目名称:hadoop,代码行数:29,代码来源:TestContainerAllocation.java


示例20: testNormalContainerAllocationWhenDNSUnavailable

import org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerState; //导入依赖的package包/类
@Test
public void testNormalContainerAllocationWhenDNSUnavailable() throws Exception{
  MockRM rm1 = new MockRM(conf);
  rm1.start();
  MockNM nm1 = rm1.registerNode("unknownhost:1234", 8000);
  RMApp app1 = rm1.submitApp(200);
  MockAM am1 = MockRM.launchAndRegisterAM(app1, rm1, nm1);

  // request a container.
  am1.allocate("127.0.0.1", 1024, 1, new ArrayList<ContainerId>());
  ContainerId containerId2 =
      ContainerId.newContainerId(am1.getApplicationAttemptId(), 2);
  rm1.waitForState(nm1, containerId2, RMContainerState.ALLOCATED);

  // acquire the container.
  SecurityUtilTestHelper.setTokenServiceUseIp(true);
  List<Container> containers =
      am1.allocate(new ArrayList<ResourceRequest>(),
        new ArrayList<ContainerId>()).getAllocatedContainers();
  // not able to fetch the container;
  Assert.assertEquals(0, containers.size());

  SecurityUtilTestHelper.setTokenServiceUseIp(false);
  containers =
      am1.allocate(new ArrayList<ResourceRequest>(),
        new ArrayList<ContainerId>()).getAllocatedContainers();
  // should be able to fetch the container;
  Assert.assertEquals(1, containers.size());
}
 
开发者ID:naver,项目名称:hadoop,代码行数:30,代码来源:TestContainerAllocation.java



注:本文中的org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerState类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
Java TestCaseName类代码示例发布时间:2022-05-22
下一篇:
Java BasicOutputBuffer类代码示例发布时间:2022-05-22
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap