• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

Java NodeType类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中org.apache.hadoop.yarn.server.resourcemanager.scheduler.NodeType的典型用法代码示例。如果您正苦于以下问题:Java NodeType类的具体用法?Java NodeType怎么用?Java NodeType使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



NodeType类属于org.apache.hadoop.yarn.server.resourcemanager.scheduler包,在下文中一共展示了NodeType类的20个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: assignNodeLocalContainers

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.NodeType; //导入依赖的package包/类
private int assignNodeLocalContainers(FiCaSchedulerNode node, 
    FiCaSchedulerApp application, Priority priority) {
  int assignedContainers = 0;
  ResourceRequest request = 
    application.getResourceRequest(priority, node.getNodeName());
  if (request != null) {
    // Don't allocate on this node if we don't need containers on this rack
    ResourceRequest rackRequest =
        application.getResourceRequest(priority, 
            node.getRMNode().getRackName());
    if (rackRequest == null || rackRequest.getNumContainers() <= 0) {
      return 0;
    }
    
    int assignableContainers = 
      Math.min(
          getMaxAllocatableContainers(application, priority, node, 
              NodeType.NODE_LOCAL), 
              request.getNumContainers());
    assignedContainers = 
      assignContainer(node, application, priority, 
          assignableContainers, request, NodeType.NODE_LOCAL);
  }
  return assignedContainers;
}
 
开发者ID:naver,项目名称:hadoop,代码行数:26,代码来源:FifoScheduler.java


示例2: assignRackLocalContainers

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.NodeType; //导入依赖的package包/类
private int assignRackLocalContainers(FiCaSchedulerNode node, 
    FiCaSchedulerApp application, Priority priority) {
  int assignedContainers = 0;
  ResourceRequest request = 
    application.getResourceRequest(priority, node.getRMNode().getRackName());
  if (request != null) {
    // Don't allocate on this rack if the application doens't need containers
    ResourceRequest offSwitchRequest =
        application.getResourceRequest(priority, ResourceRequest.ANY);
    if (offSwitchRequest.getNumContainers() <= 0) {
      return 0;
    }
    
    int assignableContainers = 
      Math.min(
          getMaxAllocatableContainers(application, priority, node, 
              NodeType.RACK_LOCAL), 
              request.getNumContainers());
    assignedContainers = 
      assignContainer(node, application, priority, 
          assignableContainers, request, NodeType.RACK_LOCAL);
  }
  return assignedContainers;
}
 
开发者ID:naver,项目名称:hadoop,代码行数:25,代码来源:FifoScheduler.java


示例3: assignReservedContainer

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.NodeType; //导入依赖的package包/类
private synchronized CSAssignment assignReservedContainer(
    FiCaSchedulerApp application, FiCaSchedulerNode node,
    RMContainer rmContainer, Resource clusterResource) {
  // Do we still need this reservation?
  Priority priority = rmContainer.getReservedPriority();
  if (application.getTotalRequiredResources(priority) == 0) {
    // Release
    return new CSAssignment(application, rmContainer);
  }

  // Try to assign if we have sufficient resources
  assignContainersOnNode(clusterResource, node, application, priority, 
      rmContainer, new ResourceLimits(Resources.none()));
  
  // Doesn't matter... since it's already charged for at time of reservation
  // "re-reservation" is *free*
  return new CSAssignment(Resources.none(), NodeType.NODE_LOCAL);
}
 
开发者ID:naver,项目名称:hadoop,代码行数:19,代码来源:LeafQueue.java


示例4: assign

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.NodeType; //导入依赖的package包/类
public synchronized void assign(List<Container> containers) 
throws IOException, YarnException {
  
  int numContainers = containers.size();
  // Schedule in priority order
  for (Priority priority : requests.keySet()) {
    assign(priority, NodeType.NODE_LOCAL, containers);
    assign(priority, NodeType.RACK_LOCAL, containers);
    assign(priority, NodeType.OFF_SWITCH, containers);

    if (containers.isEmpty()) { 
      break;
    }
  }
  
  int assignedContainers = numContainers - containers.size();
  LOG.info("Application " + applicationId + " assigned " + 
      assignedContainers + "/" + numContainers);
}
 
开发者ID:naver,项目名称:hadoop,代码行数:20,代码来源:Application.java


示例5: testLocalityLevelWithoutDelays

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.NodeType; //导入依赖的package包/类
@Test
/**
 * Ensure that when negative paramaters are given (signaling delay scheduling
 * no tin use), the least restrictive locality level is returned.
 */
public void testLocalityLevelWithoutDelays() {
  FSLeafQueue queue = Mockito.mock(FSLeafQueue.class);
  Priority prio = Mockito.mock(Priority.class);
  Mockito.when(prio.getPriority()).thenReturn(1);

  RMContext rmContext = resourceManager.getRMContext();
  ApplicationAttemptId applicationAttemptId = createAppAttemptId(1, 1);
  FSAppAttempt schedulerApp =
      new FSAppAttempt(scheduler, applicationAttemptId, "user1", queue ,
          null, rmContext);
  assertEquals(NodeType.OFF_SWITCH, schedulerApp.getAllowedLocalityLevel(
      prio, 10, -1.0, -1.0));
}
 
开发者ID:naver,项目名称:hadoop,代码行数:19,代码来源:TestFSAppAttempt.java


示例6: createReservedIncreasedCSAssignment

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.NodeType; //导入依赖的package包/类
private CSAssignment createReservedIncreasedCSAssignment(
    SchedContainerChangeRequest request) {
  CSAssignment assignment =
      new CSAssignment(request.getDeltaCapacity(), NodeType.NODE_LOCAL, null,
          application, false, false);
  Resources.addTo(assignment.getAssignmentInformation().getReserved(),
      request.getDeltaCapacity());
  assignment.getAssignmentInformation().incrReservations();
  assignment.getAssignmentInformation().addReservationDetails(
      request.getContainerId(), application.getCSLeafQueue().getQueuePath());
  assignment.setIncreasedAllocation(true);
  
  LOG.info("Reserved increase container request:" + request.toString());
  
  return assignment;
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:17,代码来源:IncreaseContainerAllocator.java


示例7: createSuccessfullyIncreasedCSAssignment

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.NodeType; //导入依赖的package包/类
private CSAssignment createSuccessfullyIncreasedCSAssignment(
    SchedContainerChangeRequest request, boolean fromReservation) {
  CSAssignment assignment =
      new CSAssignment(request.getDeltaCapacity(), NodeType.NODE_LOCAL, null,
          application, false, fromReservation);
  Resources.addTo(assignment.getAssignmentInformation().getAllocated(),
      request.getDeltaCapacity());
  assignment.getAssignmentInformation().incrAllocations();
  assignment.getAssignmentInformation().addAllocationDetails(
      request.getContainerId(), application.getCSLeafQueue().getQueuePath());
  assignment.setIncreasedAllocation(true);
  
  // notify application
  application
      .getCSLeafQueue()
      .getOrderingPolicy()
      .containerAllocated(application,
          application.getRMContainer(request.getContainerId()));

  LOG.info("Approved increase container request:" + request.toString()
      + " fromReservation=" + fromReservation);    
  
  return assignment;
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:25,代码来源:IncreaseContainerAllocator.java


示例8: assignReservedContainer

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.NodeType; //导入依赖的package包/类
private synchronized CSAssignment assignReservedContainer(
    FiCaSchedulerApp application, FiCaSchedulerNode node,
    RMContainer rmContainer, Resource clusterResource) {
  // Do we still need this reservation?
  Priority priority = rmContainer.getReservedPriority();
  //优先级是否已经达到要求以分配reservation 的 containers
  if (application.getTotalRequiredResources(priority) == 0) {
    // Release
    return new CSAssignment(application, rmContainer);
  }

  // Try to assign if we have sufficient resources
  assignContainersOnNode(clusterResource, node, application, priority, 
      rmContainer, new ResourceLimits(Resources.none()));
  
  // Doesn't matter... since it's already charged for at time of reservation
  // "re-reservation" is *free*
  return new CSAssignment(Resources.none(), NodeType.NODE_LOCAL);
}
 
开发者ID:yncxcw,项目名称:big-c,代码行数:20,代码来源:LeafQueue.java


示例9: assignReservedContainer

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.NodeType; //导入依赖的package包/类
private synchronized CSAssignment 
assignReservedContainer(FiCaSchedulerApp application, 
    FiCaSchedulerNode node, RMContainer rmContainer, Resource clusterResource) {
  // Do we still need this reservation?
  Priority priority = rmContainer.getReservedPriority();
  if (application.getTotalRequiredResources(priority) == 0) {
    // Release
    return new CSAssignment(application, rmContainer);
  }

  // Try to assign if we have sufficient resources
  assignContainersOnNode(clusterResource, node, application, priority, 
      rmContainer, false);
  
  // Doesn't matter... since it's already charged for at time of reservation
  // "re-reservation" is *free*
  return new CSAssignment(Resources.none(), NodeType.NODE_LOCAL);
}
 
开发者ID:Nextzero,项目名称:hadoop-2.6.0-cdh5.4.3,代码行数:19,代码来源:LeafQueue.java


示例10: assignNodeLocalContainers

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.NodeType; //导入依赖的package包/类
private int assignNodeLocalContainers(FiCaSchedulerNode node, 
    FiCaSchedulerApp application, Priority priority) {
  int assignedContainers = 0;
  ResourceRequest request = 
    application.getResourceRequest(priority, node.getHostName());
  if (request != null) {
    // Don't allocate on this node if we don't need containers on this rack
    ResourceRequest rackRequest =
        application.getResourceRequest(priority, 
            node.getRMNode().getRackName());
    if (rackRequest == null || rackRequest.getNumContainers() <= 0) {
      return 0;
    }
    
    int assignableContainers = 
      Math.min(
          getMaxAllocatableContainers(application, priority, node, 
              NodeType.NODE_LOCAL), 
              request.getNumContainers());
    assignedContainers = 
      assignContainer(node, application, priority, 
          assignableContainers, request, NodeType.NODE_LOCAL);
  }
  return assignedContainers;
}
 
开发者ID:ict-carch,项目名称:hadoop-plus,代码行数:26,代码来源:FifoScheduler.java


示例11: assignReservedContainer

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.NodeType; //导入依赖的package包/类
private synchronized CSAssignment 
assignReservedContainer(FiCaSchedulerApp application, 
    FiCaSchedulerNode node, RMContainer rmContainer, Resource clusterResource) {
  // Do we still need this reservation?
  Priority priority = rmContainer.getReservedPriority();
  if (application.getTotalRequiredResources(priority) == 0) {
    // Release
    return new CSAssignment(application, rmContainer);
  }

  // Try to assign if we have sufficient resources
  assignContainersOnNode(clusterResource, node, application, priority, 
      rmContainer);
  
  // Doesn't matter... since it's already charged for at time of reservation
  // "re-reservation" is *free*
  return new CSAssignment(Resources.none(), NodeType.NODE_LOCAL);
}
 
开发者ID:ict-carch,项目名称:hadoop-plus,代码行数:19,代码来源:LeafQueue.java


示例12: testLocalityLevelWithoutDelays

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.NodeType; //导入依赖的package包/类
@Test
/**
 * Ensure that when negative paramaters are given (signaling delay scheduling
 * no tin use), the least restrictive locality level is returned.
 */
public void testLocalityLevelWithoutDelays() {
  Queue queue = Mockito.mock(Queue.class);
  Priority prio = Mockito.mock(Priority.class);
  Mockito.when(prio.getPriority()).thenReturn(1);

  ApplicationAttemptId applicationAttemptId = createAppAttemptId(1, 1);
  FSSchedulerApp schedulerApp =
      new FSSchedulerApp(applicationAttemptId, "user1", queue , null, null);
  assertEquals(NodeType.OFF_SWITCH, schedulerApp.getAllowedLocalityLevel(
      prio, 10, -1.0, -1.0));
}
 
开发者ID:ict-carch,项目名称:hadoop-plus,代码行数:17,代码来源:TestFSSchedulerApp.java


示例13: createReservedIncreasedCSAssignment

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.NodeType; //导入依赖的package包/类
private CSAssignment createReservedIncreasedCSAssignment(
    SchedContainerChangeRequest request) {
  CSAssignment assignment =
      new CSAssignment(request.getDeltaCapacity(), NodeType.NODE_LOCAL, null,
          application, CSAssignment.SkippedType.NONE, false);
  Resources.addTo(assignment.getAssignmentInformation().getReserved(),
      request.getDeltaCapacity());
  assignment.getAssignmentInformation().incrReservations();
  assignment.getAssignmentInformation().addReservationDetails(
      request.getContainerId(), application.getCSLeafQueue().getQueuePath());
  assignment.setIncreasedAllocation(true);
  
  LOG.info("Reserved increase container request:" + request.toString());
  
  return assignment;
}
 
开发者ID:hopshadoop,项目名称:hops,代码行数:17,代码来源:IncreaseContainerAllocator.java


示例14: createSuccessfullyIncreasedCSAssignment

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.NodeType; //导入依赖的package包/类
private CSAssignment createSuccessfullyIncreasedCSAssignment(
    SchedContainerChangeRequest request, boolean fromReservation) {
  CSAssignment assignment =
      new CSAssignment(request.getDeltaCapacity(), NodeType.NODE_LOCAL, null,
          application, CSAssignment.SkippedType.NONE, fromReservation);
  Resources.addTo(assignment.getAssignmentInformation().getAllocated(),
      request.getDeltaCapacity());
  assignment.getAssignmentInformation().incrAllocations();
  assignment.getAssignmentInformation().addAllocationDetails(
      request.getContainerId(), application.getCSLeafQueue().getQueuePath());
  assignment.setIncreasedAllocation(true);
  
  // notify application
  application
      .getCSLeafQueue()
      .getOrderingPolicy()
      .containerAllocated(application,
          application.getRMContainer(request.getContainerId()));

  LOG.info("Approved increase container request:" + request.toString()
      + " fromReservation=" + fromReservation);    
  
  return assignment;
}
 
开发者ID:hopshadoop,项目名称:hops,代码行数:25,代码来源:IncreaseContainerAllocator.java


示例15: getMaxAllocatableContainers

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.NodeType; //导入依赖的package包/类
private int getMaxAllocatableContainers(FiCaSchedulerApp application,
    Priority priority, FiCaSchedulerNode node, NodeType type) {
  int maxContainers = 0;
  
  ResourceRequest offSwitchRequest = 
    application.getResourceRequest(priority, ResourceRequest.ANY);
  if (offSwitchRequest != null) {
    maxContainers = offSwitchRequest.getNumContainers();
  }

  if (type == NodeType.OFF_SWITCH) {
    return maxContainers;
  }

  if (type == NodeType.RACK_LOCAL) {
    ResourceRequest rackLocalRequest = 
      application.getResourceRequest(priority, node.getRMNode().getRackName());
    if (rackLocalRequest == null) {
      return maxContainers;
    }

    maxContainers = Math.min(maxContainers, rackLocalRequest.getNumContainers());
  }

  if (type == NodeType.NODE_LOCAL) {
    ResourceRequest nodeLocalRequest = 
      application.getResourceRequest(priority, node.getRMNode().getNodeAddress());
    if (nodeLocalRequest != null) {
      maxContainers = Math.min(maxContainers, nodeLocalRequest.getNumContainers());
    }
  }

  return maxContainers;
}
 
开发者ID:naver,项目名称:hadoop,代码行数:35,代码来源:FifoScheduler.java


示例16: assignOffSwitchContainers

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.NodeType; //导入依赖的package包/类
private int assignOffSwitchContainers(FiCaSchedulerNode node, 
    FiCaSchedulerApp application, Priority priority) {
  int assignedContainers = 0;
  ResourceRequest request = 
    application.getResourceRequest(priority, ResourceRequest.ANY);
  if (request != null) {
    assignedContainers = 
      assignContainer(node, application, priority, 
          request.getNumContainers(), request, NodeType.OFF_SWITCH);
  }
  return assignedContainers;
}
 
开发者ID:naver,项目名称:hadoop,代码行数:13,代码来源:FifoScheduler.java


示例17: getAllowedLocalityLevel

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.NodeType; //导入依赖的package包/类
/**
 * Return the level at which we are allowed to schedule containers, given the
 * current size of the cluster and thresholds indicating how many nodes to
 * fail at (as a fraction of cluster size) before relaxing scheduling
 * constraints.
 */
public synchronized NodeType getAllowedLocalityLevel(Priority priority,
    int numNodes, double nodeLocalityThreshold, double rackLocalityThreshold) {
  // upper limit on threshold
  if (nodeLocalityThreshold > 1.0) { nodeLocalityThreshold = 1.0; }
  if (rackLocalityThreshold > 1.0) { rackLocalityThreshold = 1.0; }

  // If delay scheduling is not being used, can schedule anywhere
  if (nodeLocalityThreshold < 0.0 || rackLocalityThreshold < 0.0) {
    return NodeType.OFF_SWITCH;
  }

  // Default level is NODE_LOCAL
  if (!allowedLocalityLevel.containsKey(priority)) {
    allowedLocalityLevel.put(priority, NodeType.NODE_LOCAL);
    return NodeType.NODE_LOCAL;
  }

  NodeType allowed = allowedLocalityLevel.get(priority);

  // If level is already most liberal, we're done
  if (allowed.equals(NodeType.OFF_SWITCH)) return NodeType.OFF_SWITCH;

  double threshold = allowed.equals(NodeType.NODE_LOCAL) ? nodeLocalityThreshold :
    rackLocalityThreshold;

  // Relax locality constraints once we've surpassed threshold.
  if (getSchedulingOpportunities(priority) > (numNodes * threshold)) {
    if (allowed.equals(NodeType.NODE_LOCAL)) {
      allowedLocalityLevel.put(priority, NodeType.RACK_LOCAL);
      resetSchedulingOpportunities(priority);
    }
    else if (allowed.equals(NodeType.RACK_LOCAL)) {
      allowedLocalityLevel.put(priority, NodeType.OFF_SWITCH);
      resetSchedulingOpportunities(priority);
    }
  }
  return allowedLocalityLevel.get(priority);
}
 
开发者ID:naver,项目名称:hadoop,代码行数:45,代码来源:FSAppAttempt.java


示例18: resetAllowedLocalityLevel

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.NodeType; //导入依赖的package包/类
/**
 * Should be called when the scheduler assigns a container at a higher
 * degree of locality than the current threshold. Reset the allowed locality
 * level to a higher degree of locality.
 */
public synchronized void resetAllowedLocalityLevel(Priority priority,
    NodeType level) {
  NodeType old = allowedLocalityLevel.get(priority);
  LOG.info("Raising locality level from " + old + " to " + level + " at " +
      " priority " + priority);
  allowedLocalityLevel.put(priority, level);
}
 
开发者ID:naver,项目名称:hadoop,代码行数:13,代码来源:FSAppAttempt.java


示例19: CSAssignment

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.NodeType; //导入依赖的package包/类
public CSAssignment(Resource resource, NodeType type) {
  this.resource = resource;
  this.type = type;
  this.application = null;
  this.excessReservation = null;
  this.skipped = false;
}
 
开发者ID:naver,项目名称:hadoop,代码行数:8,代码来源:CSAssignment.java


示例20: assignNodeLocalContainers

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.NodeType; //导入依赖的package包/类
private Resource assignNodeLocalContainers(Resource clusterResource,
    ResourceRequest nodeLocalResourceRequest, FiCaSchedulerNode node,
    FiCaSchedulerApp application, Priority priority,
    RMContainer reservedContainer, MutableObject allocatedContainer,
    ResourceLimits currentResoureLimits) {
  if (canAssign(application, priority, node, NodeType.NODE_LOCAL, 
      reservedContainer)) {
    return assignContainer(clusterResource, node, application, priority,
        nodeLocalResourceRequest, NodeType.NODE_LOCAL, reservedContainer,
        allocatedContainer, currentResoureLimits);
  }
  
  return Resources.none();
}
 
开发者ID:naver,项目名称:hadoop,代码行数:15,代码来源:LeafQueue.java



注:本文中的org.apache.hadoop.yarn.server.resourcemanager.scheduler.NodeType类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
Java OutputFieldsGetter类代码示例发布时间:2022-05-22
下一篇:
Java RsWithStatus类代码示例发布时间:2022-05-22
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap