• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

Java FSQueue类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue的典型用法代码示例。如果您正苦于以下问题:Java FSQueue类的具体用法?Java FSQueue怎么用?Java FSQueue使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



FSQueue类属于org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair包,在下文中一共展示了FSQueue类的18个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: handleFixedFairShares

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue; //导入依赖的package包/类
/**
 * Helper method to handle Schedulabes with fixed fairshares.
 * Returns the resources taken by fixed fairshare schedulables,
 * and adds the remaining to the passed nonFixedSchedulables.
 */
private static int handleFixedFairShares(
    Collection<? extends Schedulable> schedulables,
    Collection<Schedulable> nonFixedSchedulables,
    boolean isSteadyShare, ResourceType type) {
  int totalResource = 0;

  for (Schedulable sched : schedulables) {
    int fixedShare = getFairShareIfFixed(sched, isSteadyShare, type);
    if (fixedShare < 0) {
      nonFixedSchedulables.add(sched);
    } else {
      setResourceValue(fixedShare,
          isSteadyShare
              ? ((FSQueue)sched).getSteadyFairShare()
              : sched.getFairShare(),
          type);
      totalResource = (int) Math.min((long)totalResource + (long)fixedShare,
          Integer.MAX_VALUE);
    }
  }
  return totalResource;
}
 
开发者ID:naver,项目名称:hadoop,代码行数:28,代码来源:ComputeFairShares.java


示例2: getFairShareIfFixed

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue; //导入依赖的package包/类
/**
 * Get the fairshare for the {@link Schedulable} if it is fixed, -1 otherwise.
 *
 * The fairshare is fixed if either the maxShare is 0, weight is 0,
 * or the Schedulable is not active for instantaneous fairshare.
 */
private static int getFairShareIfFixed(Schedulable sched,
    boolean isSteadyShare, ResourceType type) {

  // Check if maxShare is 0
  if (getResourceValue(sched.getMaxShare(), type) <= 0) {
    return 0;
  }

  // For instantaneous fairshares, check if queue is active
  if (!isSteadyShare &&
      (sched instanceof FSQueue) && !((FSQueue)sched).isActive()) {
    return 0;
  }

  // Check if weight is 0
  if (sched.getWeights().getWeight(type) <= 0) {
    int minShare = getResourceValue(sched.getMinShare(), type);
    return (minShare <= 0) ? 0 : minShare;
  }

  return -1;
}
 
开发者ID:naver,项目名称:hadoop,代码行数:29,代码来源:ComputeFairShares.java


示例3: getChildQueues

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue; //导入依赖的package包/类
protected FairSchedulerQueueInfoList getChildQueues(FSQueue queue,
                                                    FairScheduler scheduler) {
  // Return null to omit 'childQueues' field from the return value of
  // REST API if it is empty. We omit the field to keep the consistency
  // with CapacitySchedulerQueueInfo, which omits 'queues' field if empty.
  Collection<FSQueue> children = queue.getChildQueues();
  if (children.isEmpty()) {
    return null;
  }
  FairSchedulerQueueInfoList list = new FairSchedulerQueueInfoList();
  for (FSQueue child : children) {
    if (child instanceof FSLeafQueue) {
      list.addToQueueInfoList(
          new FairSchedulerLeafQueueInfo((FSLeafQueue) child, scheduler));
    } else {
      list.addToQueueInfoList(
          new FairSchedulerQueueInfo(child, scheduler));
    }
  }
  return list;
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:22,代码来源:FairSchedulerQueueInfo.java


示例4: testEmptyChildQueues

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue; //导入依赖的package包/类
@Test
public void testEmptyChildQueues() throws Exception {
  FairSchedulerConfiguration conf = new FairSchedulerConfiguration();
  FairScheduler scheduler = mock(FairScheduler.class);
  AllocationConfiguration allocConf = new AllocationConfiguration(conf);
  when(scheduler.getAllocationConfiguration()).thenReturn(allocConf);
  when(scheduler.getConf()).thenReturn(conf);
  when(scheduler.getClusterResource()).thenReturn(Resource.newInstance(1, 1));
  SystemClock clock = new SystemClock();
  when(scheduler.getClock()).thenReturn(clock);
  QueueManager queueManager = new QueueManager(scheduler);
  queueManager.initialize(conf);

  FSQueue testQueue = queueManager.getLeafQueue("test", true);
  FairSchedulerQueueInfo queueInfo =
      new FairSchedulerQueueInfo(testQueue, scheduler);
  Collection<FairSchedulerQueueInfo> childQueues =
      queueInfo.getChildQueues();
  Assert.assertNotNull(childQueues);
  Assert.assertEquals("Child QueueInfo was not empty", 0, childQueues.size());
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:22,代码来源:TestFairSchedulerQueueInfo.java


示例5: handleFixedFairShares

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue; //导入依赖的package包/类
/**
 * Helper method to handle Schedulabes with fixed fairshares.
 * Returns the resources taken by fixed fairshare schedulables,
 * and adds the remaining to the passed nonFixedSchedulables.
 */
private static int handleFixedFairShares(
    Collection<? extends Schedulable> schedulables,
    Collection<Schedulable> nonFixedSchedulables,
    boolean isSteadyShare, ResourceType type) {
  int totalResource = 0;

  for (Schedulable sched : schedulables) {
    long fixedShare = getFairShareIfFixed(sched, isSteadyShare, type);
    if (fixedShare < 0) {
      nonFixedSchedulables.add(sched);
    } else {
      setResourceValue(fixedShare,
          isSteadyShare
              ? ((FSQueue)sched).getSteadyFairShare()
              : sched.getFairShare(),
          type);
      totalResource = (int) Math.min((long)totalResource + (long)fixedShare,
          Integer.MAX_VALUE);
    }
  }
  return totalResource;
}
 
开发者ID:hopshadoop,项目名称:hops,代码行数:28,代码来源:ComputeFairShares.java


示例6: getFairShareIfFixed

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue; //导入依赖的package包/类
/**
 * Get the fairshare for the {@link Schedulable} if it is fixed, -1 otherwise.
 *
 * The fairshare is fixed if either the maxShare is 0, weight is 0,
 * or the Schedulable is not active for instantaneous fairshare.
 */
private static long getFairShareIfFixed(Schedulable sched,
    boolean isSteadyShare, ResourceType type) {

  // Check if maxShare is 0
  if (getResourceValue(sched.getMaxShare(), type) <= 0) {
    return 0;
  }

  // For instantaneous fairshares, check if queue is active
  if (!isSteadyShare &&
      (sched instanceof FSQueue) && !((FSQueue)sched).isActive()) {
    return 0;
  }

  // Check if weight is 0
  if (sched.getWeights().getWeight(type) <= 0) {
    long minShare = getResourceValue(sched.getMinShare(), type);
    return (minShare <= 0) ? 0 : minShare;
  }

  return -1;
}
 
开发者ID:hopshadoop,项目名称:hops,代码行数:29,代码来源:ComputeFairShares.java


示例7: computeSteadyShares

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue; //导入依赖的package包/类
@Override
public void computeSteadyShares(Collection<? extends FSQueue> queues,
    Resource totalResources) {
  for (ResourceType type : ResourceType.values()) {
    ComputeFairShares.computeSteadyShares(queues, totalResources, type);
  }
}
 
开发者ID:naver,项目名称:hadoop,代码行数:8,代码来源:DominantResourceFairnessPolicy.java


示例8: FairSchedulerQueueInfo

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue; //导入依赖的package包/类
public FairSchedulerQueueInfo(FSQueue queue, FairScheduler scheduler) {
  AllocationConfiguration allocConf = scheduler.getAllocationConfiguration();
  
  queueName = queue.getName();
  schedulingPolicy = queue.getPolicy().getName();
  
  clusterResources = new ResourceInfo(scheduler.getClusterResource());
  
  usedResources = new ResourceInfo(queue.getResourceUsage());
  fractionMemUsed = (float)usedResources.getMemory() /
      clusterResources.getMemory();

  steadyFairResources = new ResourceInfo(queue.getSteadyFairShare());
  fairResources = new ResourceInfo(queue.getFairShare());
  minResources = new ResourceInfo(queue.getMinShare());
  maxResources = new ResourceInfo(queue.getMaxShare());
  maxResources = new ResourceInfo(
      Resources.componentwiseMin(queue.getMaxShare(),
          scheduler.getClusterResource()));

  fractionMemSteadyFairShare =
      (float)steadyFairResources.getMemory() / clusterResources.getMemory();
  fractionMemFairShare = (float) fairResources.getMemory()
      / clusterResources.getMemory();
  fractionMemMinShare = (float)minResources.getMemory() / clusterResources.getMemory();
  fractionMemMaxShare = (float)maxResources.getMemory() / clusterResources.getMemory();
  
  maxApps = allocConf.getQueueMaxApps(queueName);

  pendingContainers = queue.getMetrics().getPendingContainers();
  allocatedContainers = queue.getMetrics().getAllocatedContainers();
  reservedContainers = queue.getMetrics().getReservedContainers();

  if (allocConf.isReservable(queueName) &&
      !allocConf.getShowReservationAsQueues(queueName)) {
    return;
  }

  childQueues = getChildQueues(queue, scheduler);
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:41,代码来源:FairSchedulerQueueInfo.java


示例9: FairSchedulerQueueInfo

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue; //导入依赖的package包/类
public FairSchedulerQueueInfo(FSQueue queue, FairScheduler scheduler) {
  AllocationConfiguration allocConf = scheduler.getAllocationConfiguration();
  
  queueName = queue.getName();
  schedulingPolicy = queue.getPolicy().getName();
  
  clusterResources = new ResourceInfo(scheduler.getClusterResource());
  
  usedResources = new ResourceInfo(queue.getResourceUsage());
  fractionMemUsed = (float)usedResources.getMemory() /
      clusterResources.getMemory();

  steadyFairResources = new ResourceInfo(queue.getSteadyFairShare());
  fairResources = new ResourceInfo(queue.getFairShare());
  minResources = new ResourceInfo(queue.getMinShare());
  maxResources = new ResourceInfo(queue.getMaxShare());
  maxResources = new ResourceInfo(
      Resources.componentwiseMin(queue.getMaxShare(),
          scheduler.getClusterResource()));

  fractionMemSteadyFairShare =
      (float)steadyFairResources.getMemory() / clusterResources.getMemory();
  fractionMemFairShare = (float) fairResources.getMemory()
      / clusterResources.getMemory();
  fractionMemMinShare = (float)minResources.getMemory() / clusterResources.getMemory();
  fractionMemMaxShare = (float)maxResources.getMemory() / clusterResources.getMemory();
  
  maxApps = allocConf.getQueueMaxApps(queueName);
  
  Collection<FSQueue> children = queue.getChildQueues();
  childQueues = new ArrayList<FairSchedulerQueueInfo>();
  for (FSQueue child : children) {
    if (child instanceof FSLeafQueue) {
      childQueues.add(new FairSchedulerLeafQueueInfo((FSLeafQueue)child, scheduler));
    } else {
      childQueues.add(new FairSchedulerQueueInfo(child, scheduler));
    }
  }
}
 
开发者ID:Nextzero,项目名称:hadoop-2.6.0-cdh5.4.3,代码行数:40,代码来源:FairSchedulerQueueInfo.java


示例10: FairSchedulerQueueInfo

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue; //导入依赖的package包/类
public FairSchedulerQueueInfo(FSQueue queue, FairScheduler scheduler) {
  QueueManager manager = scheduler.getQueueManager();
  
  queueName = queue.getName();
      
  clusterResources = scheduler.getClusterCapacity();
  
  usedResources = queue.getResourceUsage();
  fractionMemUsed = (float)usedResources.getMemory() /
      clusterResources.getMemory();
  
  fairResources = queue.getFairShare();
  minResources = queue.getMinShare();
  maxResources = queue.getMaxShare();
  maxResources = Resources.componentwiseMin(maxResources, clusterResources);
  
  fractionMemFairShare = (float)fairResources.getMemory() / clusterResources.getMemory();
  fractionMemMinShare = (float)minResources.getMemory() / clusterResources.getMemory();
  fractionMemMaxShare = (float)maxResources.getMemory() / clusterResources.getMemory();
  
  maxApps = manager.getQueueMaxApps(queueName);
  
  Collection<FSQueue> childQueues = queue.getChildQueues();
  childInfos = new ArrayList<FairSchedulerQueueInfo>();
  for (FSQueue child : childQueues) {
    if (child instanceof FSLeafQueue) {
      childInfos.add(new FairSchedulerLeafQueueInfo((FSLeafQueue)child, scheduler));
    } else {
      childInfos.add(new FairSchedulerQueueInfo(child, scheduler));
    }
  }
}
 
开发者ID:ict-carch,项目名称:hadoop-plus,代码行数:33,代码来源:FairSchedulerQueueInfo.java


示例11: FairSchedulerQueueInfo

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue; //导入依赖的package包/类
public FairSchedulerQueueInfo(FSQueue queue, FairScheduler scheduler) {
  QueueManager manager = scheduler.getQueueManager();
  
  queueName = queue.getName();
  schedulingPolicy = queue.getPolicy().getName();
  
  clusterResources = new ResourceInfo(scheduler.getClusterCapacity());
  
  usedResources = new ResourceInfo(queue.getResourceUsage());
  fractionMemUsed = (float)usedResources.getMemory() /
      clusterResources.getMemory();
  
  fairResources = new ResourceInfo(queue.getFairShare());
  minResources = new ResourceInfo(queue.getMinShare());
  maxResources = new ResourceInfo(queue.getMaxShare());
  maxResources = new ResourceInfo(
      Resources.componentwiseMin(queue.getMaxShare(),
          scheduler.getClusterCapacity()));
  
  fractionMemFairShare = (float)fairResources.getMemory() / clusterResources.getMemory();
  fractionMemMinShare = (float)minResources.getMemory() / clusterResources.getMemory();
  fractionMemMaxShare = (float)maxResources.getMemory() / clusterResources.getMemory();
  
  maxApps = manager.getQueueMaxApps(queueName);
  
  Collection<FSQueue> children = queue.getChildQueues();
  childQueues = new ArrayList<FairSchedulerQueueInfo>();
  for (FSQueue child : children) {
    if (child instanceof FSLeafQueue) {
      childQueues.add(new FairSchedulerLeafQueueInfo((FSLeafQueue)child, scheduler));
    } else {
      childQueues.add(new FairSchedulerQueueInfo(child, scheduler));
    }
  }
}
 
开发者ID:chendave,项目名称:hadoop-TCP,代码行数:36,代码来源:FairSchedulerQueueInfo.java


示例12: FairSchedulerQueueInfo

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue; //导入依赖的package包/类
public FairSchedulerQueueInfo(FSQueue queue, FairScheduler scheduler) {
  AllocationConfiguration allocConf = scheduler.getAllocationConfiguration();
  
  queueName = queue.getName();
  schedulingPolicy = queue.getPolicy().getName();
  
  clusterResources = new ResourceInfo(scheduler.getClusterCapacity());
  
  usedResources = new ResourceInfo(queue.getResourceUsage());
  fractionMemUsed = (float)usedResources.getMemory() /
      clusterResources.getMemory();
  
  fairResources = new ResourceInfo(queue.getFairShare());
  minResources = new ResourceInfo(queue.getMinShare());
  maxResources = new ResourceInfo(queue.getMaxShare());
  maxResources = new ResourceInfo(
      Resources.componentwiseMin(queue.getMaxShare(),
          scheduler.getClusterCapacity()));
  
  fractionMemFairShare = (float)fairResources.getMemory() / clusterResources.getMemory();
  fractionMemMinShare = (float)minResources.getMemory() / clusterResources.getMemory();
  fractionMemMaxShare = (float)maxResources.getMemory() / clusterResources.getMemory();
  
  maxApps = allocConf.getQueueMaxApps(queueName);
  
  Collection<FSQueue> children = queue.getChildQueues();
  childQueues = new ArrayList<FairSchedulerQueueInfo>();
  for (FSQueue child : children) {
    if (child instanceof FSLeafQueue) {
      childQueues.add(new FairSchedulerLeafQueueInfo((FSLeafQueue)child, scheduler));
    } else {
      childQueues.add(new FairSchedulerQueueInfo(child, scheduler));
    }
  }
}
 
开发者ID:Seagate,项目名称:hadoop-on-lustre2,代码行数:36,代码来源:FairSchedulerQueueInfo.java


示例13: FairSchedulerQueueInfo

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue; //导入依赖的package包/类
public FairSchedulerQueueInfo(FSQueue queue, FairScheduler scheduler) {
  AllocationConfiguration allocConf = scheduler.getAllocationConfiguration();
  
  queueName = queue.getName();
  schedulingPolicy = queue.getPolicy().getName();
  
  clusterResources = new ResourceInfo(scheduler.getClusterResource());
  
  usedResources = new ResourceInfo(queue.getResourceUsage());
  fractionMemUsed = (float)usedResources.getMemory() /
      clusterResources.getMemory();

  steadyFairResources = new ResourceInfo(queue.getSteadyFairShare());
  fairResources = new ResourceInfo(queue.getFairShare());
  minResources = new ResourceInfo(queue.getMinShare());
  maxResources = new ResourceInfo(queue.getMaxShare());
  maxResources = new ResourceInfo(
      Resources.componentwiseMin(queue.getMaxShare(),
          scheduler.getClusterResource()));

  fractionMemSteadyFairShare =
      (float)steadyFairResources.getMemory() / clusterResources.getMemory();
  fractionMemFairShare = (float) fairResources.getMemory()
      / clusterResources.getMemory();
  fractionMemMinShare = (float)minResources.getMemory() / clusterResources.getMemory();
  fractionMemMaxShare = (float)maxResources.getMemory() / clusterResources.getMemory();
  
  maxApps = allocConf.getQueueMaxApps(queueName);

  childQueues = new ArrayList<FairSchedulerQueueInfo>();
  if (allocConf.isReservable(queueName) &&
      !allocConf.getShowReservationAsQueues(queueName)) {
    return;
  }

  Collection<FSQueue> children = queue.getChildQueues();
  for (FSQueue child : children) {
    if (child instanceof FSLeafQueue) {
      childQueues.add(new FairSchedulerLeafQueueInfo((FSLeafQueue)child, scheduler));
    } else {
      childQueues.add(new FairSchedulerQueueInfo(child, scheduler));
    }
  }
}
 
开发者ID:naver,项目名称:hadoop,代码行数:45,代码来源:FairSchedulerQueueInfo.java


示例14: getChildReservationQueues

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue; //导入依赖的package包/类
@Override
protected List<? extends Queue> getChildReservationQueues(Queue queue) {
  FSQueue planQueue = (FSQueue)queue;
  List<FSQueue> childQueues = planQueue.getChildQueues();
  return childQueues;
}
 
开发者ID:naver,项目名称:hadoop,代码行数:7,代码来源:FairSchedulerPlanFollower.java


示例15: computeSteadyShares

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue; //导入依赖的package包/类
@Override
public void computeSteadyShares(Collection<? extends FSQueue> queues,
    Resource totalResources) {
  ComputeFairShares.computeSteadyShares(queues, totalResources,
      ResourceType.MEMORY);
}
 
开发者ID:naver,项目名称:hadoop,代码行数:7,代码来源:FairSharePolicy.java


示例16: computeSteadyShares

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue; //导入依赖的package包/类
@Override
public void computeSteadyShares(Collection<? extends FSQueue> queues,
    Resource totalResources) {
  // Nothing needs to do, as leaf queue doesn't have to calculate steady
  // fair shares for applications.
}
 
开发者ID:naver,项目名称:hadoop,代码行数:7,代码来源:FifoPolicy.java


示例17: verifyCapacity

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue; //导入依赖的package包/类
@Override
protected void verifyCapacity(Queue defQ) {
  assertTrue(((FSQueue) defQ).getWeights().getWeight(ResourceType.MEMORY) >
      0.9);
}
 
开发者ID:naver,项目名称:hadoop,代码行数:6,代码来源:TestFairSchedulerPlanFollower.java


示例18: verifyCapacity

import org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue; //导入依赖的package包/类
@Override
protected void verifyCapacity(Queue defQ) {
  assertTrue(((FSQueue) defQ).getWeights().getWeight(ResourceType.MEMORY) > 0.9);
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:5,代码来源:TestFairSchedulerPlanFollower.java



注:本文中的org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
Java GetClusterNodesRequestProto类代码示例发布时间:2022-05-22
下一篇:
Java ChannelException类代码示例发布时间:2022-05-22
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap