本文整理汇总了Java中org.apache.hadoop.mapreduce.v2.app.speculate.TaskRuntimeEstimator类的典型用法代码示例。如果您正苦于以下问题:Java TaskRuntimeEstimator类的具体用法?Java TaskRuntimeEstimator怎么用?Java TaskRuntimeEstimator使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。
TaskRuntimeEstimator类属于org.apache.hadoop.mapreduce.v2.app.speculate包,在下文中一共展示了TaskRuntimeEstimator类的3个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。
示例1: runSpecTest
import org.apache.hadoop.mapreduce.v2.app.speculate.TaskRuntimeEstimator; //导入依赖的package包/类
private Job runSpecTest(boolean mapspec, boolean redspec)
throws IOException, ClassNotFoundException, InterruptedException {
Path first = createTempFile("specexec_map_input1", "a\nz");
Path secnd = createTempFile("specexec_map_input2", "a\nz");
Configuration conf = mrCluster.getConfig();
conf.setBoolean(MRJobConfig.MAP_SPECULATIVE,mapspec);
conf.setBoolean(MRJobConfig.REDUCE_SPECULATIVE,redspec);
conf.setClass(MRJobConfig.MR_AM_TASK_ESTIMATOR,
TestSpecEstimator.class,
TaskRuntimeEstimator.class);
Job job = Job.getInstance(conf);
job.setJarByClass(TestSpeculativeExecution.class);
job.setMapperClass(SpeculativeMapper.class);
job.setReducerClass(SpeculativeReducer.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);
job.setNumReduceTasks(2);
FileInputFormat.setInputPaths(job, first);
FileInputFormat.addInputPath(job, secnd);
FileOutputFormat.setOutputPath(job, TEST_OUT_DIR);
// Delete output directory if it exists.
try {
localFs.delete(TEST_OUT_DIR,true);
} catch (IOException e) {
// ignore
}
// Creates the Job Configuration
job.addFileToClassPath(APP_JAR); // The AppMaster jar itself.
job.setMaxMapAttempts(2);
job.submit();
return job;
}
开发者ID:naver,项目名称:hadoop,代码行数:40,代码来源:TestSpeculativeExecution.java
示例2: testLegacyEstimator
import org.apache.hadoop.mapreduce.v2.app.speculate.TaskRuntimeEstimator; //导入依赖的package包/类
@Test
public void testLegacyEstimator() throws Exception {
TaskRuntimeEstimator specificEstimator = new LegacyTaskRuntimeEstimator();
coreTestEstimator(specificEstimator, 3);
}
开发者ID:naver,项目名称:hadoop,代码行数:6,代码来源:TestRuntimeEstimators.java
示例3: testExponentialEstimator
import org.apache.hadoop.mapreduce.v2.app.speculate.TaskRuntimeEstimator; //导入依赖的package包/类
@Test
public void testExponentialEstimator() throws Exception {
TaskRuntimeEstimator specificEstimator
= new ExponentiallySmoothedTaskRuntimeEstimator();
coreTestEstimator(specificEstimator, 3);
}
开发者ID:naver,项目名称:hadoop,代码行数:7,代码来源:TestRuntimeEstimators.java
注:本文中的org.apache.hadoop.mapreduce.v2.app.speculate.TaskRuntimeEstimator类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。 |
请发表评论