• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

Java NMNullStateStoreService类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService的典型用法代码示例。如果您正苦于以下问题:Java NMNullStateStoreService类的具体用法?Java NMNullStateStoreService怎么用?Java NMNullStateStoreService使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



NMNullStateStoreService类属于org.apache.hadoop.yarn.server.nodemanager.recovery包,在下文中一共展示了NMNullStateStoreService类的20个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: testMinimumPerDirectoryFileLimit

import org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService; //导入依赖的package包/类
@Test(timeout = 10000)
public void testMinimumPerDirectoryFileLimit() {
  YarnConfiguration conf = new YarnConfiguration();
  conf.set(YarnConfiguration.NM_LOCAL_CACHE_MAX_FILES_PER_DIRECTORY, "1");
  Exception e = null;
  NMContext nmContext =
      new NMContext(new NMContainerTokenSecretManager(conf),
        new NMTokenSecretManagerInNM(), null,
        new ApplicationACLsManager(conf), new NMNullStateStoreService());
  ResourceLocalizationService service =
      new ResourceLocalizationService(null, null, null, null, nmContext);
  try {
    service.init(conf);
  } catch (Exception e1) {
    e = e1;
  }
  Assert.assertNotNull(e);
  Assert.assertEquals(YarnRuntimeException.class, e.getClass());
  Assert.assertEquals(e.getMessage(),
    YarnConfiguration.NM_LOCAL_CACHE_MAX_FILES_PER_DIRECTORY
        + " parameter is configured with a value less than 37.");

}
 
开发者ID:naver,项目名称:hadoop,代码行数:24,代码来源:TestLocalCacheDirectoryManager.java


示例2: createMockTracker

import org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService; //导入依赖的package包/类
LocalResourcesTracker createMockTracker(String user, final long rsrcSize,
    long nRsrcs, long timestamp, long tsstep) {
  Configuration conf = new Configuration();
  ConcurrentMap<LocalResourceRequest,LocalizedResource> trackerResources =
    new ConcurrentHashMap<LocalResourceRequest,LocalizedResource>();
  LocalResourcesTracker ret = spy(new LocalResourcesTrackerImpl(user, null,
    null, trackerResources, false, conf, new NMNullStateStoreService(),null));
  for (int i = 0; i < nRsrcs; ++i) {
    final LocalResourceRequest req = new LocalResourceRequest(
        new Path("file:///" + user + "/rsrc" + i), timestamp + i * tsstep,
        LocalResourceType.FILE, LocalResourceVisibility.PUBLIC, null);
    final long ts = timestamp + i * tsstep;
    final Path p = new Path("file:///local/" + user + "/rsrc" + i);
    LocalizedResource rsrc = new LocalizedResource(req, null) {
      @Override public int getRefCount() { return 0; }
      @Override public long getSize() { return rsrcSize; }
      @Override public Path getLocalPath() { return p; }
      @Override public long getTimestamp() { return ts; }
      @Override
      public ResourceState getState() { return ResourceState.LOCALIZED; }
    };
    trackerResources.put(req, rsrc);
  }
  return ret;
}
 
开发者ID:naver,项目名称:hadoop,代码行数:26,代码来源:TestResourceRetention.java


示例3: testStop

import org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService; //导入依赖的package包/类
@Test
public void testStop() throws Exception {
  NonAggregatingLogHandler aggregatingLogHandler = 
      new NonAggregatingLogHandler(null, null, null,
          new NMNullStateStoreService());

  // It should not throw NullPointerException
  aggregatingLogHandler.stop();

  NonAggregatingLogHandlerWithMockExecutor logHandler = 
      new NonAggregatingLogHandlerWithMockExecutor(null, null, null);
  logHandler.init(new Configuration());
  logHandler.stop();
  verify(logHandler.mockSched).shutdown();
  verify(logHandler.mockSched)
      .awaitTermination(eq(10l), eq(TimeUnit.SECONDS));
  verify(logHandler.mockSched).shutdownNow();
  logHandler.close();
  aggregatingLogHandler.close();
}
 
开发者ID:naver,项目名称:hadoop,代码行数:21,代码来源:TestNonAggregatingLogHandler.java


示例4: testHandlingApplicationFinishedEvent

import org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService; //导入依赖的package包/类
@Test
public void testHandlingApplicationFinishedEvent() throws IOException {
  DeletionService delService = new DeletionService(null);
  NonAggregatingLogHandler aggregatingLogHandler =
      new NonAggregatingLogHandler(new InlineDispatcher(),
          delService,
          dirsHandler,
          new NMNullStateStoreService());

  dirsHandler.init(conf);
  dirsHandler.start();
  delService.init(conf);
  delService.start();
  aggregatingLogHandler.init(conf);
  aggregatingLogHandler.start();
  
  // It should NOT throw RejectedExecutionException
  aggregatingLogHandler.handle(new LogHandlerAppFinishedEvent(appId));
  aggregatingLogHandler.stop();

  // It should NOT throw RejectedExecutionException after stopping
  // handler service.
  aggregatingLogHandler.handle(new LogHandlerAppFinishedEvent(appId));
  aggregatingLogHandler.close();
}
 
开发者ID:naver,项目名称:hadoop,代码行数:26,代码来源:TestNonAggregatingLogHandler.java


示例5: testMinimumPerDirectoryFileLimit

import org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService; //导入依赖的package包/类
@Test(timeout = 10000)
public void testMinimumPerDirectoryFileLimit() {
  YarnConfiguration conf = new YarnConfiguration();
  conf.set(YarnConfiguration.NM_LOCAL_CACHE_MAX_FILES_PER_DIRECTORY, "1");
  Exception e = null;
  NMContext nmContext =
      new NMContext(new NMContainerTokenSecretManager(conf),
        new NMTokenSecretManagerInNM(), null,
        new ApplicationACLsManager(conf), new NMNullStateStoreService(), null);
  ResourceLocalizationService service =
      new ResourceLocalizationService(null, null, null, null, nmContext);
  try {
    service.init(conf);
  } catch (Exception e1) {
    e = e1;
  }
  Assert.assertNotNull(e);
  Assert.assertEquals(YarnRuntimeException.class, e.getClass());
  Assert.assertEquals(e.getMessage(),
    YarnConfiguration.NM_LOCAL_CACHE_MAX_FILES_PER_DIRECTORY
        + " parameter is configured with a value less than 37.");

}
 
开发者ID:yncxcw,项目名称:big-c,代码行数:24,代码来源:TestLocalCacheDirectoryManager.java


示例6: createMockTracker

import org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService; //导入依赖的package包/类
LocalResourcesTracker createMockTracker(String user, final long rsrcSize,
    long nRsrcs, long timestamp, long tsstep) {
  Configuration conf = new Configuration();
  ConcurrentMap<LocalResourceRequest,LocalizedResource> trackerResources =
    new ConcurrentHashMap<LocalResourceRequest,LocalizedResource>();
  LocalResourcesTracker ret = spy(new LocalResourcesTrackerImpl(user, null,
    null, trackerResources, false, conf, new NMNullStateStoreService()));
  for (int i = 0; i < nRsrcs; ++i) {
    final LocalResourceRequest req = new LocalResourceRequest(
        new Path("file:///" + user + "/rsrc" + i), timestamp + i * tsstep,
        LocalResourceType.FILE, LocalResourceVisibility.PUBLIC, null);
    final long ts = timestamp + i * tsstep;
    final Path p = new Path("file:///local/" + user + "/rsrc" + i);
    LocalizedResource rsrc = new LocalizedResource(req, null) {
      @Override public int getRefCount() { return 0; }
      @Override public long getSize() { return rsrcSize; }
      @Override public Path getLocalPath() { return p; }
      @Override public long getTimestamp() { return ts; }
      @Override
      public ResourceState getState() { return ResourceState.LOCALIZED; }
    };
    trackerResources.put(req, rsrc);
  }
  return ret;
}
 
开发者ID:yncxcw,项目名称:big-c,代码行数:26,代码来源:TestResourceRetention.java


示例7: testMinimumPerDirectoryFileLimit

import org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService; //导入依赖的package包/类
@Test
public void testMinimumPerDirectoryFileLimit() {
  YarnConfiguration conf = new YarnConfiguration();
  conf.set(YarnConfiguration.NM_LOCAL_CACHE_MAX_FILES_PER_DIRECTORY, "1");
  Exception e = null;
  NMContext nmContext =
      new NMContext(new NMContainerTokenSecretManager(conf),
        new NMTokenSecretManagerInNM(), null,
        new ApplicationACLsManager(conf), new NMNullStateStoreService());
  ResourceLocalizationService service =
      new ResourceLocalizationService(null, null, null, null, nmContext);
  try {
    service.init(conf);
  } catch (Exception e1) {
    e = e1;
  }
  Assert.assertNotNull(e);
  Assert.assertEquals(YarnRuntimeException.class, e.getClass());
  Assert.assertEquals(e.getMessage(),
    YarnConfiguration.NM_LOCAL_CACHE_MAX_FILES_PER_DIRECTORY
        + " parameter is configured with a value less than 37.");

}
 
开发者ID:hopshadoop,项目名称:hops,代码行数:24,代码来源:TestLocalCacheDirectoryManager.java


示例8: setup

import org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService; //导入依赖的package包/类
@Before
public void setup() throws IOException {
  conf = new Configuration();
  spylfs = spy(FileContext.getLocalFSFileContext().getDefaultFileSystem());
  lfs = FileContext.getFileContext(spylfs, conf);

  String logDir = lfs.makeQualified(new Path(basedir, "logdir ")).toString();
  conf.set(YarnConfiguration.NM_LOG_DIRS, logDir);
  nmContext = new NMContext(new NMContainerTokenSecretManager(
    conf), new NMTokenSecretManagerInNM(), null,
    new ApplicationACLsManager(conf), new NMNullStateStoreService());
}
 
开发者ID:naver,项目名称:hadoop,代码行数:13,代码来源:TestResourceLocalizationService.java


示例9: testFailedDirLogDeletion

import org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService; //导入依赖的package包/类
@Test
public void testFailedDirLogDeletion() throws Exception {

  File[] localLogDirs = getLocalLogDirFiles(this.getClass().getName(), 7);
  final List<String> localLogDirPaths =
      new ArrayList<String>(localLogDirs.length);
  for (int i = 0; i < localLogDirs.length; i++) {
    localLogDirPaths.add(localLogDirs[i].getAbsolutePath());
  }

  String localLogDirsString = StringUtils.join(localLogDirPaths, ",");

  conf.set(YarnConfiguration.NM_LOG_DIRS, localLogDirsString);
  conf.setBoolean(YarnConfiguration.LOG_AGGREGATION_ENABLED, false);
  conf.setLong(YarnConfiguration.NM_LOG_RETAIN_SECONDS, 0l);

  LocalDirsHandlerService mockDirsHandler = mock(LocalDirsHandlerService.class);

  NonAggregatingLogHandler rawLogHandler =
      new NonAggregatingLogHandler(dispatcher, mockDelService,
          mockDirsHandler, new NMNullStateStoreService());
  NonAggregatingLogHandler logHandler = spy(rawLogHandler);
  AbstractFileSystem spylfs =
      spy(FileContext.getLocalFSFileContext().getDefaultFileSystem());
  FileContext lfs = FileContext.getFileContext(spylfs, conf);
  doReturn(lfs).when(logHandler)
    .getLocalFileContext(isA(Configuration.class));
  logHandler.init(conf);
  logHandler.start();
  runMockedFailedDirs(logHandler, appId, user, mockDelService,
    mockDirsHandler, conf, spylfs, lfs, localLogDirs);
  logHandler.close();
}
 
开发者ID:naver,项目名称:hadoop,代码行数:34,代码来源:TestNonAggregatingLogHandler.java


示例10: setup

import org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService; //导入依赖的package包/类
@Before
public void setup() throws IOException {
  conf = new Configuration();
  spylfs = spy(FileContext.getLocalFSFileContext().getDefaultFileSystem());
  lfs = FileContext.getFileContext(spylfs, conf);

  String logDir = lfs.makeQualified(new Path(basedir, "logdir ")).toString();
  conf.set(YarnConfiguration.NM_LOG_DIRS, logDir);
  nmContext = new NMContext(new NMContainerTokenSecretManager(
    conf), new NMTokenSecretManagerInNM(), null,
    new ApplicationACLsManager(conf), new NMNullStateStoreService(), null);
}
 
开发者ID:yncxcw,项目名称:big-c,代码行数:13,代码来源:TestResourceLocalizationService.java


示例11: NMContainerTokenSecretManager

import org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService; //导入依赖的package包/类
public NMContainerTokenSecretManager(Configuration conf) {
  this(conf, new NMNullStateStoreService());
}
 
开发者ID:naver,项目名称:hadoop,代码行数:4,代码来源:NMContainerTokenSecretManager.java


示例12: NMTokenSecretManagerInNM

import org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService; //导入依赖的package包/类
public NMTokenSecretManagerInNM() {
  this(new NMNullStateStoreService());
}
 
开发者ID:naver,项目名称:hadoop,代码行数:4,代码来源:NMTokenSecretManagerInNM.java


示例13: DeletionService

import org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService; //导入依赖的package包/类
public DeletionService(ContainerExecutor exec) {
  this(exec, new NMNullStateStoreService());
}
 
开发者ID:naver,项目名称:hadoop,代码行数:4,代码来源:DeletionService.java


示例14: testContainerLogDirs

import org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService; //导入依赖的package包/类
@Test(timeout=30000)
public void testContainerLogDirs() throws IOException, YarnException {
  File absLogDir = new File("target",
    TestNMWebServer.class.getSimpleName() + "LogDir").getAbsoluteFile();
  String logdirwithFile = absLogDir.toURI().toString();
  Configuration conf = new Configuration();
  conf.set(YarnConfiguration.NM_LOG_DIRS, logdirwithFile);
  NodeHealthCheckerService healthChecker = createNodeHealthCheckerService(conf);
  healthChecker.init(conf);
  LocalDirsHandlerService dirsHandler = healthChecker.getDiskHandler();
  NMContext nmContext = new NodeManager.NMContext(null, null, dirsHandler,
      new ApplicationACLsManager(conf), new NMNullStateStoreService());
  // Add an application and the corresponding containers
  RecordFactory recordFactory = RecordFactoryProvider.getRecordFactory(conf);
  String user = "nobody";
  long clusterTimeStamp = 1234;
  ApplicationId appId = BuilderUtils.newApplicationId(recordFactory,
      clusterTimeStamp, 1);
  Application app = mock(Application.class);
  when(app.getUser()).thenReturn(user);
  when(app.getAppId()).thenReturn(appId);
  ApplicationAttemptId appAttemptId = BuilderUtils.newApplicationAttemptId(
      appId, 1);
  ContainerId container1 = BuilderUtils.newContainerId(recordFactory, appId,
      appAttemptId, 0);
  nmContext.getApplications().put(appId, app);

  MockContainer container =
      new MockContainer(appAttemptId, new AsyncDispatcher(), conf, user,
          appId, 1);
  container.setState(ContainerState.RUNNING);
  nmContext.getContainers().put(container1, container);   
  List<File> files = null;
  files = ContainerLogsUtils.getContainerLogDirs(container1, user, nmContext);
  Assert.assertTrue(!(files.get(0).toString().contains("file:")));
  
  // After container is completed, it is removed from nmContext
  nmContext.getContainers().remove(container1);
  Assert.assertNull(nmContext.getContainers().get(container1));
  files = ContainerLogsUtils.getContainerLogDirs(container1, user, nmContext);
  Assert.assertTrue(!(files.get(0).toString().contains("file:")));

  // Create a new context to check if correct container log dirs are fetched
  // on full disk.
  LocalDirsHandlerService dirsHandlerForFullDisk = spy(dirsHandler);
  // good log dirs are empty and nm log dir is in the full log dir list.
  when(dirsHandlerForFullDisk.getLogDirs()).
      thenReturn(new ArrayList<String>());
  when(dirsHandlerForFullDisk.getLogDirsForRead()).
      thenReturn(Arrays.asList(new String[] {absLogDir.getAbsolutePath()}));
  nmContext = new NodeManager.NMContext(null, null, dirsHandlerForFullDisk,
      new ApplicationACLsManager(conf), new NMNullStateStoreService());
  nmContext.getApplications().put(appId, app);
  container.setState(ContainerState.RUNNING);
  nmContext.getContainers().put(container1, container);
  List<File> dirs =
      ContainerLogsUtils.getContainerLogDirs(container1, user, nmContext);
  File containerLogDir = new File(absLogDir, appId + "/" + container1);
  Assert.assertTrue(dirs.contains(containerLogDir));
}
 
开发者ID:naver,项目名称:hadoop,代码行数:61,代码来源:TestContainerLogsPage.java


示例15: testConsistency

import org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService; //导入依赖的package包/类
@Test(timeout=10000)
@SuppressWarnings("unchecked")
public void testConsistency() {
  String user = "testuser";
  DrainDispatcher dispatcher = null;
  try {
    Configuration conf = new Configuration();
    dispatcher = createDispatcher(conf);
    EventHandler<LocalizerEvent> localizerEventHandler = mock(EventHandler.class);
    EventHandler<LocalizerEvent> containerEventHandler = mock(EventHandler.class);
    dispatcher.register(LocalizerEventType.class, localizerEventHandler);
    dispatcher.register(ContainerEventType.class, containerEventHandler);

    ContainerId cId1 = BuilderUtils.newContainerId(1, 1, 1, 1);
    LocalizerContext lc1 = new LocalizerContext(user, cId1, null);
    LocalResourceRequest req1 = createLocalResourceRequest(user, 1, 1,
        LocalResourceVisibility.PUBLIC);
    LocalizedResource lr1 = createLocalizedResource(req1, dispatcher);
    ConcurrentMap<LocalResourceRequest, LocalizedResource> localrsrc = new ConcurrentHashMap<LocalResourceRequest, LocalizedResource>();
    localrsrc.put(req1, lr1);
    LocalResourcesTracker tracker = new LocalResourcesTrackerImpl(user,
        null, dispatcher, localrsrc, false, conf,
        new NMNullStateStoreService(), null);

    ResourceEvent req11Event = new ResourceRequestEvent(req1,
        LocalResourceVisibility.PUBLIC, lc1);

    ResourceEvent rel11Event = new ResourceReleaseEvent(req1, cId1);

    // Localize R1 for C1
    tracker.handle(req11Event);

    dispatcher.await();

    // Verify refCount for R1 is 1
    Assert.assertEquals(1, lr1.getRefCount());

    dispatcher.await();
    verifyTrackedResourceCount(tracker, 1);

    // Localize resource1
    ResourceLocalizedEvent rle = new ResourceLocalizedEvent(req1, new Path(
        "file:///tmp/r1"), 1);
    lr1.handle(rle);
    Assert.assertTrue(lr1.getState().equals(ResourceState.LOCALIZED));
    Assert.assertTrue(createdummylocalizefile(new Path("file:///tmp/r1")));
    LocalizedResource rsrcbefore = tracker.iterator().next();
    File resFile = new File(lr1.getLocalPath().toUri().getRawPath()
        .toString());
    Assert.assertTrue(resFile.exists());
    Assert.assertTrue(resFile.delete());

    // Localize R1 for C1
    tracker.handle(req11Event);

    dispatcher.await();
    lr1.handle(rle);
    Assert.assertTrue(lr1.getState().equals(ResourceState.LOCALIZED));
    LocalizedResource rsrcafter = tracker.iterator().next();
    if (rsrcbefore == rsrcafter) {
      Assert.fail("Localized resource should not be equal");
    }
    // Release resource1
    tracker.handle(rel11Event);
  } finally {
    if (dispatcher != null) {
      dispatcher.stop();
    }
  }
}
 
开发者ID:naver,项目名称:hadoop,代码行数:71,代码来源:TestLocalResourcesTrackerImpl.java


示例16: testResourcePresentInGoodDir

import org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService; //导入依赖的package包/类
@SuppressWarnings("unchecked")
@Test
public void testResourcePresentInGoodDir() throws IOException {
  String user = "testuser";
  DrainDispatcher dispatcher = null;
  try {
    Configuration conf = new Configuration();
    dispatcher = createDispatcher(conf);
    EventHandler<LocalizerEvent> localizerEventHandler =
        mock(EventHandler.class);
    EventHandler<LocalizerEvent> containerEventHandler =
        mock(EventHandler.class);
    dispatcher.register(LocalizerEventType.class, localizerEventHandler);
    dispatcher.register(ContainerEventType.class, containerEventHandler);

    ContainerId cId1 = BuilderUtils.newContainerId(1, 1, 1, 1);
    LocalizerContext lc1 = new LocalizerContext(user, cId1, null);
    LocalResourceRequest req1 =
        createLocalResourceRequest(user, 1, 1, LocalResourceVisibility.PUBLIC);
    LocalResourceRequest req2 =
        createLocalResourceRequest(user, 2, 1, LocalResourceVisibility.PUBLIC);
    LocalizedResource lr1 = createLocalizedResource(req1, dispatcher);
    LocalizedResource lr2 = createLocalizedResource(req2, dispatcher);
    ConcurrentMap<LocalResourceRequest, LocalizedResource> localrsrc =
        new ConcurrentHashMap<LocalResourceRequest, LocalizedResource>();
    localrsrc.put(req1, lr1);
    localrsrc.put(req2, lr2);
    LocalDirsHandlerService dirsHandler = mock(LocalDirsHandlerService.class);
    List<String> goodDirs = new ArrayList<String>();
    // /tmp/somedir2 is bad
    goodDirs.add("/tmp/somedir1/");
    goodDirs.add("/tmp/somedir2");
    Mockito.when(dirsHandler.getLocalDirs()).thenReturn(goodDirs);
    Mockito.when(dirsHandler.getLocalDirsForRead()).thenReturn(goodDirs);
    LocalResourcesTrackerImpl tracker =
        new LocalResourcesTrackerImpl(user, null, dispatcher, localrsrc,
            true , conf, new NMNullStateStoreService(), dirsHandler);
    ResourceEvent req11Event =
        new ResourceRequestEvent(req1, LocalResourceVisibility.PUBLIC, lc1);
    ResourceEvent req21Event =
        new ResourceRequestEvent(req2, LocalResourceVisibility.PUBLIC, lc1);
    // Localize R1 for C1
    tracker.handle(req11Event);
    // Localize R2 for C1
    tracker.handle(req21Event);
    dispatcher.await();
    // Localize resource1
    Path p1 = tracker.getPathForLocalization(req1,
        new Path("/tmp/somedir1"), null);
    Path p2 = tracker.getPathForLocalization(req2,
        new Path("/tmp/somedir2"), null);
    ResourceLocalizedEvent rle1 = new ResourceLocalizedEvent(req1, p1, 1);
    tracker.handle(rle1);
    ResourceLocalizedEvent rle2 = new ResourceLocalizedEvent(req2, p2, 1);
    tracker.handle(rle2);
    dispatcher.await();
    // Remove somedir2 from gooddirs
    Assert.assertTrue(tracker.checkLocalResource(lr2));
    goodDirs.remove(1);
    Assert.assertFalse(tracker.checkLocalResource(lr2));
  } finally {
    if (dispatcher != null) {
      dispatcher.stop();
    }
  }
}
 
开发者ID:naver,项目名称:hadoop,代码行数:67,代码来源:TestLocalResourcesTrackerImpl.java


示例17: testLogDeletion

import org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService; //导入依赖的package包/类
@Test
public void testLogDeletion() throws IOException {
  File[] localLogDirs = getLocalLogDirFiles(this.getClass().getName(), 2);
  String localLogDirsString =
      localLogDirs[0].getAbsolutePath() + ","
          + localLogDirs[1].getAbsolutePath();

  conf.set(YarnConfiguration.NM_LOG_DIRS, localLogDirsString);
  conf.setBoolean(YarnConfiguration.LOG_AGGREGATION_ENABLED, false);
  conf.setLong(YarnConfiguration.NM_LOG_RETAIN_SECONDS, 0l);

  dirsHandler.init(conf);

  NonAggregatingLogHandler rawLogHandler =
      new NonAggregatingLogHandler(dispatcher, mockDelService, dirsHandler,
          new NMNullStateStoreService());
  NonAggregatingLogHandler logHandler = spy(rawLogHandler);
  AbstractFileSystem spylfs =
      spy(FileContext.getLocalFSFileContext().getDefaultFileSystem());
  FileContext lfs = FileContext.getFileContext(spylfs, conf);
  doReturn(lfs).when(logHandler)
    .getLocalFileContext(isA(Configuration.class));
  FsPermission defaultPermission =
      FsPermission.getDirDefault().applyUMask(lfs.getUMask());
  final FileStatus fs =
      new FileStatus(0, true, 1, 0, System.currentTimeMillis(), 0,
        defaultPermission, "", "",
        new Path(localLogDirs[0].getAbsolutePath()));
  doReturn(fs).when(spylfs).getFileStatus(isA(Path.class));

  logHandler.init(conf);
  logHandler.start();

  logHandler.handle(new LogHandlerAppStartedEvent(appId, user, null,
      ContainerLogsRetentionPolicy.ALL_CONTAINERS, null));

  logHandler.handle(new LogHandlerContainerFinishedEvent(container11, 0));

  logHandler.handle(new LogHandlerAppFinishedEvent(appId));

  Path[] localAppLogDirs = new Path[2];
  localAppLogDirs[0] =
      new Path(localLogDirs[0].getAbsolutePath(), appId.toString());
  localAppLogDirs[1] =
      new Path(localLogDirs[1].getAbsolutePath(), appId.toString());

  testDeletionServiceCall(mockDelService, user, 5000, localAppLogDirs);
  logHandler.close();
  for (int i = 0; i < localLogDirs.length; i++) {
    FileUtils.deleteDirectory(localLogDirs[i]);
  }
}
 
开发者ID:naver,项目名称:hadoop,代码行数:53,代码来源:TestNonAggregatingLogHandler.java


示例18: NonAggregatingLogHandlerWithMockExecutor

import org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService; //导入依赖的package包/类
public NonAggregatingLogHandlerWithMockExecutor(Dispatcher dispatcher,
    DeletionService delService, LocalDirsHandlerService dirsHandler) {
  this(dispatcher, delService, dirsHandler, new NMNullStateStoreService());
}
 
开发者ID:naver,项目名称:hadoop,代码行数:5,代码来源:TestNonAggregatingLogHandler.java


示例19: MyNMContext

import org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService; //导入依赖的package包/类
public MyNMContext(
    NMContainerTokenSecretManager containerTokenSecretManager,
    NMTokenSecretManagerInNM nmTokenSecretManager) {
  super(containerTokenSecretManager, nmTokenSecretManager, null, null,
      new NMNullStateStoreService());
}
 
开发者ID:naver,项目名称:hadoop,代码行数:7,代码来源:TestNodeStatusUpdater.java


示例20: testContainerLogFile

import org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService; //导入依赖的package包/类
@Test(timeout=30000)
public void testContainerLogFile() throws IOException, YarnException {
  File absLogDir = new File("target",
      TestNMWebServer.class.getSimpleName() + "LogDir").getAbsoluteFile();
  String logdirwithFile = absLogDir.toURI().toString();
  Configuration conf = new Configuration();
  conf.set(YarnConfiguration.NM_LOG_DIRS, logdirwithFile);
  conf.setFloat(YarnConfiguration.NM_MAX_PER_DISK_UTILIZATION_PERCENTAGE,
      0.0f);
  LocalDirsHandlerService dirsHandler = new LocalDirsHandlerService();
  dirsHandler.init(conf);
  NMContext nmContext = new NodeManager.NMContext(null, null, dirsHandler,
      new ApplicationACLsManager(conf), new NMNullStateStoreService());
  // Add an application and the corresponding containers
  String user = "nobody";
  long clusterTimeStamp = 1234;
  ApplicationId appId = BuilderUtils.newApplicationId(
      clusterTimeStamp, 1);
  Application app = mock(Application.class);
  when(app.getUser()).thenReturn(user);
  when(app.getAppId()).thenReturn(appId);
  ApplicationAttemptId appAttemptId = BuilderUtils.newApplicationAttemptId(
      appId, 1);
  ContainerId containerId = BuilderUtils.newContainerId(
      appAttemptId, 1);
  nmContext.getApplications().put(appId, app);

  MockContainer container =
      new MockContainer(appAttemptId, new AsyncDispatcher(), conf, user,
          appId, 1);
  container.setState(ContainerState.RUNNING);
  nmContext.getContainers().put(containerId, container);
  File containerLogDir = new File(absLogDir,
      ContainerLaunch.getRelativeContainerLogDir(appId.toString(),
          containerId.toString()));
  containerLogDir.mkdirs();
  String fileName = "fileName";
  File containerLogFile = new File(containerLogDir, fileName);
  containerLogFile.createNewFile();
  File file = ContainerLogsUtils.getContainerLogFile(containerId,
      fileName, user, nmContext);
  Assert.assertEquals(containerLogFile.toURI().toString(),
      file.toURI().toString());
  FileUtil.fullyDelete(absLogDir);
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:46,代码来源:TestContainerLogsPage.java



注:本文中的org.apache.hadoop.yarn.server.nodemanager.recovery.NMNullStateStoreService类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
Java CreateTable类代码示例发布时间:2022-05-22
下一篇:
Java Strategy类代码示例发布时间:2022-05-22
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap