• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

Java ContainerLaunch类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch的典型用法代码示例。如果您正苦于以下问题:Java ContainerLaunch类的具体用法?Java ContainerLaunch怎么用?Java ContainerLaunch使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



ContainerLaunch类属于org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher包,在下文中一共展示了ContainerLaunch类的19个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: getContainerLogFile

import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
/**
 * Finds the log file with the given filename for the given container.
 */
public static File getContainerLogFile(ContainerId containerId,
    String fileName, String remoteUser, Context context) throws YarnException {
  Container container = context.getContainers().get(containerId);
  
  Application application = getApplicationForContainer(containerId, context);
  checkAccess(remoteUser, application, context);
  if (container != null) {
    checkState(container.getContainerState());
  }
  
  try {
    LocalDirsHandlerService dirsHandler = context.getLocalDirsHandler();
    String relativeContainerLogDir = ContainerLaunch.getRelativeContainerLogDir(
        application.getAppId().toString(), containerId.toString());
    Path logPath = dirsHandler.getLogPathToRead(
        relativeContainerLogDir + Path.SEPARATOR + fileName);
    URI logPathURI = new File(logPath.toString()).toURI();
    File logFile = new File(logPathURI.getPath());
    return logFile;
  } catch (IOException e) {
    LOG.warn("Failed to find log file", e);
    throw new NotFoundException("Cannot find this log on the local disk.");
  }
}
 
开发者ID:naver,项目名称:hadoop,代码行数:28,代码来源:ContainerLogsUtils.java


示例2: testKillOnLocalizedWhenContainerLaunched

import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Test
public void testKillOnLocalizedWhenContainerLaunched() throws Exception {
  WrappedContainer wc = null;
  try {
    wc = new WrappedContainer(17, 314159265358979L, 4344, "yak");
    wc.initContainer();
    wc.localizeResources();
    assertEquals(ContainerState.LOCALIZED, wc.c.getContainerState());
    ContainerLaunch launcher = wc.launcher.running.get(wc.c.getContainerId());
    launcher.call();
    wc.drainDispatcherEvents();
    assertEquals(ContainerState.EXITED_WITH_FAILURE,
        wc.c.getContainerState());
    wc.killContainer();
    assertEquals(ContainerState.EXITED_WITH_FAILURE,
        wc.c.getContainerState());
    assertNull(wc.c.getLocalizedResources());
    verifyCleanupCall(wc);
  } finally {
    if (wc != null) {
      wc.finished();
    }
  }
}
 
开发者ID:naver,项目名称:hadoop,代码行数:25,代码来源:TestContainer.java


示例3: getContainerLogFile

import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
/**
 * Finds the log file with the given filename for the given container.
 */
public static File getContainerLogFile(ContainerId containerId,
    String fileName, String remoteUser, Context context, String userFolder) throws YarnException {
  Container container = context.getContainers().get(containerId);
  
  Application application = getApplicationForContainer(containerId, context);
  checkAccess(remoteUser, application, context);
  if (container != null) {
    checkState(container.getContainerState());
  }
  
  try {
    LocalDirsHandlerService dirsHandler = context.getLocalDirsHandler();
    String relativeContainerLogDir = ContainerLaunch.getRelativeContainerLogDir(
        application.getAppId().toString(), containerId.toString(), userFolder);
    Path logPath = dirsHandler.getLogPathToRead(
        relativeContainerLogDir + Path.SEPARATOR + fileName);
    URI logPathURI = new File(logPath.toString()).toURI();
    File logFile = new File(logPathURI.getPath());
    return logFile;
  } catch (IOException e) {
    LOG.warn("Failed to find log file", e);
    throw new NotFoundException("Cannot find this log on the local disk.");
  }
}
 
开发者ID:hopshadoop,项目名称:hops,代码行数:28,代码来源:ContainerLogsUtils.java


示例4: testKillOnLocalizedWhenContainerLaunched

import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Test
public void testKillOnLocalizedWhenContainerLaunched() throws Exception {
  WrappedContainer wc = null;
  try {
    wc = new WrappedContainer(17, 314159265358979L, 4344, "yak", "yakFolder");
    wc.initContainer();
    wc.localizeResources();
    assertEquals(ContainerState.LOCALIZED, wc.c.getContainerState());
    ContainerLaunch launcher = wc.launcher.running.get(wc.c.getContainerId());
    launcher.call();
    wc.drainDispatcherEvents();
    assertEquals(ContainerState.EXITED_WITH_FAILURE,
        wc.c.getContainerState());
    wc.killContainer();
    assertEquals(ContainerState.EXITED_WITH_FAILURE,
        wc.c.getContainerState());
    assertNull(wc.c.getLocalizedResources());
    verifyCleanupCall(wc);
  } finally {
    if (wc != null) {
      wc.finished();
    }
  }
}
 
开发者ID:hopshadoop,项目名称:hops,代码行数:25,代码来源:TestContainer.java


示例5: testKillOnLocalizedWhenContainerNotLaunched

import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Test
public void testKillOnLocalizedWhenContainerNotLaunched() throws Exception {
  WrappedContainer wc = null;
  try {
    wc = new WrappedContainer(17, 314159265358979L, 4344, "yak");
    wc.initContainer();
    wc.localizeResources();
    assertEquals(ContainerState.LOCALIZED, wc.c.getContainerState());
    ContainerLaunch launcher = wc.launcher.running.get(wc.c.getContainerId());
    wc.killContainer();
    assertEquals(ContainerState.KILLING, wc.c.getContainerState());
    launcher.call();
    wc.drainDispatcherEvents();
    assertEquals(ContainerState.CONTAINER_CLEANEDUP_AFTER_KILL,
        wc.c.getContainerState());
    assertNull(wc.c.getLocalizedResources());
    verifyCleanupCall(wc);
  } finally {
    if (wc != null) {
      wc.finished();
    }
  }
}
 
开发者ID:chendave,项目名称:hadoop-TCP,代码行数:24,代码来源:TestContainer.java


示例6: testKillOnLocalizedWhenContainerNotLaunched

import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Test
public void testKillOnLocalizedWhenContainerNotLaunched() throws Exception {
  WrappedContainer wc = null;
  try {
    wc = new WrappedContainer(17, 314159265358979L, 4344, "yak");
    wc.initContainer();
    wc.localizeResources();
    assertEquals(ContainerState.LOCALIZED, wc.c.getContainerState());
    ContainerLaunch launcher = wc.launcher.running.get(wc.c.getContainerId());
    wc.killContainer();
    assertEquals(ContainerState.KILLING, wc.c.getContainerState());
    launcher.call();
    wc.drainDispatcherEvents();
    assertEquals(ContainerState.CONTAINER_CLEANEDUP_AFTER_KILL,
        wc.c.getContainerState());
    assertNull(wc.c.getLocalizedResources());
    verifyCleanupCall(wc);
    wc.c.handle(new ContainerEvent(wc.c.getContainerId(),
        ContainerEventType.CONTAINER_RESOURCES_CLEANEDUP));
    assertEquals(0, metrics.getRunningContainers());
  } finally {
    if (wc != null) {
      wc.finished();
    }
  }
}
 
开发者ID:Seagate,项目名称:hadoop-on-lustre2,代码行数:27,代码来源:TestContainer.java


示例7: writeLocalWrapperScript

import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Override
public void writeLocalWrapperScript(Path launchDst, Path pidFile,
                                    PrintStream pout) {

  String exitCodeFile = ContainerLaunch.getExitCodeFile(
    pidFile.toString());
  String tmpFile = exitCodeFile + ".tmp";
  pout.println("#!/usr/bin/env bash");
  pout.println("bash \"" + sessionScriptPath.toString() + "\"");
  pout.println("rc=$?");
  pout.println("echo $rc > \"" + tmpFile + "\"");
  pout.println("mv -f \"" + tmpFile + "\" \"" + exitCodeFile + "\"");
  pout.println("exit $rc");
}
 
开发者ID:naver,项目名称:hadoop,代码行数:15,代码来源:DockerContainerExecutor.java


示例8: writeLocalWrapperScript

import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Override
public void writeLocalWrapperScript(Path launchDst, Path pidFile,
    PrintStream pout) {
  String exitCodeFile = ContainerLaunch.getExitCodeFile(
      pidFile.toString());
  String tmpFile = exitCodeFile + ".tmp";
  pout.println("#!/bin/bash");
  pout.println("/bin/bash \"" + sessionScriptPath.toString() + "\"");
  pout.println("rc=$?");
  pout.println("echo $rc > \"" + tmpFile + "\"");
  pout.println("/bin/mv -f \"" + tmpFile + "\" \"" + exitCodeFile + "\"");
  pout.println("exit $rc");
}
 
开发者ID:naver,项目名称:hadoop,代码行数:14,代码来源:DefaultContainerExecutor.java


示例9: testKillOnLocalizedWhenContainerNotLaunched

import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Test
public void testKillOnLocalizedWhenContainerNotLaunched() throws Exception {
  WrappedContainer wc = null;
  try {
    wc = new WrappedContainer(17, 314159265358979L, 4344, "yak");
    wc.initContainer();
    wc.localizeResources();
    assertEquals(ContainerState.LOCALIZED, wc.c.getContainerState());
    ContainerLaunch launcher = wc.launcher.running.get(wc.c.getContainerId());
    wc.killContainer();
    assertEquals(ContainerState.KILLING, wc.c.getContainerState());
    launcher.call();
    wc.drainDispatcherEvents();
    assertEquals(ContainerState.CONTAINER_CLEANEDUP_AFTER_KILL,
        wc.c.getContainerState());
    assertNull(wc.c.getLocalizedResources());
    verifyCleanupCall(wc);
    int killed = metrics.getKilledContainers();
    wc.c.handle(new ContainerEvent(wc.c.getContainerId(),
        ContainerEventType.CONTAINER_RESOURCES_CLEANEDUP));
    assertEquals(ContainerState.DONE, wc.c.getContainerState());
    assertEquals(killed + 1, metrics.getKilledContainers());
    assertEquals(0, metrics.getRunningContainers());
  } finally {
    if (wc != null) {
      wc.finished();
    }
  }
}
 
开发者ID:naver,项目名称:hadoop,代码行数:30,代码来源:TestContainer.java


示例10: writeLocalWrapperScript

import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Override
public void writeLocalWrapperScript(Path launchDst, Path pidFile,
  PrintStream pout) {
  String exitCodeFile = ContainerLaunch.getExitCodeFile(
    pidFile.toString());
  String tmpFile = exitCodeFile + ".tmp";
  pout.println("#!/usr/bin/env bash");
  pout.println("bash \"" + sessionScriptPath.toString() + "\"");
  pout.println("rc=$?");
  pout.println("echo $rc > \"" + tmpFile + "\"");
  pout.println("mv -f \"" + tmpFile + "\" \"" + exitCodeFile + "\"");
  pout.println("exit $rc");
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:14,代码来源:DockerContainerExecutor.java


示例11: testKillOnLocalizedWhenContainerNotLaunchedContainerKilled

import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Test
public void testKillOnLocalizedWhenContainerNotLaunchedContainerKilled()
    throws Exception {
  WrappedContainer wc = null;
  try {
    wc = new WrappedContainer(17, 314159265358979L, 4344, "yak");
    wc.initContainer();
    wc.localizeResources();
    assertEquals(ContainerState.LOCALIZED, wc.c.getContainerState());
    ContainerLaunch launcher = wc.launcher.running.get(wc.c.getContainerId());
    wc.killContainer();
    assertEquals(ContainerState.KILLING, wc.c.getContainerState());
    launcher.call();
    wc.drainDispatcherEvents();
    assertEquals(ContainerState.CONTAINER_CLEANEDUP_AFTER_KILL,
        wc.c.getContainerState());
    assertNull(wc.c.getLocalizedResources());
    verifyCleanupCall(wc);
    int killed = metrics.getKilledContainers();
    wc.c.handle(new ContainerEvent(wc.c.getContainerId(),
        ContainerEventType.CONTAINER_RESOURCES_CLEANEDUP));
    assertEquals(ContainerState.DONE, wc.c.getContainerState());
    assertEquals(killed + 1, metrics.getKilledContainers());
    assertEquals(0, metrics.getRunningContainers());
  } finally {
    if (wc != null) {
      wc.finished();
    }
  }
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:31,代码来源:TestContainer.java


示例12: writeLaunchEnv

import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
public void writeLaunchEnv(OutputStream out, Map<String, String> environment, Map<Path, List<String>> resources, List<String> command) throws IOException{
  ContainerLaunch.ShellScriptBuilder sb = ContainerLaunch.ShellScriptBuilder.create();
 
  if (environment != null) {
    for (Map.Entry<String,String> env : environment.entrySet()) {
      sb.env(env.getKey().toString(), env.getValue().toString());
      LOG.info("key:"+env.getKey().toString()+"value:"+env.getValue().toString());
    }
  }
  
  sb.cd("$PWD");
  
  if (resources != null) {
    for (Map.Entry<Path,List<String>> entry : resources.entrySet()) {
      for (String linkName : entry.getValue()) {
        sb.symlink(entry.getKey(), new Path(linkName));
        LOG.info("key:"+entry.getKey().toString()+"value:"+entry.getValue().toString());
      }
    }
  }

  LOG.info("commands "+command);
  
  sb.command(command);

  PrintStream pout = null;
  try {
    pout = new PrintStream(out, false, "UTF-8");
    sb.write(pout);
  } finally {
    if (out != null) {
      out.close();
    }
  }
}
 
开发者ID:yncxcw,项目名称:big-c,代码行数:36,代码来源:ContainerExecutor.java


示例13: writeLaunchEnv

import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
public void writeLaunchEnv(OutputStream out, Map<String, String> environment, Map<Path, List<String>> resources, List<String> command) throws IOException{
  ContainerLaunch.ShellScriptBuilder sb = ContainerLaunch.ShellScriptBuilder.create();
  if (environment != null) {
    for (Map.Entry<String,String> env : environment.entrySet()) {
      sb.env(env.getKey().toString(), env.getValue().toString());
    }
  }
  if (resources != null) {
    for (Map.Entry<Path,List<String>> entry : resources.entrySet()) {
      for (String linkName : entry.getValue()) {
        sb.symlink(entry.getKey(), new Path(linkName));
      }
    }
  }

  sb.command(command);

  PrintStream pout = null;
  try {
    pout = new PrintStream(out);
    sb.write(pout);
  } finally {
    if (out != null) {
      out.close();
    }
  }
}
 
开发者ID:Nextzero,项目名称:hadoop-2.6.0-cdh5.4.3,代码行数:28,代码来源:ContainerExecutor.java


示例14: testKillOnLocalizedWhenContainerNotLaunchedContainerKilled

import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Test
public void testKillOnLocalizedWhenContainerNotLaunchedContainerKilled()
    throws Exception {
  WrappedContainer wc = null;
  try {
    wc = new WrappedContainer(17, 314159265358979L, 4344, "yak", "yakFolder");
    wc.initContainer();
    wc.localizeResources();
    assertEquals(ContainerState.LOCALIZED, wc.c.getContainerState());
    ContainerLaunch launcher = wc.launcher.running.get(wc.c.getContainerId());
    wc.killContainer();
    assertEquals(ContainerState.KILLING, wc.c.getContainerState());
    launcher.call();
    wc.drainDispatcherEvents();
    assertEquals(ContainerState.CONTAINER_CLEANEDUP_AFTER_KILL,
        wc.c.getContainerState());
    assertNull(wc.c.getLocalizedResources());
    verifyCleanupCall(wc);
    int killed = metrics.getKilledContainers();
    wc.c.handle(new ContainerEvent(wc.c.getContainerId(),
        ContainerEventType.CONTAINER_RESOURCES_CLEANEDUP));
    assertEquals(ContainerState.DONE, wc.c.getContainerState());
    assertEquals(killed + 1, metrics.getKilledContainers());
    assertEquals(0, metrics.getRunningContainers());
  } finally {
    if (wc != null) {
      wc.finished();
    }
  }
}
 
开发者ID:hopshadoop,项目名称:hops,代码行数:31,代码来源:TestContainer.java


示例15: reacquireContainer

import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
/**
 * Recover an already existing container. This is a blocking call and returns
 * only when the container exits.  Note that the container must have been
 * activated prior to this call.
 * @param ctx encapsulates information necessary to reacquire container
 * @return The exit code of the pre-existing container
 * @throws IOException
 * @throws InterruptedException 
 */
public int reacquireContainer(ContainerReacquisitionContext ctx)
    throws IOException, InterruptedException {
  Container container = ctx.getContainer();
  String user = ctx.getUser();
  ContainerId containerId = ctx.getContainerId();


  Path pidPath = getPidFilePath(containerId);
  if (pidPath == null) {
    LOG.warn(containerId + " is not active, returning terminated error");
    return ExitCode.TERMINATED.getExitCode();
  }

  String pid = null;
  pid = ProcessIdFileReader.getProcessId(pidPath);
  if (pid == null) {
    throw new IOException("Unable to determine pid for " + containerId);
  }

  LOG.info("Reacquiring " + containerId + " with pid " + pid);
  ContainerLivenessContext livenessContext = new ContainerLivenessContext
      .Builder()
      .setContainer(container)
      .setUser(user)
      .setPid(pid)
      .build();
  while(isContainerAlive(livenessContext)) {
    Thread.sleep(1000);
  }

  // wait for exit code file to appear
  String exitCodeFile = ContainerLaunch.getExitCodeFile(pidPath.toString());
  File file = new File(exitCodeFile);
  final int sleepMsec = 100;
  int msecLeft = 2000;
  while (!file.exists() && msecLeft >= 0) {
    if (!isContainerActive(containerId)) {
      LOG.info(containerId + " was deactivated");
      return ExitCode.TERMINATED.getExitCode();
    }
    
    Thread.sleep(sleepMsec);
    
    msecLeft -= sleepMsec;
  }
  if (msecLeft < 0) {
    throw new IOException("Timeout while waiting for exit code from "
        + containerId);
  }

  try {
    return Integer.parseInt(FileUtils.readFileToString(file).trim());
  } catch (NumberFormatException e) {
    throw new IOException("Error parsing exit code from pid " + pid, e);
  }
}
 
开发者ID:naver,项目名称:hadoop,代码行数:66,代码来源:ContainerExecutor.java


示例16: testContainerLogs

import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Test
public void testContainerLogs() throws IOException {
  WebResource r = resource();
  final ContainerId containerId = BuilderUtils.newContainerId(0, 0, 0, 0);
  final String containerIdStr = BuilderUtils.newContainerId(0, 0, 0, 0)
      .toString();
  final ApplicationAttemptId appAttemptId = containerId.getApplicationAttemptId();
  final ApplicationId appId = appAttemptId.getApplicationId();
  final String appIdStr = appId.toString();
  final String filename = "logfile1";
  final String logMessage = "log message\n";
  nmContext.getApplications().put(appId, new ApplicationImpl(null, "user",
      appId, null, nmContext));
  
  MockContainer container = new MockContainer(appAttemptId,
      new AsyncDispatcher(), new Configuration(), "user", appId, 1);
  container.setState(ContainerState.RUNNING);
  nmContext.getContainers().put(containerId, container);
  
  // write out log file
  Path path = dirsHandler.getLogPathForWrite(
      ContainerLaunch.getRelativeContainerLogDir(
          appIdStr, containerIdStr) + "/" + filename, false);
  
  File logFile = new File(path.toUri().getPath());
  logFile.deleteOnExit();
  assertTrue("Failed to create log dir", logFile.getParentFile().mkdirs());
  PrintWriter pw = new PrintWriter(logFile);
  pw.print(logMessage);
  pw.close();

  // ask for it
  ClientResponse response = r.path("ws").path("v1").path("node")
      .path("containerlogs").path(containerIdStr).path(filename)
      .accept(MediaType.TEXT_PLAIN).get(ClientResponse.class);
  String responseText = response.getEntity(String.class);
  assertEquals(logMessage, responseText);
  
  // ask for file that doesn't exist
  response = r.path("ws").path("v1").path("node")
      .path("containerlogs").path(containerIdStr).path("uhhh")
      .accept(MediaType.TEXT_PLAIN).get(ClientResponse.class);
  Assert.assertEquals(Status.NOT_FOUND.getStatusCode(), response.getStatus());
  responseText = response.getEntity(String.class);
  assertTrue(responseText.contains("Cannot find this log on the local disk."));
  
  // After container is completed, it is removed from nmContext
  nmContext.getContainers().remove(containerId);
  Assert.assertNull(nmContext.getContainers().get(containerId));
  response =
      r.path("ws").path("v1").path("node").path("containerlogs")
          .path(containerIdStr).path(filename).accept(MediaType.TEXT_PLAIN)
          .get(ClientResponse.class);
  responseText = response.getEntity(String.class);
  assertEquals(logMessage, responseText);
}
 
开发者ID:naver,项目名称:hadoop,代码行数:57,代码来源:TestNMWebServices.java


示例17: writeLaunchEnv

import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Override
/**
 * Filter the environment variables that may conflict with the ones set in
 * the docker image and write them out to an OutputStream.
 */
public void writeLaunchEnv(OutputStream out, Map<String, String> environment,
  Map<Path, List<String>> resources, List<String> command, Path logDir)
  throws IOException {
  ContainerLaunch.ShellScriptBuilder sb =
    ContainerLaunch.ShellScriptBuilder.create();

  //Remove environments that may conflict with the ones in Docker image.
  Set<String> exclusionSet = new HashSet<String>();
  exclusionSet.add(YarnConfiguration.NM_DOCKER_CONTAINER_EXECUTOR_IMAGE_NAME);
  exclusionSet.add(ApplicationConstants.Environment.HADOOP_YARN_HOME.name());
  exclusionSet.add(ApplicationConstants.Environment.HADOOP_COMMON_HOME.name());
  exclusionSet.add(ApplicationConstants.Environment.HADOOP_HDFS_HOME.name());
  exclusionSet.add(ApplicationConstants.Environment.HADOOP_CONF_DIR.name());
  exclusionSet.add(ApplicationConstants.Environment.JAVA_HOME.name());

  if (environment != null) {
    for (Map.Entry<String,String> env : environment.entrySet()) {
      if (!exclusionSet.contains(env.getKey())) {
        sb.env(env.getKey().toString(), env.getValue().toString());
      }
    }
  }
  if (resources != null) {
    for (Map.Entry<Path,List<String>> entry : resources.entrySet()) {
      for (String linkName : entry.getValue()) {
        sb.symlink(entry.getKey(), new Path(linkName));
      }
    }
  }

  // dump debugging information if configured
  if (getConf() != null && getConf().getBoolean(
      YarnConfiguration.NM_LOG_CONTAINER_DEBUG_INFO,
      YarnConfiguration.DEFAULT_NM_LOG_CONTAINER_DEBUG_INFO)) {
    sb.copyDebugInformation(new Path(ContainerLaunch.CONTAINER_SCRIPT),
        new Path(logDir, ContainerLaunch.CONTAINER_SCRIPT));
    sb.listDebugInformation(new Path(logDir, DIRECTORY_CONTENTS));
  }

  sb.command(command);

  PrintStream pout = null;
  PrintStream ps = null;
  ByteArrayOutputStream baos = new ByteArrayOutputStream();
  try {
    pout = new PrintStream(out, false, "UTF-8");
    if (LOG.isDebugEnabled()) {
      ps = new PrintStream(baos, false, "UTF-8");
      sb.write(ps);
    }
    sb.write(pout);

  } finally {
    if (out != null) {
      out.close();
    }
    if (ps != null) {
      ps.close();
    }
  }
  if (LOG.isDebugEnabled()) {
    LOG.debug("Script: " + baos.toString("UTF-8"));
  }
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:70,代码来源:DockerContainerExecutor.java


示例18: writeLaunchEnv

import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@VisibleForTesting
public void writeLaunchEnv(OutputStream out,
    Map<String, String> environment, Map<Path, List<String>> resources,
    List<String> command, Path logDir, String outFilename)
    throws IOException {
  ContainerLaunch.ShellScriptBuilder sb =
    ContainerLaunch.ShellScriptBuilder.create();
  Set<String> whitelist = new HashSet<String>();
  whitelist.add(YarnConfiguration.NM_DOCKER_CONTAINER_EXECUTOR_IMAGE_NAME);
  whitelist.add(ApplicationConstants.Environment.HADOOP_YARN_HOME.name());
  whitelist.add(ApplicationConstants.Environment.HADOOP_COMMON_HOME.name());
  whitelist.add(ApplicationConstants.Environment.HADOOP_HDFS_HOME.name());
  whitelist.add(ApplicationConstants.Environment.HADOOP_CONF_DIR.name());
  whitelist.add(ApplicationConstants.Environment.JAVA_HOME.name());
  if (environment != null) {
    for (Map.Entry<String,String> env : environment.entrySet()) {
      if (!whitelist.contains(env.getKey())) {
        sb.env(env.getKey().toString(), env.getValue().toString());
      } else {
        sb.whitelistedEnv(env.getKey().toString(), env.getValue().toString());
      }
    }
  }
  if (resources != null) {
    for (Map.Entry<Path,List<String>> entry : resources.entrySet()) {
      for (String linkName : entry.getValue()) {
        sb.symlink(entry.getKey(), new Path(linkName));
      }
    }
  }

  // dump debugging information if configured
  if (getConf() != null && getConf().getBoolean(
      YarnConfiguration.NM_LOG_CONTAINER_DEBUG_INFO,
      YarnConfiguration.DEFAULT_NM_LOG_CONTAINER_DEBUG_INFO)) {
    sb.copyDebugInformation(new Path(outFilename), new Path(logDir, outFilename));
    sb.listDebugInformation(new Path(logDir, DIRECTORY_CONTENTS));
  }

  sb.command(command);

  PrintStream pout = null;
  try {
    pout = new PrintStream(out, false, "UTF-8");
    sb.write(pout);
  } finally {
    if (out != null) {
      out.close();
    }
  }
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:52,代码来源:ContainerExecutor.java


示例19: testContainerLogFile

import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Test(timeout=30000)
public void testContainerLogFile() throws IOException, YarnException {
  File absLogDir = new File("target",
      TestNMWebServer.class.getSimpleName() + "LogDir").getAbsoluteFile();
  String logdirwithFile = absLogDir.toURI().toString();
  Configuration conf = new Configuration();
  conf.set(YarnConfiguration.NM_LOG_DIRS, logdirwithFile);
  conf.setFloat(YarnConfiguration.NM_MAX_PER_DISK_UTILIZATION_PERCENTAGE,
      0.0f);
  LocalDirsHandlerService dirsHandler = new LocalDirsHandlerService();
  dirsHandler.init(conf);
  NMContext nmContext = new NodeManager.NMContext(null, null, dirsHandler,
      new ApplicationACLsManager(conf), new NMNullStateStoreService());
  // Add an application and the corresponding containers
  String user = "nobody";
  long clusterTimeStamp = 1234;
  ApplicationId appId = BuilderUtils.newApplicationId(
      clusterTimeStamp, 1);
  Application app = mock(Application.class);
  when(app.getUser()).thenReturn(user);
  when(app.getAppId()).thenReturn(appId);
  ApplicationAttemptId appAttemptId = BuilderUtils.newApplicationAttemptId(
      appId, 1);
  ContainerId containerId = BuilderUtils.newContainerId(
      appAttemptId, 1);
  nmContext.getApplications().put(appId, app);

  MockContainer container =
      new MockContainer(appAttemptId, new AsyncDispatcher(), conf, user,
          appId, 1);
  container.setState(ContainerState.RUNNING);
  nmContext.getContainers().put(containerId, container);
  File containerLogDir = new File(absLogDir,
      ContainerLaunch.getRelativeContainerLogDir(appId.toString(),
          containerId.toString()));
  containerLogDir.mkdirs();
  String fileName = "fileName";
  File containerLogFile = new File(containerLogDir, fileName);
  containerLogFile.createNewFile();
  File file = ContainerLogsUtils.getContainerLogFile(containerId,
      fileName, user, nmContext);
  Assert.assertEquals(containerLogFile.toURI().toString(),
      file.toURI().toString());
  FileUtil.fullyDelete(absLogDir);
}
 
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:46,代码来源:TestContainerLogsPage.java



注:本文中的org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
Java ActivityContentVideoViewClient类代码示例发布时间:2022-05-22
下一篇:
Java ContainersInfo类代码示例发布时间:2022-05-22
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap