本文整理汇总了Java中org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch类的典型用法代码示例。如果您正苦于以下问题:Java ContainerLaunch类的具体用法?Java ContainerLaunch怎么用?Java ContainerLaunch使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。
ContainerLaunch类属于org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher包,在下文中一共展示了ContainerLaunch类的19个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。
示例1: getContainerLogFile
import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
/**
* Finds the log file with the given filename for the given container.
*/
public static File getContainerLogFile(ContainerId containerId,
String fileName, String remoteUser, Context context) throws YarnException {
Container container = context.getContainers().get(containerId);
Application application = getApplicationForContainer(containerId, context);
checkAccess(remoteUser, application, context);
if (container != null) {
checkState(container.getContainerState());
}
try {
LocalDirsHandlerService dirsHandler = context.getLocalDirsHandler();
String relativeContainerLogDir = ContainerLaunch.getRelativeContainerLogDir(
application.getAppId().toString(), containerId.toString());
Path logPath = dirsHandler.getLogPathToRead(
relativeContainerLogDir + Path.SEPARATOR + fileName);
URI logPathURI = new File(logPath.toString()).toURI();
File logFile = new File(logPathURI.getPath());
return logFile;
} catch (IOException e) {
LOG.warn("Failed to find log file", e);
throw new NotFoundException("Cannot find this log on the local disk.");
}
}
开发者ID:naver,项目名称:hadoop,代码行数:28,代码来源:ContainerLogsUtils.java
示例2: testKillOnLocalizedWhenContainerLaunched
import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Test
public void testKillOnLocalizedWhenContainerLaunched() throws Exception {
WrappedContainer wc = null;
try {
wc = new WrappedContainer(17, 314159265358979L, 4344, "yak");
wc.initContainer();
wc.localizeResources();
assertEquals(ContainerState.LOCALIZED, wc.c.getContainerState());
ContainerLaunch launcher = wc.launcher.running.get(wc.c.getContainerId());
launcher.call();
wc.drainDispatcherEvents();
assertEquals(ContainerState.EXITED_WITH_FAILURE,
wc.c.getContainerState());
wc.killContainer();
assertEquals(ContainerState.EXITED_WITH_FAILURE,
wc.c.getContainerState());
assertNull(wc.c.getLocalizedResources());
verifyCleanupCall(wc);
} finally {
if (wc != null) {
wc.finished();
}
}
}
开发者ID:naver,项目名称:hadoop,代码行数:25,代码来源:TestContainer.java
示例3: getContainerLogFile
import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
/**
* Finds the log file with the given filename for the given container.
*/
public static File getContainerLogFile(ContainerId containerId,
String fileName, String remoteUser, Context context, String userFolder) throws YarnException {
Container container = context.getContainers().get(containerId);
Application application = getApplicationForContainer(containerId, context);
checkAccess(remoteUser, application, context);
if (container != null) {
checkState(container.getContainerState());
}
try {
LocalDirsHandlerService dirsHandler = context.getLocalDirsHandler();
String relativeContainerLogDir = ContainerLaunch.getRelativeContainerLogDir(
application.getAppId().toString(), containerId.toString(), userFolder);
Path logPath = dirsHandler.getLogPathToRead(
relativeContainerLogDir + Path.SEPARATOR + fileName);
URI logPathURI = new File(logPath.toString()).toURI();
File logFile = new File(logPathURI.getPath());
return logFile;
} catch (IOException e) {
LOG.warn("Failed to find log file", e);
throw new NotFoundException("Cannot find this log on the local disk.");
}
}
开发者ID:hopshadoop,项目名称:hops,代码行数:28,代码来源:ContainerLogsUtils.java
示例4: testKillOnLocalizedWhenContainerLaunched
import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Test
public void testKillOnLocalizedWhenContainerLaunched() throws Exception {
WrappedContainer wc = null;
try {
wc = new WrappedContainer(17, 314159265358979L, 4344, "yak", "yakFolder");
wc.initContainer();
wc.localizeResources();
assertEquals(ContainerState.LOCALIZED, wc.c.getContainerState());
ContainerLaunch launcher = wc.launcher.running.get(wc.c.getContainerId());
launcher.call();
wc.drainDispatcherEvents();
assertEquals(ContainerState.EXITED_WITH_FAILURE,
wc.c.getContainerState());
wc.killContainer();
assertEquals(ContainerState.EXITED_WITH_FAILURE,
wc.c.getContainerState());
assertNull(wc.c.getLocalizedResources());
verifyCleanupCall(wc);
} finally {
if (wc != null) {
wc.finished();
}
}
}
开发者ID:hopshadoop,项目名称:hops,代码行数:25,代码来源:TestContainer.java
示例5: testKillOnLocalizedWhenContainerNotLaunched
import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Test
public void testKillOnLocalizedWhenContainerNotLaunched() throws Exception {
WrappedContainer wc = null;
try {
wc = new WrappedContainer(17, 314159265358979L, 4344, "yak");
wc.initContainer();
wc.localizeResources();
assertEquals(ContainerState.LOCALIZED, wc.c.getContainerState());
ContainerLaunch launcher = wc.launcher.running.get(wc.c.getContainerId());
wc.killContainer();
assertEquals(ContainerState.KILLING, wc.c.getContainerState());
launcher.call();
wc.drainDispatcherEvents();
assertEquals(ContainerState.CONTAINER_CLEANEDUP_AFTER_KILL,
wc.c.getContainerState());
assertNull(wc.c.getLocalizedResources());
verifyCleanupCall(wc);
} finally {
if (wc != null) {
wc.finished();
}
}
}
开发者ID:chendave,项目名称:hadoop-TCP,代码行数:24,代码来源:TestContainer.java
示例6: testKillOnLocalizedWhenContainerNotLaunched
import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Test
public void testKillOnLocalizedWhenContainerNotLaunched() throws Exception {
WrappedContainer wc = null;
try {
wc = new WrappedContainer(17, 314159265358979L, 4344, "yak");
wc.initContainer();
wc.localizeResources();
assertEquals(ContainerState.LOCALIZED, wc.c.getContainerState());
ContainerLaunch launcher = wc.launcher.running.get(wc.c.getContainerId());
wc.killContainer();
assertEquals(ContainerState.KILLING, wc.c.getContainerState());
launcher.call();
wc.drainDispatcherEvents();
assertEquals(ContainerState.CONTAINER_CLEANEDUP_AFTER_KILL,
wc.c.getContainerState());
assertNull(wc.c.getLocalizedResources());
verifyCleanupCall(wc);
wc.c.handle(new ContainerEvent(wc.c.getContainerId(),
ContainerEventType.CONTAINER_RESOURCES_CLEANEDUP));
assertEquals(0, metrics.getRunningContainers());
} finally {
if (wc != null) {
wc.finished();
}
}
}
开发者ID:Seagate,项目名称:hadoop-on-lustre2,代码行数:27,代码来源:TestContainer.java
示例7: writeLocalWrapperScript
import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Override
public void writeLocalWrapperScript(Path launchDst, Path pidFile,
PrintStream pout) {
String exitCodeFile = ContainerLaunch.getExitCodeFile(
pidFile.toString());
String tmpFile = exitCodeFile + ".tmp";
pout.println("#!/usr/bin/env bash");
pout.println("bash \"" + sessionScriptPath.toString() + "\"");
pout.println("rc=$?");
pout.println("echo $rc > \"" + tmpFile + "\"");
pout.println("mv -f \"" + tmpFile + "\" \"" + exitCodeFile + "\"");
pout.println("exit $rc");
}
开发者ID:naver,项目名称:hadoop,代码行数:15,代码来源:DockerContainerExecutor.java
示例8: writeLocalWrapperScript
import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Override
public void writeLocalWrapperScript(Path launchDst, Path pidFile,
PrintStream pout) {
String exitCodeFile = ContainerLaunch.getExitCodeFile(
pidFile.toString());
String tmpFile = exitCodeFile + ".tmp";
pout.println("#!/bin/bash");
pout.println("/bin/bash \"" + sessionScriptPath.toString() + "\"");
pout.println("rc=$?");
pout.println("echo $rc > \"" + tmpFile + "\"");
pout.println("/bin/mv -f \"" + tmpFile + "\" \"" + exitCodeFile + "\"");
pout.println("exit $rc");
}
开发者ID:naver,项目名称:hadoop,代码行数:14,代码来源:DefaultContainerExecutor.java
示例9: testKillOnLocalizedWhenContainerNotLaunched
import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Test
public void testKillOnLocalizedWhenContainerNotLaunched() throws Exception {
WrappedContainer wc = null;
try {
wc = new WrappedContainer(17, 314159265358979L, 4344, "yak");
wc.initContainer();
wc.localizeResources();
assertEquals(ContainerState.LOCALIZED, wc.c.getContainerState());
ContainerLaunch launcher = wc.launcher.running.get(wc.c.getContainerId());
wc.killContainer();
assertEquals(ContainerState.KILLING, wc.c.getContainerState());
launcher.call();
wc.drainDispatcherEvents();
assertEquals(ContainerState.CONTAINER_CLEANEDUP_AFTER_KILL,
wc.c.getContainerState());
assertNull(wc.c.getLocalizedResources());
verifyCleanupCall(wc);
int killed = metrics.getKilledContainers();
wc.c.handle(new ContainerEvent(wc.c.getContainerId(),
ContainerEventType.CONTAINER_RESOURCES_CLEANEDUP));
assertEquals(ContainerState.DONE, wc.c.getContainerState());
assertEquals(killed + 1, metrics.getKilledContainers());
assertEquals(0, metrics.getRunningContainers());
} finally {
if (wc != null) {
wc.finished();
}
}
}
开发者ID:naver,项目名称:hadoop,代码行数:30,代码来源:TestContainer.java
示例10: writeLocalWrapperScript
import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Override
public void writeLocalWrapperScript(Path launchDst, Path pidFile,
PrintStream pout) {
String exitCodeFile = ContainerLaunch.getExitCodeFile(
pidFile.toString());
String tmpFile = exitCodeFile + ".tmp";
pout.println("#!/usr/bin/env bash");
pout.println("bash \"" + sessionScriptPath.toString() + "\"");
pout.println("rc=$?");
pout.println("echo $rc > \"" + tmpFile + "\"");
pout.println("mv -f \"" + tmpFile + "\" \"" + exitCodeFile + "\"");
pout.println("exit $rc");
}
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:14,代码来源:DockerContainerExecutor.java
示例11: testKillOnLocalizedWhenContainerNotLaunchedContainerKilled
import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Test
public void testKillOnLocalizedWhenContainerNotLaunchedContainerKilled()
throws Exception {
WrappedContainer wc = null;
try {
wc = new WrappedContainer(17, 314159265358979L, 4344, "yak");
wc.initContainer();
wc.localizeResources();
assertEquals(ContainerState.LOCALIZED, wc.c.getContainerState());
ContainerLaunch launcher = wc.launcher.running.get(wc.c.getContainerId());
wc.killContainer();
assertEquals(ContainerState.KILLING, wc.c.getContainerState());
launcher.call();
wc.drainDispatcherEvents();
assertEquals(ContainerState.CONTAINER_CLEANEDUP_AFTER_KILL,
wc.c.getContainerState());
assertNull(wc.c.getLocalizedResources());
verifyCleanupCall(wc);
int killed = metrics.getKilledContainers();
wc.c.handle(new ContainerEvent(wc.c.getContainerId(),
ContainerEventType.CONTAINER_RESOURCES_CLEANEDUP));
assertEquals(ContainerState.DONE, wc.c.getContainerState());
assertEquals(killed + 1, metrics.getKilledContainers());
assertEquals(0, metrics.getRunningContainers());
} finally {
if (wc != null) {
wc.finished();
}
}
}
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:31,代码来源:TestContainer.java
示例12: writeLaunchEnv
import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
public void writeLaunchEnv(OutputStream out, Map<String, String> environment, Map<Path, List<String>> resources, List<String> command) throws IOException{
ContainerLaunch.ShellScriptBuilder sb = ContainerLaunch.ShellScriptBuilder.create();
if (environment != null) {
for (Map.Entry<String,String> env : environment.entrySet()) {
sb.env(env.getKey().toString(), env.getValue().toString());
LOG.info("key:"+env.getKey().toString()+"value:"+env.getValue().toString());
}
}
sb.cd("$PWD");
if (resources != null) {
for (Map.Entry<Path,List<String>> entry : resources.entrySet()) {
for (String linkName : entry.getValue()) {
sb.symlink(entry.getKey(), new Path(linkName));
LOG.info("key:"+entry.getKey().toString()+"value:"+entry.getValue().toString());
}
}
}
LOG.info("commands "+command);
sb.command(command);
PrintStream pout = null;
try {
pout = new PrintStream(out, false, "UTF-8");
sb.write(pout);
} finally {
if (out != null) {
out.close();
}
}
}
开发者ID:yncxcw,项目名称:big-c,代码行数:36,代码来源:ContainerExecutor.java
示例13: writeLaunchEnv
import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
public void writeLaunchEnv(OutputStream out, Map<String, String> environment, Map<Path, List<String>> resources, List<String> command) throws IOException{
ContainerLaunch.ShellScriptBuilder sb = ContainerLaunch.ShellScriptBuilder.create();
if (environment != null) {
for (Map.Entry<String,String> env : environment.entrySet()) {
sb.env(env.getKey().toString(), env.getValue().toString());
}
}
if (resources != null) {
for (Map.Entry<Path,List<String>> entry : resources.entrySet()) {
for (String linkName : entry.getValue()) {
sb.symlink(entry.getKey(), new Path(linkName));
}
}
}
sb.command(command);
PrintStream pout = null;
try {
pout = new PrintStream(out);
sb.write(pout);
} finally {
if (out != null) {
out.close();
}
}
}
开发者ID:Nextzero,项目名称:hadoop-2.6.0-cdh5.4.3,代码行数:28,代码来源:ContainerExecutor.java
示例14: testKillOnLocalizedWhenContainerNotLaunchedContainerKilled
import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Test
public void testKillOnLocalizedWhenContainerNotLaunchedContainerKilled()
throws Exception {
WrappedContainer wc = null;
try {
wc = new WrappedContainer(17, 314159265358979L, 4344, "yak", "yakFolder");
wc.initContainer();
wc.localizeResources();
assertEquals(ContainerState.LOCALIZED, wc.c.getContainerState());
ContainerLaunch launcher = wc.launcher.running.get(wc.c.getContainerId());
wc.killContainer();
assertEquals(ContainerState.KILLING, wc.c.getContainerState());
launcher.call();
wc.drainDispatcherEvents();
assertEquals(ContainerState.CONTAINER_CLEANEDUP_AFTER_KILL,
wc.c.getContainerState());
assertNull(wc.c.getLocalizedResources());
verifyCleanupCall(wc);
int killed = metrics.getKilledContainers();
wc.c.handle(new ContainerEvent(wc.c.getContainerId(),
ContainerEventType.CONTAINER_RESOURCES_CLEANEDUP));
assertEquals(ContainerState.DONE, wc.c.getContainerState());
assertEquals(killed + 1, metrics.getKilledContainers());
assertEquals(0, metrics.getRunningContainers());
} finally {
if (wc != null) {
wc.finished();
}
}
}
开发者ID:hopshadoop,项目名称:hops,代码行数:31,代码来源:TestContainer.java
示例15: reacquireContainer
import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
/**
* Recover an already existing container. This is a blocking call and returns
* only when the container exits. Note that the container must have been
* activated prior to this call.
* @param ctx encapsulates information necessary to reacquire container
* @return The exit code of the pre-existing container
* @throws IOException
* @throws InterruptedException
*/
public int reacquireContainer(ContainerReacquisitionContext ctx)
throws IOException, InterruptedException {
Container container = ctx.getContainer();
String user = ctx.getUser();
ContainerId containerId = ctx.getContainerId();
Path pidPath = getPidFilePath(containerId);
if (pidPath == null) {
LOG.warn(containerId + " is not active, returning terminated error");
return ExitCode.TERMINATED.getExitCode();
}
String pid = null;
pid = ProcessIdFileReader.getProcessId(pidPath);
if (pid == null) {
throw new IOException("Unable to determine pid for " + containerId);
}
LOG.info("Reacquiring " + containerId + " with pid " + pid);
ContainerLivenessContext livenessContext = new ContainerLivenessContext
.Builder()
.setContainer(container)
.setUser(user)
.setPid(pid)
.build();
while(isContainerAlive(livenessContext)) {
Thread.sleep(1000);
}
// wait for exit code file to appear
String exitCodeFile = ContainerLaunch.getExitCodeFile(pidPath.toString());
File file = new File(exitCodeFile);
final int sleepMsec = 100;
int msecLeft = 2000;
while (!file.exists() && msecLeft >= 0) {
if (!isContainerActive(containerId)) {
LOG.info(containerId + " was deactivated");
return ExitCode.TERMINATED.getExitCode();
}
Thread.sleep(sleepMsec);
msecLeft -= sleepMsec;
}
if (msecLeft < 0) {
throw new IOException("Timeout while waiting for exit code from "
+ containerId);
}
try {
return Integer.parseInt(FileUtils.readFileToString(file).trim());
} catch (NumberFormatException e) {
throw new IOException("Error parsing exit code from pid " + pid, e);
}
}
开发者ID:naver,项目名称:hadoop,代码行数:66,代码来源:ContainerExecutor.java
示例16: testContainerLogs
import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Test
public void testContainerLogs() throws IOException {
WebResource r = resource();
final ContainerId containerId = BuilderUtils.newContainerId(0, 0, 0, 0);
final String containerIdStr = BuilderUtils.newContainerId(0, 0, 0, 0)
.toString();
final ApplicationAttemptId appAttemptId = containerId.getApplicationAttemptId();
final ApplicationId appId = appAttemptId.getApplicationId();
final String appIdStr = appId.toString();
final String filename = "logfile1";
final String logMessage = "log message\n";
nmContext.getApplications().put(appId, new ApplicationImpl(null, "user",
appId, null, nmContext));
MockContainer container = new MockContainer(appAttemptId,
new AsyncDispatcher(), new Configuration(), "user", appId, 1);
container.setState(ContainerState.RUNNING);
nmContext.getContainers().put(containerId, container);
// write out log file
Path path = dirsHandler.getLogPathForWrite(
ContainerLaunch.getRelativeContainerLogDir(
appIdStr, containerIdStr) + "/" + filename, false);
File logFile = new File(path.toUri().getPath());
logFile.deleteOnExit();
assertTrue("Failed to create log dir", logFile.getParentFile().mkdirs());
PrintWriter pw = new PrintWriter(logFile);
pw.print(logMessage);
pw.close();
// ask for it
ClientResponse response = r.path("ws").path("v1").path("node")
.path("containerlogs").path(containerIdStr).path(filename)
.accept(MediaType.TEXT_PLAIN).get(ClientResponse.class);
String responseText = response.getEntity(String.class);
assertEquals(logMessage, responseText);
// ask for file that doesn't exist
response = r.path("ws").path("v1").path("node")
.path("containerlogs").path(containerIdStr).path("uhhh")
.accept(MediaType.TEXT_PLAIN).get(ClientResponse.class);
Assert.assertEquals(Status.NOT_FOUND.getStatusCode(), response.getStatus());
responseText = response.getEntity(String.class);
assertTrue(responseText.contains("Cannot find this log on the local disk."));
// After container is completed, it is removed from nmContext
nmContext.getContainers().remove(containerId);
Assert.assertNull(nmContext.getContainers().get(containerId));
response =
r.path("ws").path("v1").path("node").path("containerlogs")
.path(containerIdStr).path(filename).accept(MediaType.TEXT_PLAIN)
.get(ClientResponse.class);
responseText = response.getEntity(String.class);
assertEquals(logMessage, responseText);
}
开发者ID:naver,项目名称:hadoop,代码行数:57,代码来源:TestNMWebServices.java
示例17: writeLaunchEnv
import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Override
/**
* Filter the environment variables that may conflict with the ones set in
* the docker image and write them out to an OutputStream.
*/
public void writeLaunchEnv(OutputStream out, Map<String, String> environment,
Map<Path, List<String>> resources, List<String> command, Path logDir)
throws IOException {
ContainerLaunch.ShellScriptBuilder sb =
ContainerLaunch.ShellScriptBuilder.create();
//Remove environments that may conflict with the ones in Docker image.
Set<String> exclusionSet = new HashSet<String>();
exclusionSet.add(YarnConfiguration.NM_DOCKER_CONTAINER_EXECUTOR_IMAGE_NAME);
exclusionSet.add(ApplicationConstants.Environment.HADOOP_YARN_HOME.name());
exclusionSet.add(ApplicationConstants.Environment.HADOOP_COMMON_HOME.name());
exclusionSet.add(ApplicationConstants.Environment.HADOOP_HDFS_HOME.name());
exclusionSet.add(ApplicationConstants.Environment.HADOOP_CONF_DIR.name());
exclusionSet.add(ApplicationConstants.Environment.JAVA_HOME.name());
if (environment != null) {
for (Map.Entry<String,String> env : environment.entrySet()) {
if (!exclusionSet.contains(env.getKey())) {
sb.env(env.getKey().toString(), env.getValue().toString());
}
}
}
if (resources != null) {
for (Map.Entry<Path,List<String>> entry : resources.entrySet()) {
for (String linkName : entry.getValue()) {
sb.symlink(entry.getKey(), new Path(linkName));
}
}
}
// dump debugging information if configured
if (getConf() != null && getConf().getBoolean(
YarnConfiguration.NM_LOG_CONTAINER_DEBUG_INFO,
YarnConfiguration.DEFAULT_NM_LOG_CONTAINER_DEBUG_INFO)) {
sb.copyDebugInformation(new Path(ContainerLaunch.CONTAINER_SCRIPT),
new Path(logDir, ContainerLaunch.CONTAINER_SCRIPT));
sb.listDebugInformation(new Path(logDir, DIRECTORY_CONTENTS));
}
sb.command(command);
PrintStream pout = null;
PrintStream ps = null;
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try {
pout = new PrintStream(out, false, "UTF-8");
if (LOG.isDebugEnabled()) {
ps = new PrintStream(baos, false, "UTF-8");
sb.write(ps);
}
sb.write(pout);
} finally {
if (out != null) {
out.close();
}
if (ps != null) {
ps.close();
}
}
if (LOG.isDebugEnabled()) {
LOG.debug("Script: " + baos.toString("UTF-8"));
}
}
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:70,代码来源:DockerContainerExecutor.java
示例18: writeLaunchEnv
import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@VisibleForTesting
public void writeLaunchEnv(OutputStream out,
Map<String, String> environment, Map<Path, List<String>> resources,
List<String> command, Path logDir, String outFilename)
throws IOException {
ContainerLaunch.ShellScriptBuilder sb =
ContainerLaunch.ShellScriptBuilder.create();
Set<String> whitelist = new HashSet<String>();
whitelist.add(YarnConfiguration.NM_DOCKER_CONTAINER_EXECUTOR_IMAGE_NAME);
whitelist.add(ApplicationConstants.Environment.HADOOP_YARN_HOME.name());
whitelist.add(ApplicationConstants.Environment.HADOOP_COMMON_HOME.name());
whitelist.add(ApplicationConstants.Environment.HADOOP_HDFS_HOME.name());
whitelist.add(ApplicationConstants.Environment.HADOOP_CONF_DIR.name());
whitelist.add(ApplicationConstants.Environment.JAVA_HOME.name());
if (environment != null) {
for (Map.Entry<String,String> env : environment.entrySet()) {
if (!whitelist.contains(env.getKey())) {
sb.env(env.getKey().toString(), env.getValue().toString());
} else {
sb.whitelistedEnv(env.getKey().toString(), env.getValue().toString());
}
}
}
if (resources != null) {
for (Map.Entry<Path,List<String>> entry : resources.entrySet()) {
for (String linkName : entry.getValue()) {
sb.symlink(entry.getKey(), new Path(linkName));
}
}
}
// dump debugging information if configured
if (getConf() != null && getConf().getBoolean(
YarnConfiguration.NM_LOG_CONTAINER_DEBUG_INFO,
YarnConfiguration.DEFAULT_NM_LOG_CONTAINER_DEBUG_INFO)) {
sb.copyDebugInformation(new Path(outFilename), new Path(logDir, outFilename));
sb.listDebugInformation(new Path(logDir, DIRECTORY_CONTENTS));
}
sb.command(command);
PrintStream pout = null;
try {
pout = new PrintStream(out, false, "UTF-8");
sb.write(pout);
} finally {
if (out != null) {
out.close();
}
}
}
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:52,代码来源:ContainerExecutor.java
示例19: testContainerLogFile
import org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch; //导入依赖的package包/类
@Test(timeout=30000)
public void testContainerLogFile() throws IOException, YarnException {
File absLogDir = new File("target",
TestNMWebServer.class.getSimpleName() + "LogDir").getAbsoluteFile();
String logdirwithFile = absLogDir.toURI().toString();
Configuration conf = new Configuration();
conf.set(YarnConfiguration.NM_LOG_DIRS, logdirwithFile);
conf.setFloat(YarnConfiguration.NM_MAX_PER_DISK_UTILIZATION_PERCENTAGE,
0.0f);
LocalDirsHandlerService dirsHandler = new LocalDirsHandlerService();
dirsHandler.init(conf);
NMContext nmContext = new NodeManager.NMContext(null, null, dirsHandler,
new ApplicationACLsManager(conf), new NMNullStateStoreService());
// Add an application and the corresponding containers
String user = "nobody";
long clusterTimeStamp = 1234;
ApplicationId appId = BuilderUtils.newApplicationId(
clusterTimeStamp, 1);
Application app = mock(Application.class);
when(app.getUser()).thenReturn(user);
when(app.getAppId()).thenReturn(appId);
ApplicationAttemptId appAttemptId = BuilderUtils.newApplicationAttemptId(
appId, 1);
ContainerId containerId = BuilderUtils.newContainerId(
appAttemptId, 1);
nmContext.getApplications().put(appId, app);
MockContainer container =
new MockContainer(appAttemptId, new AsyncDispatcher(), conf, user,
appId, 1);
container.setState(ContainerState.RUNNING);
nmContext.getContainers().put(containerId, container);
File containerLogDir = new File(absLogDir,
ContainerLaunch.getRelativeContainerLogDir(appId.toString(),
containerId.toString()));
containerLogDir.mkdirs();
String fileName = "fileName";
File containerLogFile = new File(containerLogDir, fileName);
containerLogFile.createNewFile();
File file = ContainerLogsUtils.getContainerLogFile(containerId,
fileName, user, nmContext);
Assert.assertEquals(containerLogFile.toURI().toString(),
file.toURI().toString());
FileUtil.fullyDelete(absLogDir);
}
开发者ID:aliyun-beta,项目名称:aliyun-oss-hadoop-fs,代码行数:46,代码来源:TestContainerLogsPage.java
注:本文中的org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。 |
请发表评论