I have a very simple script example from hadoop real world solution cookbook and I try it on amazon cloudera clustertogov04 ami and it gives me the java exception of not able to mkdir??
(我有一个来自hadoop现实世界解决方案食谱的非常简单的脚本示例,我在亚马逊cloudera clustertogov04 ami上尝试过,它给了我无法mkdir的Java异常?)
but I have enough disk space?? (但是我有足够的磁盘空间?)
[ec2-user]$ df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/xvde1 8255928 3307252 4529300 43% /
tmpfs 3757068 0 3757068 0% /dev/shm
/dev/xvdk 103212320 192116 97777324 1% /data
heres the script,command,error output
(这是脚本,命令,错误输出)
weblogs = load '/data2/weblogs/weblog_entries.txt' as
(md5:chararray,
url:chararray,
date:chararray,
time:chararray,
ip:chararray);
md5_grp = group weblogs by md5 parallel 4;
store md5_grp into '/data/weblogs/weblogs_md5_groups.bcp';
pig -x local -f pig02 2>err02
2013-06-20 19:57:29,499 [Thread-4] INFO org.apache.hadoop.mapred.Merger - Down to the last merge-pass, with 1 segments left of total size: 299132 bytes 2013-06-20 19:57:29,499 [Thread-4] INFO org.apache.hadoop.mapred.LocalJobRunner - 2013-06-20 19:57:29,519 [Thread-4] WARN org.apache.hadoop.mapred.LocalJobRunner - job_local_0001 java.io.IOException: Mkdirs failed to create file:/data/weblogs/weblogs_md5_groups.bcp/_temporary/_attempt_local_0001_r_000000_0 at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:434) at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:420) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:805) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:786) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:685) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigTextOutputFormat.getRecordWriter(PigTextOutputFormat.java:98) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.
(2013-06-20 19:57:29,499 [Thread-4] INFO org.apache.hadoop.mapred.Merger- 向下到最后一次合并遍历,总共剩余1个段:299132字节 2013-06-20 19 :57:29,499 [Thread-4]信息org.apache.hadoop.mapred.LocalJobRunner-2013-06-20 19:57:29,519 [Thread-4] WARN org.apache.hadoop.mapred.LocalJobRunner-job_local_0001 java.io .IOException:Mkdirs无法创建文件:/data/weblogs/weblogs_md5_groups.bcp/_temporary/_attempt_local_0001_r_000000_0 at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:434)在org.apache.hadoop.fs.ChecksumFile在org.apache.hadoop.fs.FileSystem.create(FileSystem.java:805)的org.apache.hadoop.fs.FileSystem.create(FileSystem.java:786)的.create(ChecksumFileSystem.java:420)。 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigTextOutputFormat.getRecordWriter(PigTextOutputFormat.java:98)位于org.apache.pig.backend处的apache.hadoop.fs.FileSystem.create(FileSystem.java:685)。 hadoop.executionengine.mapReduceLayer。)
PigOutputFormat.getRecordWriter(PigOutputFormat.java:84) at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:582) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:433) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:309) 2013-06-20 19:57:33,176 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - job job_local_0001 has failed! (org.org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:433)的PigOutputFormat.getRecordWriter(PigOutputFormat.java:84)在org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:582)在org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:433) .apache.hadoop.mapred.LocalJobRunner $ Job.run(LocalJobRunner.java:309)2013-06-20 19:57:33,176 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher-作业job_local_0001失败!)
Stop running all dependent jobs (停止运行所有依赖作业)
2013-06-20 19:57:33,180 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
(2013-06-20 19:57:33,180 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher-100%完成)
2013-06-20 19:57:33,182 [main] ERROR org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
(2013-06-20 19:57:33,182 [main]错误org.apache.pig.tools.pigstats.PigStatsUtil-1个地图缩小作业失败!)
2013-06-20 19:57:33,182 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats - Detected Local mode.
(2013-06-20 19:57:33,182 [main]信息org.apache.pig.tools.pigstats.SimplePigStats-检测到本地模式。)
Stats reported below may be incomplete (以下报告的统计信息可能不完整)
2013-06-20 19:57:33,185 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
(2013-06-20 19:57:33,185 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats-脚本统计信息:)
HadoopVersion PigVersion UserId StartedAt FinishedAt Features
(HadoopVersion PigVersion用户ID StartedAt FinishedAt功能)
2.0.0-cdh4.1.2 0.10.0-cdh4.1.2 ec2-user 2013-06-20 19:57:27 2013-06-20 19:57:33 GROUP_BY
(2.0.0-cdh4.1.2 0.10.0-cdh4.1.2 ec2-user 2013-06-20 19:57:27 2013-06-20 19:57:33 GROUP_BY)
Failed!
(失败了!)
Pig Stack Trace ---------------
(猪栈痕迹---------------)
ERROR 2244: Job failed, hadoop does not return any error message
(错误2244:作业失败,hadoop不返回任何错误消息)
org.apache.pig.backend.executionengine.ExecException: ERROR 2244: Job failed, hadoop does not return any error message at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:140) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:193)
(org.apache.pig.backend.executionengine.ExecException:错误2244:作业失败,Hadoop在org.apache的org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:140)上未返回任何错误消息.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:193))
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165) at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
(在org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)在org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84))
at org.apache.pig.Main.run(Main.java:430)
(在org.apache.pig.Main.run(Main.java:430))
at org.apache.pig.Main.main(Main.java:111) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597)
(在org.apache.pig.Main.main(Main.java:111)在sun.reflect.NativeMethodAccessorImpl.invoke0(在本机方法)在sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)在sun.reflect.DelegatingMethodAccessorImpl .invoke(DelegatingMethodAccessorImpl.java:25)at java.lang.reflect.Method.invoke(Method.java:597))
at org.apache.hadoop.util.RunJar.main(RunJar.java:208) (在org.apache.hadoop.util.RunJar.main(RunJar.java:208))
ask by Andrew C. Kee translate from so