eclipse下连Hadoop报java.lang.OutOfMemoryError: Java heap space的解决方法

eclipse下连Hadoop报java.lang.OutOfMemoryError: Java heap space的解决办法

eclipse下java.lang.OutOfMemoryError: Java heap space的解决办法:

 

一、在eclipse下远程连接Hadoop出现如下问题:

 14/03/28 22:39:03 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/03/28 22:39:03 ERROR security.UserGroupInformation: PriviledgedActionException as:Administrator cause:java.io.IOException: Failed to set permissions of path: \tmp\hadoop-Administrator\mapred\staging\Administrator-546884332\.staging to 0700
Exception in thread "main" java.io.IOException: Failed to set permissions of path: \tmp\hadoop-Administrator\mapred\staging\Administrator-546884332\.staging to 0700
 at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:689)
 at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:662)
 at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
 at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
 at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
 at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116)
 at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:918)
 at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:1)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:396)
 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
 at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912)
 at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
 at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
 at test.WordCountApp.main(WordCountApp.java:106)

 

二、解决方法:

1,从Intalled JREs里修改;window->Preferences->Java->Installed JREs,选择当前的JRE,然后edit它;在新窗口里设置Default VM Arguments为 -Xms128M -Xmx512M即可;这个设置应该对所有的工程都有效;

 

2,如果仅仅是想对某个工程的有效的话,从Debug... 或Run...里修改,在(x)=Arguments里设置和上面相同的参数即可;

 

三、解释:

JVM的堆的内存, 是通过下面两个参数控制的 :

-Xms 最小堆的大小, 也就是当你的虚拟机启动后, 就会分配这么大的堆内存给你 
-Xmx 是最大堆的大小 

当最小堆占满后,会尝试进行GC,如果GC之后还不能得到足够的内存(GC未必会收集到所有当前可用内存),分配新的对象,那么就会扩展堆,如果-Xmx设置的太小,扩展堆就会失败,导致OutOfMemoryError错误提示。