Hadoop报错ERROR [org.apache.hadoop.util.Shell] - Failed to locate the winutils binary in the hadoop binary path

发表于 | 分类于 | | 阅读次数:
字数统计: 247 | 阅读时长 ≈ 1

环境

Windows10 + IDEA2020.2专业版 + JDK8 + Hadoop2.6.0

报错信息

1
2
3
4
5
2020-12-03 14:37:52,196 WARN [org.apache.hadoop.util.NativeCodeLoader] - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2020-12-03 14:37:52,202 ERROR [org.apache.hadoop.util.Shell] - Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
	at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:355)
	at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:370)

image-20201203144329960

解决方法

  1. 下载winutils的windows版本，并放在%HADOOP_HOME%\bin下

  2. 设置环境变量，设置%HADOOP_HOME%为你的Hadoop根目录，并且Path中加入%HADOOP_HOME%\bin，之后重启idea

  3. 手动在代码中指定Hadoop根目录的位置（IDEA貌似必须这样，不知为何，而cmd命令行启动jar包只需要前两个就行了）

    1
    System.setProperty("hadoop.home.dir", "你的Hadoop根目录，注意右斜线的转义");

以上三个解决方案依次尝试即可。

参考：

  1. windows下运行spark程序报错：Failed to locate the winutils binary in the hadoop binary path