Here are some of the errors you face while setting up kerberos enabled hadoop
1) Be sure to validate your ticket and keytab file.
Ticket Validation:
klist
Output:
Ticket cache: FILE:/tmp/krb5cc_1001
Default principal: zookeeper/localhost@EXAMPLE.COM
Valid starting Expires Service principal
2017-05-22T18:40:52 2017-05-23T04:40:52 krbtgt/EXAMPLE.COM@EXAMPLE.COM
renew until 2017-05-29T18:40:52
Keytab validation:
kinit <PRINCIPAL> -k -t <KEYTAB_PATH>
It will return success if your keytab is valid.
2) Caused by: javax.security.auth.login.LoginException: No key to store
at com.sun.security.auth.module.Krb5LoginModule.commit(Krb5LoginModule.java:1072)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
at javax.security.auth.login.LoginContext.login(LoginContext.java:596)
at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:169)
... 24 more
2014-06-07 21:11:33,511 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1
2014-06-07 21:11:33,512 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG:
Cause: Ticket got expired.
Solution: rm -f /tmp/krb*
3) .keystore doesnot exist.
Cause: ssl is not correctly setup.Please follow the previous post
Solution: http://lxpert.blogspot.in/2017/05/setting-up-kerberos-enabled-hadoop.html
1) Be sure to validate your ticket and keytab file.
Ticket Validation:
klist
Output:
Ticket cache: FILE:/tmp/krb5cc_1001
Default principal: zookeeper/localhost@EXAMPLE.COM
Valid starting Expires Service principal
2017-05-22T18:40:52 2017-05-23T04:40:52 krbtgt/EXAMPLE.COM@EXAMPLE.COM
renew until 2017-05-29T18:40:52
Keytab validation:
kinit <PRINCIPAL> -k -t <KEYTAB_PATH>
It will return success if your keytab is valid.
2) Caused by: javax.security.auth.login.LoginException: No key to store
at com.sun.security.auth.module.Krb5LoginModule.commit(Krb5LoginModule.java:1072)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
at javax.security.auth.login.LoginContext.login(LoginContext.java:596)
at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:169)
... 24 more
2014-06-07 21:11:33,511 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1
2014-06-07 21:11:33,512 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG:
Cause: Ticket got expired.
Solution: rm -f /tmp/krb*
3) .keystore doesnot exist.
Cause: ssl is not correctly setup.Please follow the previous post
Solution: http://lxpert.blogspot.in/2017/05/setting-up-kerberos-enabled-hadoop.html
Monday
// //
0
comments
//
For YARN,add following properties in yarn-site.xml:
<!-- resource manager secure configuration info -->
<property>
<name>yarn.resourcemanager.principal</name>
<value><PRINCIPAL></value>
</property>
<property>
<name>yarn.resourcemanager.keytab</name>
<value><KEYTAB_PATH></value>
</property>
<!-- remember the principal for the node manager is the principal for the host this yarn-site.xml file is on -->
<!-- these (next four) need only be set on node manager nodes -->
<property>
<name>yarn.nodemanager.principal</name>
<value><PRINCIPAL></value>
</property>
<property>
<name>yarn.nodemanager.keytab</name>
<value><KEYTAB_PATH></value>
</property>
<!--<property>
<name>yarn.nodemanager.container-executor.class</name>
<value>org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor</value>
</property>
<property>
<name>yarn.nodemanager.linux-container-executor.group</name>
<value>yarn</value>
</property> -->
<!-- OPTIONAL - set these to enable secure proxy server node -->
<property>
<name>yarn.web-proxy.keytab</name>
<value><KEYTAB_PATH></value>
</property>
<property>
<name>yarn.web-proxy.principal</name>
<value><PRINCIPAL></value>
</property>
<!--<property>
<name>yarn.nodemanager.pmem-check-enabled</name>
<value>false</value>
</property>
<property>
<name>yarn.nodemanager.vmem-check-enabled</name>
<value>false</value>
</property> -->
<!-- resource manager secure configuration info -->
<property>
<name>yarn.resourcemanager.principal</name>
<value><PRINCIPAL></value>
</property>
<property>
<name>yarn.resourcemanager.keytab</name>
<value><KEYTAB_PATH></value>
</property>
<!-- remember the principal for the node manager is the principal for the host this yarn-site.xml file is on -->
<!-- these (next four) need only be set on node manager nodes -->
<property>
<name>yarn.nodemanager.principal</name>
<value><PRINCIPAL></value>
</property>
<property>
<name>yarn.nodemanager.keytab</name>
<value><KEYTAB_PATH></value>
</property>
<!--<property>
<name>yarn.nodemanager.container-executor.class</name>
<value>org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor</value>
</property>
<property>
<name>yarn.nodemanager.linux-container-executor.group</name>
<value>yarn</value>
</property> -->
<!-- OPTIONAL - set these to enable secure proxy server node -->
<property>
<name>yarn.web-proxy.keytab</name>
<value><KEYTAB_PATH></value>
</property>
<property>
<name>yarn.web-proxy.principal</name>
<value><PRINCIPAL></value>
</property>
<!--<property>
<name>yarn.nodemanager.pmem-check-enabled</name>
<value>false</value>
</property>
<property>
<name>yarn.nodemanager.vmem-check-enabled</name>
<value>false</value>
</property> -->
1) Add following properties in hdfs-site.xml
<!-- NameNode security config -->
<property>
<name>dfs.namenode.keytab.file</name>
<value><KEYTAB_PATH></value> <!-- path to the HDFS keytab -->
</property>
<property>
<name>dfs.namenode.kerberos.principal</name>
<value><PRINCIPAL></value>
</property>
<property>
<name>dfs.datanode.keytab.file</name>
<value><KEYTAB_PATH></value> <!-- path to the HDFS keytab -->
</property>
<property>
<name>dfs.datanode.kerberos.principal</name>
<value><PRINCIPAL></value>
</property>
<!---Secondary NameNode config-->
<property>
<name>dfs.secondary.namenode.keytab.file</name>
<value><KEYTAB_PATH></value>
</property>
<property>
<name>dfs.secondary.namenode.kerberos.principal</name>
<value><PRINCIPAL></value>
</property>
<!---DataNode config-->
<property>
<name>dfs.block.access.token.enable</name>
<value>true</value>
</property>
<property>
<name>dfs.datanode.address</name>
<value>0.0.0.0:1025</value>
</property>
<property>
<name>dfs.datanode.http.address</name>
<value>0.0.0.0:1027</value>
</property>
<property>
<name>dfs.data.transfer.protection</name>
<value>authentication</value>
</property>
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
<property>
<name>dfs.http.policy</name>
<value>HTTPS_ONLY</value>
</property>
<property>
<name>dfs.web.authentication.kerberos.principal</name>
<value><PRINCIPAL></value>
</property>
<property>
<name>dfs.web.authentication.kerberos.keytab</name>
<value><KEYTAB_PATH></value> <!-- path to the HTTP keytab -->
</property>
<property>
<name>dfs.namenode.kerberos.internal.spnego.principal</name>
<value>${dfs.web.authentication.kerberos.principal}</value>
</property>
<property>
<name>dfs.secondary.namenode.kerberos.internal.spnego.principal</name>
<value>>${dfs.web.authentication.kerberos.principal}</value>
</property>
2) Add following properties in core-site.xml:
<property>
<name>hadoop.security.authentication</name>
<value>kerberos</value> <!-- A value of "simple" would disable security. -->
</property>
<property>
<name>hadoop.security.authorization</name>
<value>true</value>
</property>
Now, we have create ssl configurations as in kerberos enabled hadoop it doesnt work with jvc.
Run following commands for the above:
keytool -genkey -keyalg RSA -alias c6401 -keystore /tmp/keystore.jks -storepass bigdata -validity 360 -keysize 2048
3) create ssl-client.xml in etc/hadoop:
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<configuration>
<property>
<name>ssl.client.truststore.location</name>
<value>/tmp/keystore.jks</value>
<description>Truststore to be used by clients like distcp. Must be
specified.
</description>
</property>
<property>
<name>ssl.client.truststore.password</name>
<value>bigdata</value>
<description>Optional. Default value is "".
</description>
</property>
<property>
<name>ssl.client.truststore.type</name>
<value>jks</value>
<description>Optional. The keystore file format, default value is "jks".
</description>
</property>
<property>
<name>ssl.client.truststore.reload.interval</name>
<value>10000</value>
<description>Truststore reload check interval, in milliseconds.
Default value is 10000 (10 seconds).
</description>
</property>
<property>
<name>ssl.client.keystore.location</name>
<value>/tmp/keystore.jks</value>
<description>Keystore to be used by clients like distcp. Must be
specified.
</description>
</property>
<property>
<name>ssl.client.keystore.password</name>
<value>bigdata</value>
<description>Optional. Default value is "".
</description>
</property>
<property>
<name>ssl.client.keystore.keypassword</name>
<value>bigdata</value>
<description>Optional. Default value is "".
</description>
</property>
<property>
<name>ssl.client.keystore.type</name>
<value>jks</value>
<description>Optional. The keystore file format, default value is "jks".
</description>
</property>
</configuration>
4) create ssl-server.xml at the same path
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<configuration>
<property>
<name>ssl.server.truststore.location</name>
<value>/tmp/keystore.jks</value>
<description>Truststore to be used by NN and DN. Must be specified.
</description>
</property>
<property>
<name>ssl.server.truststore.password</name>
<value>bigdata</value>
<description>Optional. Default value is "".
</description>
</property>
<property>
<name>ssl.server.truststore.type</name>
<value>jks</value>
<description>Optional. The keystore file format, default value is "jks".
</description>
</property>
<property>
<name>ssl.server.truststore.reload.interval</name>
<value>10000</value>
<description>Truststore reload check interval, in milliseconds.
Default value is 10000 (10 seconds).
</description>
</property>
<property>
<name>ssl.server.keystore.location</name>
<value>/tmp/keystore.jks</value>
<description>Keystore to be used by NN and DN. Must be specified.
</description>
</property>
<property>
<name>ssl.server.keystore.password</name>
<value>bigdata</value>
<description>Must be specified.
</description>
</property>
<property>
<name>ssl.server.keystore.keypassword</name>
<value>bigdata</value>
<description>Must be specified.
</description>
</property>
<property>
<name>ssl.server.keystore.type</name>
<value>jks</value>
<description>Optional. The keystore file format, default value is "jks".
</description>
</property>
</configuration>
<!-- NameNode security config -->
<property>
<name>dfs.namenode.keytab.file</name>
<value><KEYTAB_PATH></value> <!-- path to the HDFS keytab -->
</property>
<property>
<name>dfs.namenode.kerberos.principal</name>
<value><PRINCIPAL></value>
</property>
<property>
<name>dfs.datanode.keytab.file</name>
<value><KEYTAB_PATH></value> <!-- path to the HDFS keytab -->
</property>
<property>
<name>dfs.datanode.kerberos.principal</name>
<value><PRINCIPAL></value>
</property>
<!---Secondary NameNode config-->
<property>
<name>dfs.secondary.namenode.keytab.file</name>
<value><KEYTAB_PATH></value>
</property>
<property>
<name>dfs.secondary.namenode.kerberos.principal</name>
<value><PRINCIPAL></value>
</property>
<!---DataNode config-->
<property>
<name>dfs.block.access.token.enable</name>
<value>true</value>
</property>
<property>
<name>dfs.datanode.address</name>
<value>0.0.0.0:1025</value>
</property>
<property>
<name>dfs.datanode.http.address</name>
<value>0.0.0.0:1027</value>
</property>
<property>
<name>dfs.data.transfer.protection</name>
<value>authentication</value>
</property>
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
<property>
<name>dfs.http.policy</name>
<value>HTTPS_ONLY</value>
</property>
<property>
<name>dfs.web.authentication.kerberos.principal</name>
<value><PRINCIPAL></value>
</property>
<property>
<name>dfs.web.authentication.kerberos.keytab</name>
<value><KEYTAB_PATH></value> <!-- path to the HTTP keytab -->
</property>
<property>
<name>dfs.namenode.kerberos.internal.spnego.principal</name>
<value>${dfs.web.authentication.kerberos.principal}</value>
</property>
<property>
<name>dfs.secondary.namenode.kerberos.internal.spnego.principal</name>
<value>>${dfs.web.authentication.kerberos.principal}</value>
</property>
2) Add following properties in core-site.xml:
<property>
<name>hadoop.security.authentication</name>
<value>kerberos</value> <!-- A value of "simple" would disable security. -->
</property>
<property>
<name>hadoop.security.authorization</name>
<value>true</value>
</property>
Now, we have create ssl configurations as in kerberos enabled hadoop it doesnt work with jvc.
Run following commands for the above:
keytool -genkey -keyalg RSA -alias c6401 -keystore /tmp/keystore.jks -storepass bigdata -validity 360 -keysize 2048
3) create ssl-client.xml in etc/hadoop:
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<configuration>
<property>
<name>ssl.client.truststore.location</name>
<value>/tmp/keystore.jks</value>
<description>Truststore to be used by clients like distcp. Must be
specified.
</description>
</property>
<property>
<name>ssl.client.truststore.password</name>
<value>bigdata</value>
<description>Optional. Default value is "".
</description>
</property>
<property>
<name>ssl.client.truststore.type</name>
<value>jks</value>
<description>Optional. The keystore file format, default value is "jks".
</description>
</property>
<property>
<name>ssl.client.truststore.reload.interval</name>
<value>10000</value>
<description>Truststore reload check interval, in milliseconds.
Default value is 10000 (10 seconds).
</description>
</property>
<property>
<name>ssl.client.keystore.location</name>
<value>/tmp/keystore.jks</value>
<description>Keystore to be used by clients like distcp. Must be
specified.
</description>
</property>
<property>
<name>ssl.client.keystore.password</name>
<value>bigdata</value>
<description>Optional. Default value is "".
</description>
</property>
<property>
<name>ssl.client.keystore.keypassword</name>
<value>bigdata</value>
<description>Optional. Default value is "".
</description>
</property>
<property>
<name>ssl.client.keystore.type</name>
<value>jks</value>
<description>Optional. The keystore file format, default value is "jks".
</description>
</property>
</configuration>
4) create ssl-server.xml at the same path
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<configuration>
<property>
<name>ssl.server.truststore.location</name>
<value>/tmp/keystore.jks</value>
<description>Truststore to be used by NN and DN. Must be specified.
</description>
</property>
<property>
<name>ssl.server.truststore.password</name>
<value>bigdata</value>
<description>Optional. Default value is "".
</description>
</property>
<property>
<name>ssl.server.truststore.type</name>
<value>jks</value>
<description>Optional. The keystore file format, default value is "jks".
</description>
</property>
<property>
<name>ssl.server.truststore.reload.interval</name>
<value>10000</value>
<description>Truststore reload check interval, in milliseconds.
Default value is 10000 (10 seconds).
</description>
</property>
<property>
<name>ssl.server.keystore.location</name>
<value>/tmp/keystore.jks</value>
<description>Keystore to be used by NN and DN. Must be specified.
</description>
</property>
<property>
<name>ssl.server.keystore.password</name>
<value>bigdata</value>
<description>Must be specified.
</description>
</property>
<property>
<name>ssl.server.keystore.keypassword</name>
<value>bigdata</value>
<description>Must be specified.
</description>
</property>
<property>
<name>ssl.server.keystore.type</name>
<value>jks</value>
<description>Optional. The keystore file format, default value is "jks".
</description>
</property>
</configuration>
Powered by Blogger.