Hadoop

Create an HDFS user’s home directory

Let’s assume that you’re asked to create an HDFS user home directory for a user named “goldberg“.

First of all, verify that the user is exists in the server.

If you’re creating the user directory for goldberg and set the ownership as goldberg:goldberg with restricted permissions, then none can access the dir except user ‘goldberg’. If the user is not present in the server, then this serves no purpose.

So, first create the user account in the server if the user doesn’t exists.

To verify user exist in the server:

# cat /etc/passwd | grep goldberg 
--> This will show if the user is present in the server. If present, it
will return output in below form.
goldberg:x:1000:1000:/home/goldberg:/bin/bash

# id goldberg  
--> If present, this will return the user id, group information of user.

To create a local user account in the server:

#useradd goldberg

#passwd goldberg   --> To set a password for this user account

 

The below command will create a directory in the HDFS with given name.

# hdfs dfs -mkdir /user/goldberg

 

If the directory has to be created by the hdfs user,

# sudo -u hdfs hdfs dfs -mkdir /user/goldberg

 

To change the ownership of the directory to goldberg.

#sudo -u hdfs hdfs dfs -chown goldberg:goldberg /user/goldberg

3 thoughts on “Create an HDFS user’s home directory

  1. Hi Kannan,
    Thanks for this much needed blog. I am using this as reference for cca131 prep. I have a question regarding. Generally there are some Configuration warnings in the cluster like “Java Heap space is not as recommended” etc. Do the reviewer look at these as issues and does this affect the grading of the exam.

    Thanks,
    Venkat

    1. Hi Venkat,

      Glad to know that you’re using my posts as reference for your preparation.

      To answer your question, we usually see those kind of warnings in the cluster we build/or in VMs due to low configuration. (less memory)
      In the exam, your cluster configs would be 32/64 GB memory for nodes, so you won’t see those errors in the exam cluster.

      But in the exam, if you’re tweaking the heap size/memory for any service as part of the given problem, and if shows warning as “heap space is not as recommended”, then you have to fix it. If it still shows warning at the end of the exam, that problem will be marked incorrect.

Leave a Reply

Your email address will not be published. Required fields are marked *