Getting help or usage

Let us explore details about how to list the commands and get the help or usage for given command.

  • Even though we can run commands from almost all the nodes in the clusters, we should only use Gateway to run HDFS Commands.

  • First we need to make sure designated Gateway server is Gateway for HDFS service so that we can run commands from Gateway node. In our case we have designated gw02.itversity.com or gw03.itversity.com as Gateways.

  • Typically Namenode process will be running on port number 8020. We can also pass namenode URI to access HDFS.

%%sh

head -20 /etc/hadoop/conf/core-site.xml

%%sh

hdfs dfs -ls /user/${USER}

%%sh

hdfs dfs -ls hdfs://nn01.itversity.com:8020/user/${USER}

  • hadoop fs or hdfs dfs – list all the commands available

  • hadoop fs -usage – will give us basic usage for given command

  • hadoop fs -help – will give us additional information for all the commands. It is same as just running hadoop fs or hdfs dfs.

  • We can run help on individual commands as well - example: hadoop fs -help ls or hdfs dfs -help ls

%%sh

hdfs dfs -help

%%sh

hdfs dfs -usage ls

%%sh

hdfs dfs -help ls

%%sh

hdfs dfs -ls /public/retail_db
Using HDFS CLI Listing HDFS Files