How to mount HDFS in Linux (FUSE)

  1. Mounting HDFS as FUSE

   FUSE (Filesystem in userspace) is a simple interface for userspace programs to export a virtual filesystem to the Linux kernel. Fuse also aims to provide a secure method for non privileged users to create and mount their own filesystem implementations.

1.1. Configuring Mountable HDFS

CDH has included a FUSE interface into HDFS. The following steps needs to be followed to mount HDFS as FUSE file system.

a) install hadoop-hdfs-fuses On the system.

   sudo yum install hadoop-hdfs-fuse

Note:  It will comes part of the parcel installation.

b) To set up and test your mount point

   Before doing anything to mount set the LD_LIBRARY_PATH

export LD_LIBRARY_PATH=$JAVA_HOME/jre/lib/amd64:$JAVA_HOME/jre/lib/amd64/server:/opt/cloudera/parcels/CDH/lib64

   mkdir -p <mount_point>

   hadoop-fuse-dfs dfs://<name_node_hostname>:<namenode_port> <mount_point>

   where namenode_port is the NameNode's RPC port, dfs.namenode.servicerpc-address.

    

Ex:Created a mountdir "/mnt/hdfs-fuse-mnt"

opt/cloudera/parcels/CDH/bin/hadoop-fuse-dfs dfs://kor1046740.apac.bosch.com:8022/ /mnt/hdfs-fuse-mnt

c) To unmount the HDFS

      umount <mount_point>

   Alternatively  : fusermount -u /mnt/hdfs-fuse-mnt

d) To add a permanent HDFS mount which persists through reboots.

   Open /etc/fstab and add lines to the bottom similar to this:

   

hadoop-fuse-dfs#dfs://<name_node_hostname>:<namenode_port> <mount_point> fuse allow_other,usetrash,rw 2 0

   Ex: 

/opt/cloudera/parcels/CDH/bin/hadoop-fuse-dfs#dfs://namenode_hostname.com:8020/ /mnt/hdfs-fuse-mnt fuse allow_other,usetrash,rw 2 0

e) To test to make sure everything is working properly:

    mount <mount_point>

• Tweet
• Share
• Share
• Share
• Share

Sign up here with your email