Add a new node to an existing cluster

This task’s steps are as same as the steps involved in installing CDH but the scenario is different, as we’re adding hosts to an existing cluster. Once the cluster is setup and running, you may have the requirement for adding new nodes to the cluster. To do that, ensure initial OS configurations/prechecks are complete. Go […]

Continue Reading

Install CDH using Cloudera Manager

After the installation of Cloudera Manager (SCM server), we can install CDH on our hosts using Cloudera Manager. Step 1: Login to CM url. When you login to CM for the first time after the installation of Cloudera SCM server, the login will redirect you to the following steps. Select the desired edition. Choose the […]

Continue Reading

Install Cloudera Manager server and agents

There are three ways to install the Cloudera Manager(server, agents), CDH and services. Automated installation by Cloudera Manager Install using Cloudera Manager parcels/packages Manual installation using Cloudera Manager Tarballs   Automated installation by Cloudera Manager: This is the most preferred way to install Cloudera Manager in non-production/test environments. This is not recommended for production deployments. […]

Continue Reading

Create encrypted zones in HDFS

Encryption at rest is the process of encrypting the data stored in the HDFS. This is a very advanced topic and to create an encryption zone you need to do below steps. Enable Kerberos Enable TLS/SSL Add Java Keystore KMS service (This will act as KTS as well) In production environment, you need to create […]

Continue Reading

Install and configure Sentry

Before adding Sentry, below are the general prerequisites need to be done. This may be mentioned in the problem description. Please confirm the hive warehouse directory detail in /etc/hive/conf/hive-site.xml file. The Hive warehouse directory (/user/hive/warehouse) must be owned by the Hive user and group and should have 771 permissions. # sudo –u hdfs hadoop fs […]

Continue Reading

Add a service using Cloudera Manager

Your running cluster will be having only core services (HDFS, YARN, Zookeeper) or handful of services and your task is to add a specific service to the cluster. To add a service: Go to CM – click the drop down box near the cluster – select Add service. You will get a list of services […]

Continue Reading

Create/restore a snapshot of an HDFS directory

HDFS Snapshots are read-only point-in-time copies of the file system. Snapshots can be taken on a directory of the file system or the entire file system. To enable a snapshot on a specific directory, Go to CM – HDFS – File Browser Select the directory in the file browser, select ‘Enable Snapshots’ in the right […]

Continue Reading

Install new type of I/O compression library in cluster

File/data compression brings two major benefits: it reduces the space needed to store files and it speeds up data transfer across the network or to or from disk. When dealing with large volumes of data, both of these savings can be significant. Hadoop supports the following compression types and codecs: gzip – org.apache.hadoop.io.compress.GzipCodec bzip2 – […]

Continue Reading