This is a typical scenario based question and the solution is solely depend upon the errors/warnings appears in the cluster. Some examples: The warnings could be space issue, service health status, low resources allocations, etc., The errors could be log directories are full, services down and other critical events. In these scenarios, click on […]
Continue ReadingExecute file system commands via HTTPFS
HttpFS is a service that provides HTTP access to HDFS. i.e we can access the HDFS from other filesystems from browsers, and using programming languages. HttpFS has a REST HTTP API supporting all HDFS filesystem operations (both read and write). Using HttpFS, we can Read and write data in HDFS using HTTP utilities (such as […]
Continue ReadingSet up alerting for excessive disk fill
Alert Publisher, one of the Cloudera’s management services, used to send alert notifications by email or by SNMP. Service instances of type HDFS, MapReduce can generate alerts if so configured. Alerts can also be configured for the monitoring roles that are a part of the Cloudera Management Service. Go to CM – Cloudera Management Service […]
Continue ReadingConfigure a service using Cloudera Manager
Configuring a service is one of the hardest tasks in Apache Hadoop, but Cloudera Manager has made our lives easier. For any configurations, you only have to give the values for the property, then Cloudera Manager will take care of updating the dependent services configuration, updating all the conf files (hdfs-site, yarn-site, core-site,etc) and deploy […]
Continue ReadingAdd a new node to an existing cluster
This task’s steps are as same as the steps involved in installing CDH but the scenario is different, as we’re adding hosts to an existing cluster. Once the cluster is setup and running, you may have the requirement for adding new nodes to the cluster. To do that, ensure initial OS configurations/prechecks are complete. Go […]
Continue ReadingInstall CDH using Cloudera Manager
After the installation of Cloudera Manager (SCM server), we can install CDH on our hosts using Cloudera Manager. Step 1: Login to CM url. When you login to CM for the first time after the installation of Cloudera SCM server, the login will redirect you to the following steps. Select the desired edition. Choose the […]
Continue ReadingInstall Cloudera Manager server and agents
There are three ways to install the Cloudera Manager(server, agents), CDH and services. Automated installation by Cloudera Manager Install using Cloudera Manager parcels/packages Manual installation using Cloudera Manager Tarballs Automated installation by Cloudera Manager: This is the most preferred way to install Cloudera Manager in non-production/test environments. This is not recommended for production deployments. […]
Continue ReadingCreate encrypted zones in HDFS
Encryption at rest is the process of encrypting the data stored in the HDFS. This is a very advanced topic and to create an encryption zone you need to do below steps. Enable Kerberos Enable TLS/SSL Add Java Keystore KMS service (This will act as KTS as well) In production environment, you need to create […]
Continue ReadingInstall and configure Sentry
Before adding Sentry, below are the general prerequisites need to be done. This may be mentioned in the problem description. Please confirm the hive warehouse directory detail in /etc/hive/conf/hive-site.xml file. The Hive warehouse directory (/user/hive/warehouse) must be owned by the Hive user and group and should have 771 permissions. # sudo –u hdfs hadoop fs […]
Continue ReadingAdd a service using Cloudera Manager
Your running cluster will be having only core services (HDFS, YARN, Zookeeper) or handful of services and your task is to add a specific service to the cluster. To add a service: Go to CM – click the drop down box near the cluster – select Add service. You will get a list of services […]
Continue Reading