Hadoop

Enable/configure log and query redaction

Data redaction is the suppression of sensitive data, such as any personally identifiable information (PII) such as credit card number, email address, social security number. Cloudera has a data redaction feature, which will mask the credit card, email address with random or custom strings(we specify), so that in queries, log files those random strings will… Continue reading Enable/configure log and query redaction

Hadoop

Efficiently copy data within a cluster/between clusters

DistCp (distributed copy) is a tool used for large inter/intra-cluster copying of HDFS data. It uses MapReduce to effect its distribution, error handling and recovery, and reporting. It expands a list of files and directories into input to map tasks, each of which will copy a partition of the files specified in the source list.… Continue reading Efficiently copy data within a cluster/between clusters

Hadoop

Perform OS-level configuration for Hadoop installation

Before installing CDH in our server, we've to make the below configuration changes in OS level for successful installation. Disable SELINUX  "Security-Enhanced Linux (SELinux) is a Linux kernel security module that provides a mechanism for supporting access control security policies" If SElinux is enabled, then cloudera server installation will fail in the server. To disable… Continue reading Perform OS-level configuration for Hadoop installation

cca131 · Hadoop

CCA131 – Cloudera Administration Certification Exam Notes and Preparation Guide

In this post, we'll go through the CCA131 exam blueprint topics and show you what and how to perform the tasks in each topic. All the embedded links are my exam preparation notes. Go through each link and perform/practice it in your cluster. Please practice till the time you're confident of doing all the tasks without referring any documentation/material and capable of resolving the issues.