Hadoop

Resolve errors/warnings in Cloudera Manager

This is a typical scenario based question and the solution is solely depend upon the errors/warnings appears in the Cloudera Manager.

Some examples:

The warnings could be space issue, service health status, low resources allocations, etc.,

The errors could be log directories are full, services down and other critical events.

 

 

In these scenarios, click on the error message and it will take you the respective service/instance status page.

If it’s a space issue, login to the server and go to the respective log directory, clean up some space by zipping/moving or deleting files.

For memory related issues, calculate the total memory available in the node, memory allocated for the services and try to balance the allocation without impacting the service/server.

Problem Scenarios:

You may encounter lots of errors/issues when you’re building the cluster on your own and resolving them will give you an idea of the common errors you face in the Cloudera Manager and it’s applicable here.

Use the comments section below to post your doubts, questions and feedback.

Please follow my blog to get notified of more certification related posts, exam tips, etc.

 


 

4 thoughts on “Resolve errors/warnings in Cloudera Manager

  1. there is a warning regarding the “map task maximum heap size to map task memory ratio is above 0.85”

    I tried to change the values between both parameters to be less than 0.85, but still the same warning. How to fix it?

    1. To meet requirement of “Heap to container size ratio” of 0.8, you need set “Map Task Maximum Heap Size” equals to 0.8 times of “Map Task Memory”.
      If you set “Map Task Maximum Heap Size” anything greater than that it will throw warning.

      Consider below paramaters,

      Map Task Memory
      mapreduce.map.memory.mb = 1536 (1.5 GB)

      Heap to Container Size Ratio
      mapreduce.job.heap.memory-mb.ratio = 0.8

      Map Task Maximum Heap Size
      mapreduce.map.java.opts = 1228 (1.5 x 0.8)

      ———————————————

      Heap to Container Size Ratio
      mapreduce.job.heap.memory-mb.ratio = (Map Task Maximum Heap Size)/ (Map Task Memory)

      = (1228 /1536)

      = 0.7995 (Apprx 0.8)

      I came across very good blog on yarn container configuration, it does not mention about heap to container ratio but theory is same.
      http://alvincjin.blogspot.com/2014/10/yarn-container-configuration.html

Leave a Reply

Your email address will not be published. Required fields are marked *