Thursday, March 31, 2022

[SOLVED] Stop HDFS Balancer after using nohup

Issue

I've launched my HDFS balancer with the nohup command :

$ nohup hdfs balancer &

It is taking forever and I need to work on my cluster. Do you know how can I stop the process ?

It's a distributed process so it's quite difficult to stop by juste doing "kill PID" ...

Thank you


Solution

Actually, the HDFS balancer is not a distributed process. It is a single process that initiates block movements on the cluster; the balancing process is carried out in a distributed manner cooperatively by DataNodes, but the commands about which blocks to balance originate from a single process (the one created by hdfs balancer). Doing a kill $PID will be sufficient to stop any further balancing.

Source: HDFS Balancer documentation, personal experience.



Answered By - xkrogen
Answer Checked By - Clifford M. (WPSolving Volunteer)