How-to use cacti to monitor MongoDB

I recently setup a cacti instance to monitor MongoDB. I’d like to share it.

To begin with, I setup a cacti instance by referencing cacti installation document.

At first, I use the template from here. It works. But there are some issues in my situation:

  1. It uses ssh to access the MongoDB server. I don’t want to config ssh to every new MongoDB servers.
  2. It doesn’t have the replicat set lag which is quiet important .
  3. It’s not efficient enough because it involves ssh
  4. Last but not least, it’s difficult for me to add new metrics because I am not familiar with php.

The solution to these issues is to let cacti to call a local script which is wrote by Perl. There is already an option “use-ssh” in script ss_get_by_ssh.php which is part of the template. So I did changes as follows to make it happen:

  • Add “–use-ssh FALSE” to related MongoDB “Data Input Methods” in cacti.
  • Later I found there is a bug. The “use-ssh” option is accepted as a String from input but it’s defined as Boolean in the script. So I changed it as follows to fix it.

    function get_command_result($cmd, $options) {
    ...
       #$use_ssh = isset($options['use-ssh']) ? $options['use-ssh'] : $use_ssh;
       $use_ssh = isset($options['use-ssh']) ? (strtoupper($options['use-ssh']) == 'FALSE'? FALSE : TRUE) : $use_ssh;
    ...
    
  • Next, change the function mongodb_cmdline in script ss_get_by_ssh.php.
  • function mongodb_cmdline ( $options ) {
    #   return "echo \"db._adminCommand({serverStatus:1, repl:2})\" | mongo";
       return "/export/home/cacti/scripts/getMongoDBStats.pl -h $options[host]";
    }
    

    In addition, I changed the part of lagSeconds in mongodb_parse as follows because “lagSeconds : 0 ” is a normal value in the Perl script.

    function mongodb_parse ( $options, $output ) {
    ...
    #   if (preg_match('/"lagSeconds" : ([0-9]+)/', $output, $matches) == 0) {
    #     $result["MONGODB_slave_lag"] = -1;
    #   } else {
    #     $result["MONGODB_slave_lag"] = $matches[1];
    #   } 
    
       preg_match('/lagSeconds : ([0-9]+)/', $output, $matches);
       $result["MONGODB_slave_lag"] = $matches[1];
    ...
    

    Finally, I added new metrics about active clients and queues as follows:

    1. Add below codes to the gather data script
    2. ...
        print "activeClients_total : $result->{globalLock}->{activeClients}->{total}\n";
        print "activeClients_readers : $result->{globalLock}->{activeClients}->{readers}\n";
        print "activeClients_writers : $result->{globalLock}->{activeClients}->{writers}\n";
        
        print "currentQueue_total : $result->{globalLock}->{currentQueue}->{total}\n";
        print "currentQueue_readers : $result->{globalLock}->{currentQueue}->{readers}\n";
        print "currentQueue_writers : $result->{globalLock}->{currentQueue}->{writers}\n";
      ...
      
    3. Modify ss_get_by_ssh.sh script to parse these new entries.
    4. ...
      #added these lines to array $keys in function ss_get_by_ssh
            'MONGODB_active_total'             => 'm1',
            'MONGODB_active_readers'           => 'm2',
            'MONGODB_active_writers'           => 'm3',
            'MONGODB_queue_total'              => 'm4',
            'MONGODB_queue_readers'            => 'm5',
            'MONGODB_queue_writers'            => 'm6',
      ...
      #added follow lines to function mongodb_parse
         $result["MONGODB_active_total"] = preg_match('/activeClients_total : ([0-9]+)/', $output, $matches)? $matches[1] : -1;
         $result["MONGODB_active_readers"] = preg_match('/activeClients_readers : ([0-9]+)/', $output, $matches)? $matches[1] : -1;
         $result["MONGODB_active_writers"] = preg_match('/activeClients_writers : ([0-9]+)/', $output, $matches)? $matches[1] : -1;
         $result["MONGODB_queue_total"] = preg_match('/currentQueue_total : ([0-9]+)/', $output, $matches)? $matches[1] : -1;
         $result["MONGODB_queue_readers"] = preg_match('/currentQueue_readers : ([0-9]+)/', $output, $matches)? $matches[1] : -1;
         $result["MONGODB_queue_writers"] = preg_match('/currentQueue_writers : ([0-9]+)/', $output, $matches)? $matches[1] : -1;
      ...
      
    5. Add these metrics to the MongoDB template in cacti admin pages as follows:
    6. Add Data Input Methods -> add Data Templates -> add Graph Templates -> add Host Templates -> add the new Graph Templates to existing MongoDB hosts in Devices, and then it’s done.

    There is a bug in check_cache function: it will hang there if another thread is lock the cache file because flock function will block by default. In cacti scenario, it will cause lots of processes hang there until it used up host resources. I added LOCK_NB to avoid that from happening.

    function check_cache ( $cache_dir, $poll_time, $filename, $options ) {
       $cache_file = "$cache_dir/${filename}_cacti_stats.txt";
       debug("Cache file: $cache_file");
       if ( $fp = fopen($cache_file, 'a+') ) {
          #$locked = flock($fp, 1); # LOCK_SH
          $locked = flock($fp, LOCK_SH|LOCK_NB); # Alex, added LOCK_NB(non block)
    

    With this method, you can write the gather metrics script in whatever programing language you like, and add whatever metrics you want.

    As always, this is just a start. Your comments are welcome.

    Advertisements

    About Alex Zeng
    I would be very happy if this blog can help you. I appreciate every honest comments. Please forgive me if I'm too busy to reply your comments in time.

    7 Responses to How-to use cacti to monitor MongoDB

    1. Pingback: How-to use cacti to monitor Cassandra « Oracle Explorer: Standing on the shoulders of giants

    2. Puspharaj Selvaraj says:

      Alex,

      How about if i have a authentication against the database mean mongodb got several users and they have their own password.

      In ss_get_by_ssh.php doesnt contain the mongo user or password in the script like the ssh for mysql

      How do i implement this ?

      Thanks

      • Alex Zeng says:

        If you want to implement this in Cacti, add “Input Fields” in “Data Input Methods” of the template, and then convey that username/password to the “gather data script”. Of course, you need to change the “gather data script” to use the username/password.

        The other way is to implement this out of Cacti. You can just change the “gather data script” to read the host’s username/password from somewhere (file, db, etc.). Cacti don’t need to aware of this in this approach. I think it’s easier.

        Thanks
        Alex

    Leave a Reply

    Fill in your details below or click an icon to log in:

    WordPress.com Logo

    You are commenting using your WordPress.com account. Log Out / Change )

    Twitter picture

    You are commenting using your Twitter account. Log Out / Change )

    Facebook photo

    You are commenting using your Facebook account. Log Out / Change )

    Google+ photo

    You are commenting using your Google+ account. Log Out / Change )

    Connecting to %s

    %d bloggers like this: