Search

CheckMk Notes

References

Requirment

System Requirements

This is pretty sparse and lousy, but I found this:

Resource Requirements Based on the experience:

  • One site, dozen hosts with Check_MK agent, and some hosts with SNMP: 1 CPU core, 1 GB RAM, min. 8-16 GB HDD (e.g. mini PC).
  • One site, more dozen hosts with Check_MK agent, and dozen hosts with SNMP: 2 CPU core, 2 GB RAM, min. 40 GB HDD.
  • One site, more than one hundred hosts with Check_MK agent, including more dozen host with log monitoring and vulnerability scan, dozen hosts with SNMP, and Event Console: 4 CPU core, 4 GB RAM, min. 80 GB HDD.
  • One or more site, more than one hundred hosts with Check_MK agent, including log monitoring and vulnerability scan, more dozen hosts with SNMP, and Event Console: 6-8 CPU core, 4-8 GB RAM, min. 120 GB HDD.
  • One or more site, more hundred hosts with Check_MK agent, including log monitoring and vulnerability scan, more hundred hosts with SNMP, Event Console: min. 8 CPU core, min. 8 GB RAM, min. 200 GB HDD.

The manufacturer's recommendations: https://mathias-kettner.com/check_mk_monitoring_appliance.html

Other Requirements

  • And Elastic IP Address, so the checkserver has a reliable IP
  • SSH key for server
    • Created as checkmk-admin currently

Questions on Configuration

We could set up the monitor in two ways:

  • One monitor for all bettrdata instances
    • Advantages
      • View across all hosts in a single place
      • One instance for all sites is just one app to take care for upgrades
    • Disadvantages
      • Would need to be its own server, so adding an extra instance of EC2 by itself
  • One monitor per instance
    • Advantages
      • Could live on the same host as the bettrdata instance
    • Disadvantages
      • Add more space needed per instance
      • If the EC2 instance dies, the monitor also dies

Installing on Virtual Machine For Testing

  1. Creating virtual machine
  2. Install checkmk host on system (from deb)
  3. Create an instance of checkmk on the server
    1. You can have multiple instances on the same server
    2. You can have multiple instances on the same server
    3. Example command to create and start
    4. You will need to start the site and go to it (locally)
    5. Test site at http://checkmkadmin-VirtualBox/test_site/ in virtual box or port forward if desired
    6. Download the agent to install locally for the first test
    7. Install it
    8. Add the host in the admin UI
      1. Setup > Hosts > Add host
        1. Details
          1. Host name: monitor_system (could be anything, but no spaces)
          2. IPv4 address: whatever the local host ip address is
      2. Click on “Save & run service discovery”. A list of services appear after a few moments
      3. Choose the services you want (which could be all)
      4. Click on the changes sign (octogonal yellow icon with exclamation point, upper right) That will bring up the activation button (circular red arrow icon, middle left) Click to activate
      5. Click on the host in the top menu to get to the host summary, and click on the host name, monitor_system to see details.
      6. Click on services of host to see individual services

Install on EC2 Instance for Testing

Create an EC2 Instance

  • Latest Ubuntu Free AMI
  • Install checkmk

More Notes

Get the Agent

The agent installer is found in the checkmk ui. You will need to download it manually. Hopefully the file can be included in the ansible create and upgrade commands to both copy to the server and install. For the moment, I copied it manually to the EC2 instance

Install command (AWS OS is a version of redhat/fedora)

Open Ports Needed for CheckMk Agent

The checkmk agent runs on two port for the pull version:

  • UDP
    • 161
  • TCP
    • 6556

Open those ports to MY ipaddress for testing in the EC2 instance

Setting Up in CheckMk Admin

  1. Go to Setup > Host > Add Host
    • Host name
      • robpoc3_chkmk_agent
    • IPv4 address:
      • My ip address
    • Monitoring agents
      • Checmk agent / API integrations
  2. Then add all the service and changes etcetra as above

Install MongoDB Shell

I will do this manually for now, but we should have the mongodb shell installed by default. Most of the things we want to check are in the local mongodb instance

Installation Steps Via Yum (For now)

  1. Create the mongodb yum configuration
    2. sudo vi /etc/yum.repos.d/mongodb-org-8.0.repo
  2. Add the configuration (for Amazon LInux 2, which what we have)
    3. 
[mongodb-org-7.0]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/amazon/2/mongodb-org/7.0/$basearch/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-7.0.asc
  3. Install mongo-sh

Test Queries

Mongoshell has integration with node, so we can create node scripts and run them to get results

  1. A simple script
  1. Run the script in mongoshell
ec2-user$  ./trapperqueuecollectioncheck.js 
2 "trapper_queue_check" - "trapperqueue collection NOT found

Set Up the Local Check in Robpoc3 Instance

Create the path where the local check executable files should be located

ec2-user$ sudo su
root$ mkdir -p /usr/lib/check_mk_agent/local
root$ cd /home/ec2-user
root$ cp trapperqueuecollectioncheck.js /usr/lib/check_mk_agent/local

# run a check on the script locally
root$ cmk-agent-ctl dump | grep -v grep | grep -A2 "<<<local"
<<<local:sep(0)>>>
2 "trapper_queue_check" - "trapperqueue collection NOT found"

Connect The Local Service to The CheckMk Server

Now that we created a local check and it tests correcly, we can add it to the checkmk server

  1. In the server, go to Setup > Hosts
  2. Select the host where we added the local check, in this case robpoc3_monitor
  3. Click on the “Save & run service discovery”
  4. Now in the “Undecided services” section there is the new service, “trapper_queue_check”, which is the service we created. Click the plus sign to add it
  5. Click on the changes icon to show the “Activate pending changes” page
  6. Click on “Activate on selected sites” to activate
image