What solution do you use for configuring monitoring?

We currently have ansible, solarwinds and librenms in our environment but ran into an issue recently where a config setting was lost. I know ansible can check these things but then I'll have to configure email and stuff I guess to send alerts? What solution are you using?

12 points · 9 comments · view on lemmy.world

9 Comments

targetx@programming.dev · 4 pts · 3y

We've been using CheckMK for years, even before it was conveniently packaged as OMD, and I would definitely recommend it. We use SaltStack and have a hook on VM deploys/destroys where it adds/removes from monitoring automatically, works great for us.

zeusbottom@sh.itjust.works · 3 pts · 3y (3 replies)

Rancid for capturing configs, with some custom Python for grabbing changes as they happen and announcing via Slack.

solidgrue@lemmy.world · 1 pts · 3y (2 replies)

Why not track changes configs (derp) in a git repo and post git diff after each change?

zeusbottom@sh.itjust.works · 2 pts · 3y (1 reply)

That’s how it works, except we use the Rancid default of CVS instead of git. Rancid handles pulling the configs and puts them into a CVS repo, then the diffs are sent to the channel.

solidgrue@lemmy.world · 1 pts · 3y

Ah! Last I looked at Rancid it was backups with optional $REPO. TIL

slazer2au@lemmy.world · 2 pts · 3y (3 replies)

If you have LibreNMS, why not run the Oxidizer plugin to pull the units config?

https://docs.librenms.org/Extensions/Oxidized/

cyberpunk007@lemmy.world · 3 pts · 3y (2 replies)

Because I didn't know about it... Thank you!

Is this a plugin or a hook into yet another server that runs oxidized?

slazer2au@lemmy.world · 2 pts · 3y (1 reply)

It's up to you where to run it. We ran it on the same server as our main LibreNMS poller and didn't have any issues getting the config.

cyberpunk007@lemmy.world · 2 pts · 3y

Thanks I just got it running. Now to figure out what to do about credentials 😞. Our enable creds are different depending on region.