Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
The Something Awful Forums > Discussion > Serious Hardware/Software Crap > What SW can collect data and dynamically graph it on a web page?
 
  • Locked thread
jre
Sep 2, 2011

To the cloud ?

bolind posted:

I have a scenario where we're trying to collect a bunch of data and graph it. Data comes in at a not alarming rate; somewhere between once a day and once a minute. Things should ideally run on Linux and have a html based interface for displaying the data. Currently we're running an RRDtool based solution, which is kinda OK, but not quite.

What I'd like to do:

  • Have something collect data and stuff it into a fitting database. RRDtool does this very well. Problem is that data sources can come and go. Maybe we track metric A, B and C one week, and then A, B and D. And then maybe back to A, B and C again two weeks later. This must be supported.
  • Have something else that can display this data in nice graphs, ideally with the ability to zoom dynamically etc. Something like this: http://www.highcharts.com/demo/line-time-series
  • Be able to configure custom "views", basically being able to bundle metrics in graphs at will. Ideally with some sort of hierarchical structure (folder containing some views/graphs.)
  • Data must be able to participate in more than one graph.

Does anything like this exist? I can' t be the first nerd on this here planet who has had that need.

Edit: researching a bit, it seems like RRDtool should definitely be the backend for this.


Use grafana for visualisation , then pick a backend from graphite / opentsdb / influx

If you are used to docker, you can quickly get containers for grafana , graphite and tsdb to test it out

* # ¿ Oct 9, 2016 17:46
  • Quote
Adbot
ADBOT LOVES YOU

# ¿ Apr 29, 2024 00:01
  • Quote
jre
Sep 2, 2011

To the cloud ?

ColonelDimak posted:

ELK stacks all day erry day.
For reals though, Elastic Search, Kibana and Logstash are what you want.
It'll take any input if you can parse it with JSON and there are a bunch of prebuilt stacks for specific use cases.

The two I use the most are,
SexiLog for VMware, windows events and Veeam SNMP logging
Graylog for netflow monitoring.

Highly recommended everyone check out elk stacks and if you have any questions let me know.

ELK is great for event logging and pretty much useless for metric ( time series ) data despite that one elastic.co blog claiming otherwise. The OPs use case is for time series as they are already using rrd.

jre fucked around with this message at 22:56 on Jan 4, 2017

* # ¿ Jan 4, 2017 22:51
  • Quote
  • Locked thread
The Something Awful Forums > Discussion > Serious Hardware/Software Crap > What SW can collect data and dynamically graph it on a web page?