=head1 NAME
rrdtool - round robin database tool
=for html
=head1 SYNOPSIS
B B<-> | I
=head1 DESCRIPTION
=head2 OVERVIEW
It is pretty easy to gather status information from all sorts of
things, ranging from the temperature in your office to the number of
octets which have passed through the FDDI interface of your
router. But it is not so trivial to store this data in a efficient and
systematic manner. This is where B kicks in. It lets you
I the data you gather from all kinds of data-sources
(B). The data analysis part of rrdtool is based on the ability to
quickly generate graphical representations of the data values
collected over a definable time period.
In this man page you will find general information on the design and
functionality of the Round Robin Database Tool (rrdtool). For a more
detailed description of how to use the individual functions of the
B check the corresponding man page.
For an introduction to the usage of rrdtool make sure you check L.
=head2 FUNCTIONS
While the man pages talk of command line switches you have to set in
order to make B work it is important to note that the
B can be 'remote controlled' through a set of pipes. This
saves a considerable amount of startup time when you plan to make
B do a lot of things quickly. Check the section on L<"Remote
Control"> further down. There is also a number of language bindings
for rrdtool which allow you to use it directly from perl, python, tcl,
php, ...
=over 8
=item B
Set up a new Round Robin Database (RRD). Check L.
=item B
Store new data values into an RRD. Check L.
=item B
Create a graph from data stored in one or several RRD. Apart from
generating graphs, data can also be extracted to stdout. Check L.
=item B
Dump the contents of an RRD in plain ASCII. In connection with
restore you can use it to transport an rrd from one architecture to another.
Check L.
=item B
Restore an RRD in XML format to a binary rrd ... Check L
=item B
Get data for a certain time period from a RRD. The graph function
uses fetch to retrieve its data from an rrd. Check L.
=item B
Alter setup of an RRD. Check L.
=item B
Find last update time of an RRD. Check L.
=item B
Change the size of individual RRAs ... Dangerous! Check L.
=item B
This is a standalone tool for producing rrd graphs on the fly. Check
L.
=back
=head2 HOW DOES RRDTOOL WORK?
=over 8
=item Data acquisition
When monitoring the state of a system, it is convenient to have the
data available at a constant interval. Unfortunately you may not
always be able to fetch data at exactly the time you want
to. Therefore B lets you update the logfile at any time you
want. It will automatically interpolate the value of the data-source
(B) at the latest official time-slot and write this value to the
log. The value you have supplied is stored as well and is also taken
into account when interpolating the next log entry.
=item Consolidation
You may log data at a 1 minute interval, but you are also be
interested to know the development of the data over the last year. You
could do this by simply storing the data in 1 minute interval, for one
year. While this would take considerable disk space it would also take
a lot of time to analyze the data when you wanted to create a graph
covering the whole year. B offers a solution to this of this
problem through its data consolidation feature. When setting up
an Round Robin Database (B), you can define at which interval
this consolidation should occur, and what consolidation function
(B) (average, minimum, maximum, total, last) should be used to
build the consolidated values (see rrdcreate). You can define any
number of different consolidation setups within one B. They will
all be maintained on the fly when new data is loaded into the B.
=item Round Robin Archives
Data values of the same consolidation setup are stored into Round
Robin Archives (B). This is a very efficient manner to store data
for a certain amount of time, while using a known amount of storage
space.
It works like this: If you want to store 1000 values in 5 minute
interval, B will allocate space for 1000 data values and a
header area. In the header it will store a pointer telling
which one of the values in the storage area was last written to. New
values are written to the Round Robin Archive in a ... you guess it
... round robin manner. This automatically limits the history to the last
1000 values. Because you can define several Bs within a single B,
you can setup another one, storing 750 data values at a 2 hour interval
and thus keeping a log for the last two months although at a lower
resolution.
The use of Bs guarantees that the B does not grow over
time and that old data is automatically eliminated. By using the
consolidation feature, you can still keep data for a very long time,
while gradually reducing the resolution of the data along the time
axis. Using different consolidation functions (B) allows you to
store exactly the type of information that actually interests
you. (Maximum one minute traffic on the LAN, minimum temperature of
the wine cellar, total minutes down time ...)
=item Unknown Data
As mentioned earlier, the B stores data at a constant
interval. Now it may happen that no new data is available when a
value has to be written to the B. Data acquisition may not be
possible for one reason or an other. The B handles these
situations by storing an I<*UNKNOWN*> value into the database. The
value 'I<*UNKNOWN*>' is supported through all the functions of the
database. When consolidating the amount of I<*UNKNOWN*> data is
accumulated and when a new consolidated value is ready to be written
to its Round Robin Archive (B) a validity check is performed to
make sure that the percentage of unknown data in the new value is
below a configurable level. If so, an I<*UNKNOWN*> value will be
written to the B.
=item Graphing
The B also allows one to generate reports in numerical and
graphical form based on the data stored in one or several
Bs. The graphing feature is fully configurable. Size, color and
contents of the graph can be defined freely. Check L
for more information on this.
=item Aberrant Behavior Detection
by Jake Brutlag Ejakeb@corp.webtv.netE
The B also provides the building blocks for near real-time
aberrant behavior detection. These components include:
=over 12
=item *
An algorithm for predicting the values time series one time step into the future.
=item *
A measure of deviation between the predicted values and the observed values.
=item *
A mechanism to decide if and when an observed value
or sequence of observed values is I from the predicted value(s).
=back
Each of these components is briefly described:
Holt-Winters Time Series Forecasting Algorithm is an online, or incremental,
algorithm that adaptively predicts future observations in a time series. It's
forecast is the sum of three components: a baseline (or intercept), a linear
trend over time (or slope), and a seasonal coefficient (a periodic effect,
such as a daily cycle). There is one seasonal coefficient for each time point
in the period (cycle). After a value is observed, each of these components is
updated via exponential smoothing. So the algorithm learns from past values
and uses them to predict the future. The rate of adaptation is governed by
3 parameters, alpha (intercept), beta (slope), and gamma (seasonal). The prediction
can also be viewed as a smoothed value for the time series.
The measure of deviation is a seasonal weighted absolute deviation. The term
I means deviation is measured separately for each time point in the
seasonal cycle. As with Holt-Winters Forecasting, deviation is predicted using
the measure computed from past values (but only at that point in the seasonal cycle).
After the value is observed, the algorithm learns from the observed value via
exponential smoothing. Confidence bands for the observed time series are generated
by scaling the sequence of predicted deviation values (we usually think of the sequence
as a continuous line rather than a set of discrete points).
Aberrant behavior (a potential failure) is reported whenever the number of
times the observed value violates the confidence bands meets or exceeds a
specified threshold within a specified temporal window (i.e. 5 violations
during the past 45 minutes with a value observed every 5 mintues).
This functionality is embedded in a set of related B. In particular, a FAILURES
B logs potential failures. Presumably a front-end application to B can
utilize this B to initiate real-time alerts if that is desired.
You can find a detailed description of how to set this up in L.
=back
=head2 REMOTE CONTROL
When you start B with the command line option 'B<->', it waits
for input via standard in. With this feature you can improve
performance by attaching B to another process (mrtg is one
example) through a set of pipes. Over the pipes B accepts the
same arguments as on the command line. When a command is completed,
rrdtool will print the string 'C', followed by timing information of
the form BI BI both values are running
totals of seconds since rrdtool was started. If an error occurs, a line
of the form 'C I' will be printed. B
will not abort if possible, but follow the ERROR line with an OK line.
=head1 SEE ALSO
rrdcreate, rrdupdate, rrdgraph, rrddump, rrdfetch, rrdtune, rrdlast
=head1 BUGS
Bugs ? Features !
=head1 AUTHOR
Tobias Oetiker