X-Git-Url: https://git.octo.it/?a=blobdiff_plain;f=doc%2Frrdcached.pod;h=f40f065c48fe2aa1e16f98aa06dbf356df5b742e;hb=3ad3cec5db1ba33a23e3e0fbb8c19198cab59c3d;hp=92c65ccfb472a511a3ad49c15ed6af960e731e19;hpb=2c3d7019042a3bee86d973fb97e10e4cc3ff528e;p=rrdtool.git diff --git a/doc/rrdcached.pod b/doc/rrdcached.pod index 92c65cc..f40f065 100644 --- a/doc/rrdcached.pod +++ b/doc/rrdcached.pod @@ -16,6 +16,13 @@ passed, writes the updates to the RRD file. A I command may be used to force writing of values to disk, so that graphing facilities and similar can work with up-to-date data. +The daemon was written with big setups in mind. Those setups usually run into +IOErelated problems sooner or later for reasons that are beyond the scope +of this document. Check the wiki at the RRDTool homepage for details. Also +check L below before using this daemon! A detailed +description of how the daemon operates can be found in the L +section below. + =head1 OPTIONS =over 4 @@ -23,33 +30,286 @@ work with up-to-date data. =item B<-l> I
Tells the daemon to bind to I
and accept incoming connections on that -socket. If I
begins with C, everthing following that prefix is +socket. If I
begins with C, everything following that prefix is interpreted as the path to a UNIX domain socket. Otherwise the address or node name are resolved using L. +If the B<-l> option is not specified the default address, +C, will be used. + =item B<-w> I -Data is written to disk every I seconds. +Data is written to disk every I seconds. If this option is not +specified the default interval of 300Eseconds will be used. =item B<-f> I Every I seconds the entire cache is searched for old values which are written to disk. This only concerns files to which updates have stopped, so -setting this to a high value, such as 3600 seconds, is acceptable in most -cases. +setting this to a high value, such as 3600Eseconds, is acceptable in most +cases. This timeout defaults to 3600Eseconds. + +=item B<-p> I + +Sets the name and location of the PID-file. If not specified, the default, +C/run/rrdcached.pid> will be used. + +=item B<-b> I + +The daemon will change into a specific directory at startup. All files passed +to the daemon, that are specified by a B path, will be interpreted +to be relative to this directory. If not given the default, C, will be +used. + + +------------------------+------------------------+ + ! Command line ! File updated ! + +------------------------+------------------------+ + ! foo.rrd ! /tmp/foo.rrd ! + ! foo/bar.rrd ! /tmp/foo/bar.rrd ! + ! /var/lib/rrd/foo.rrd ! /var/lib/rrd/foo.rrd ! + +------------------------+------------------------+ + Paths given on the command line and paths actually + updated by the daemon, assuming the base directory + "/tmp". =back -=head1 BUGS +=head1 HOW IT WORKS + +When receiving an update, B does not write to disk but looks for an +entry for that file in its internal tree. If not found, an entry is created +including the current time (called "First" in the diagram below). This time is +B the time specified on the command line but the time the operating system +considers to be "now". The value and time of the value (called "Time" in the +diagram below) are appended to the tree node. + +When appending a value to a tree node, it is checked whether it's time to write +the values to disk. Values are written to disk if +S= timeout>>, where C is the timeout specified +using the B<-w> option, see L. If the values are "old enough" they +will be enqueued in the "update queue", i.Ee. they will be appended to +the linked list shown below. Because the tree nodes and the elements of the +linked list are the same data structures in memory, any update to a file that +has already been enqueued will be written with the next write to the RRD file, +too. + +A separate "update thread" constantly dequeues the first element in the update +queue and writes all its values to the appropriate file. So as long as the +update queue is not empty files are written at the highest possible rate. + +Since the timeout of files is checked only when new values are added to the +file, "dead" files, i.Ee. files that are not updated anymore, would never +be written to disk. Therefore, every now and then, controlled by the B<-f> +option, the entire tree is walked and all "old" values are enqueued. Since this +only affects "dead" files and walking the tree is relatively expensive, you +should set the "flush interval" to a reasonably high value. The default is +3600Eseconds (one hour). + +The downside of caching values is that they won't show up in graphs generated +from the RRDEfiles. To get around this, the daemon provides the "flush +command" to flush specific files. This means that the file is inserted at the +B of the update queue or moved there if it is already enqueued. The flush +command will return after the update thread has dequeued the file, so there is +a good chance that the file has been updated by the time the client receives +the response from the daemon, but there is no guarantee. + + +------+ +------+ +------+ + ! head ! ! root ! ! tail ! + +---+--+ +---+--+ +---+--+ + ! /\ ! + ! / \ ! + ! /\ /\ ! + ! /\/\ \ `----------------- ... --------, ! + V / `-------, ! V + +---+----+---+ +------+-----+ +---+----+---+ + ! File: foo ! ! File: bar ! ! File: qux ! + ! First: 101 ! ! First: 119 ! ! First: 180 ! + ! Next: ---+--->! Next: ---+---> ... --->! Next: - ! + +============+ +============+ +============+ + ! Time: 100 ! ! Time: 120 ! ! Time: 180 ! + ! Value: 10 ! ! Value: 0.1 ! ! Value: 2,2 ! + +------------+ +------------+ +------------+ + ! Time: 110 ! ! Time: 130 ! ! Time: 190 ! + ! Value: 26 ! ! Value: 0.1 ! ! Value: 7,3 ! + +------------+ +------------+ +------------+ + : : : : : : + +------------+ +------------+ +------------+ + ! Time: 230 ! ! Time: 250 ! ! Time: 310 ! + ! Value: 42 ! ! Value: 0.2 ! ! Value: 1,2 ! + +------------+ +------------+ +------------+ + +The above diagram demonstrates: =over 4 =item -Base directory is currently hard coded. The daemon will chdir to C. +Files/values are stored in a (balanced) tree. + +=item + +Tree nodes and entries in the update queue are the same data structure. + +=item + +The local time ("First") and the time specified in updates ("Time") may differ. + +=item + +Timed out values are inserted at the "tail". + +=item + +Explicitely flushed values are inserted at the "head". + +=item + +ASCII art rocks. + +=back + +=head1 SECURITY CONSIDERATIONS + +This daemon is meant to improve IOEperformance for setups with thousands +of RRDEfile to be updated. So security measures built into the daemon can +be summarized easily: B + +There is no authentication and authorization, so B will have to take care +that only authorized clients can talk to the daemon. Since we assume that graph +collection is done on a dedicated machine, i.Ee. the box doesn't do +anything else and especially does not have any interactive logins other than +root, a UNIX domain socket should take care of that. + +If you (want to) use the network capability, i.Ee. let the daemon bind to +an IPv4 or IPv6 socket, it is B job to install a packet filter or similar +mechanism to prevent unauthorized connections. Unless you have a dedicated VLAN +or VPN for this, using the network option is probably a bad idea! + +The daemon will blindly write to any file it gets told, so you really should +create a separate user just for this daemon. Also it does not do any sanity +checks, so if it gets told to write values for a time far in the future, your +files will be messed up good! + +You have been warned. + +=head1 PROTOCOL + +The daemon communicates with clients using a line based ASCII protocol which is +easy to read and easy to type. This makes it easy for scripts to implement the +protocol and possible for users to use L to connect to the daemon +and test stuff "by hand". + +The protocol is line based, this means that each record consists of one or more +lines. A line is terminated by the line feed character C<0x0A>, commonly +written as C<\n>. In the examples below, this character will be written as +CLFE> ("line feed"). + +After the connection has been established, the client is expected to send a +"command". A command consists of the command keyword, possibly some arguments, +and a terminating newline character. For a list of commands, see +L below. + +Example: + + FLUSH /tmp/foo.rrd + +The daemon answers with a line consisting of a status code and a short status +message, seperated by one or more space characters. A negative status code +signals an error, a positive status code or zero signal success. If the status +code is greater than zero, it indicates the number of lines that follow the +status line. + +Examples: + + 0 Success + + 2 Two lines follow + This is the first line + And this is the second line + +=head2 Valid Commands + +The following commands are understood by the daemon: + +=over 4 + +=item B I + +Causes the daemon to put I to the B of the update queue +(possibly moving it there if the node is already enqueued). The answer will be +sent B the node has been dequeued. + +=item B [I] + +Returns a short usage message. If no command is given, or I is +B, a list of commands supported by the daemon is returned. Otherwise a +short description, possibly containing a pointer to a manpage, is returned. +Obviously, this is meant for interactive usage and the format in which the +commands and usage summaries are returned is not well defined. + +=item B + +Returns a list of metrics which can be used to measure the daemons performance +and check its status. For a description of the values returned, see +L below. + +The format in which the values are returned is similar to many other line based +protocols: Each value is printed on a separate line, each consisting of the +name of the value, a colon, one or more spaces and the actual value. + +Example: + + 5 Statistics follow + QueueLength: 0 + UpdatesWritten: 13 + DataSetsWritten: 390 + TreeNodesNumber: 13 + TreeDepth: 4 + +=item B I I [I ...] + +Adds more data to a filename. This is B operation the daemon was designed +for, so describing the mechanism again is inappropriate. Read L +above for a detailed description. + +=back + +=head2 Performance Values + +The following counters are returned by the B command: + +=over 4 + +=item B I<(unsigned 64bit integer)> + +Number of nodes currently enqueued in the update queue. + +=item B I<(unsigned 64bit integer)> + +Depth of the tree used for fast key lookup. + +=item B I<(unsigned 64bit integer)> + +Number of nodes in the cache. + +=item B I<(unsigned 64bit integer)> + +Total number of updates, i.Ee. calls to C, since the daemon +was started. + +=item B I<(unsigned 64bit integer)> + +Total number of "data sets" written to disk since the daemon was started. A +data set is one or more values passed to the B command. For example: +C is one data set with two values. The term "data set" is used to +prevent confusion whether individual values or groups of values are counted. =back +=head1 BUGS + +No known bugs at the moment. + =head1 SEE ALSO L, L