From: oetiker <oetiker@a5681a0c-68f1-0310-ab6d-d61299d08faa>
Date: Sun, 4 Mar 2001 13:01:56 +0000 (+0000)
Subject: Aberrant Behavior Detection support. A brief overview added to rrdtool.pod.
X-Git-Url: https://git.octo.it/?p=rrdtool.git;a=commitdiff_plain;h=7c016dfa001ae254bf4e18126f814ee8f0abd821

Aberrant Behavior Detection support. A brief overview added to rrdtool.pod.
Major updates to rrd_update.c, rrd_create.c. Minor update to other core files.
This is backwards compatible! But new files using the Aberrant stuff are not readable
by old rrdtool versions. See http://cricket.sourceforge.net/aberrant/rrd_hw.htm
-- Jake Brutlag <jakeb@corp.webtv.net>


git-svn-id: svn://svn.oetiker.ch/rrdtool/trunk/program@26 a5681a0c-68f1-0310-ab6d-d61299d08faa
---

diff --git a/NEWS b/NEWS
new file mode 100644
index 0000000..b9c3a64
--- /dev/null
+++ b/NEWS
@@ -0,0 +1,18 @@
+RRDTOOL NEWS
+============
+
+In this file I am noting the Major changes to rrdtool
+for details check the cvs ChangeLog
+
+2001/03/21 Tobias Oetiker <oetiker@ee.ethz.ch>
+  Added Aberrant Patch from Jake Brutlag <jakeb@corp.webtv.net>
+  From now one, new rrd files use version tag 0002. They can
+  NOT be read by the old 1.0.x rrdtools
+
+  Jack:
+  Aberrant Behavior Detection support. A brief overview added to
+  rrdtool.pod.  Major updates to rrd_update.c, rrd_create.c. Minor update to
+  other core files.  Updated documentation: rrdcreate.pod, rrdgraph.pod,
+  rrdtune.pod.  This is backwards compatible!  See
+  http://cricket.sourceforge.net/aberrant/rrd_hw.htm
+
diff --git a/doc/rrdcreate.pod b/doc/rrdcreate.pod
index 8be0567..14b6f19 100644
--- a/doc/rrdcreate.pod
+++ b/doc/rrdcreate.pod
@@ -10,7 +10,7 @@ B<rrdtool> B<create> I<filename>
 S<[B<--start>|B<-b> I<start time>]> 
 S<[B<--step>|B<-s> I<step>]> 
 S<[B<DS:>I<ds-name>B<:>I<DST>B<:>I<heartbeat>B<:>I<min>B<:>I<max>]>
-S<[B<RRA:>I<CF>B<:>I<xff>B<:>I<steps>B<:>I<rows>]>
+S<[B<RRA:>I<CF>B<:>I<cf arguments>]>
 
 =head1 DESCRIPTION
 
@@ -106,30 +106,174 @@ I<If information on minimal/maximal expected values is available,
 always set the min and/or max properties. This will help RRDtool in
 doing a simple sanity check on the data supplied when running update.>
 
-=item B<RRA:>I<CF>B<:>I<xff>B<:>I<steps>B<:>I<rows>
-
+=item B<RRA:>I<CF>B<:>I<cf arguments>
+  
 The purpose of an B<RRD> is to store data in the round robin archives
-(B<RRA>). An archive consists of a number of data values from all the
-defined data-sources (B<DS>) and is defined with an B<RRA> line.
+(B<RRA>). An archive consists of a number of data values or statistics for 
+each of the defined data-sources (B<DS>) and is defined with an B<RRA> line.
 
 When data is entered into an B<RRD>, it is first fit into time slots of
 the length defined with the B<-s> option becoming a I<primary data point>.
 
-The data is also consolidated with the consolidation function (I<CF>)
-of the archive. The following consolidation functions are defined:
-B<AVERAGE>, B<MIN>, B<MAX>, B<LAST>.
+The data is also processed with the consolidation function (I<CF>)
+of the archive. There are several consolidation functions that consolidate
+primary data points via an aggregate function: B<AVERAGE>, B<MIN>, B<MAX>, B<LAST>.
+The format of B<RRA> line for these consolidation function is:
+
+B<RRA:>I<AVERAGE | MIN | MAX | LAST>B<:>I<xff>B<:>I<steps>B<:>I<rows>
 
 I<xff> The xfiles factor defines what part of a consolidation interval may
 be made up from I<*UNKNOWN*> data while the consolidated value is still
 regarded as known.
 
-I<steps> defines how many of these I<primary data points> are used to
-build a I<consolidated data point> which then goes into the archive.
+I<steps> defines how many of these I<primary data points> are used to build
+a I<consolidated data point> which then goes into the archive.
 
 I<rows> defines how many generations of data values are kept in an B<RRA>.
 
 =back
 
+=head1 Aberrant Behaviour detection with Holt-Winters forecasting
+
+by Jake Brutlag E<lt>jakeb@corp.webtv.netE<gt>
+
+In addition to the aggregate functions, there are a set of specialized
+functions that enable B<RRDtool> to provide data smoothing (via the
+Holt-Winters forecasting algorithm), confidence bands, and the flagging
+aberrant behavior in the data source time series:
+
+=over 4
+
+=item B<RRA:>I<HWPREDICT>B<:>I<rows>B<:>I<alpha>B<:>I<beta>B<:>I<seasonal period>B<:>I<rra num>
+
+=item B<RRA:>I<SEASONAL>B<:>I<seasonal period>B<:>I<gamma>B<:>I<rra num>
+
+=item B<RRA:>I<DEVSEASONAL>B<:>I<seasonal period>B<:>I<gamma>B<:>I<rra num>
+
+=item B<RRA:>I<DEVPREDICT>B<:>I<rows>B<:>I<rra num>
+
+=item B<RRA:>I<FAILURES>B<:>I<rows>B<:>I<threshold>B<:>I<window length>B<:>I<rra num>
+
+=back
+
+These B<RRAs> differ from the true consolidation functions in several ways.
+First, each of the B<RRA>s is updated once for every primary data point.
+Second, these B<RRAs> are interdependent. To generate real-time confidence
+bounds, then a matched set of HWPREDICT, SEASONAL, DEVSEASONAL, and
+DEVPREDICT must exist. Generating smoothed values of the primary data points
+requires both a HWPREDICT B<RRA> and SEASONAL B<RRA>. Aberrant behavior
+detection requires FAILURES, HWPREDICT, DEVSEASONAL, and SEASONAL.
+
+The actual predicted, or smoothed, values are stored in the HWPREDICT
+B<RRA>. The predicted deviations are store in DEVPREDICT (think a standard
+deviation which can be scaled to yield a confidence band). The FAILURES
+B<RRA> stores binary indicators. A 1 marks the indexed observation a
+failure; that is, the number of confidence bounds violations in the
+preceding window of observations met or exceeded a specified threshold. An
+example of using these B<RRAs> to graph confidence bounds and failures
+appears in L<rrdgraph>.
+
+The SEASONAL and DEVSEASONAL B<RRAs> store the seasonal coefficients for the
+Holt-Winters Forecasting algorithm and the seasonal deviations respectively.
+There is one entry per observation time point in the seasonal cycle. For
+example, if primary data points are generated every five minutes, and the
+seasonal cycle is 1 day, both SEASONAL and DEVSEASONAL with have 288 rows.
+
+In order to simplify the creation for the novice user, in addition to
+supporting explicit creation the HWPREDICT, SEASONAL, DEVPREDICT,
+DEVSEASONAL, and FAILURES B<RRAs>, the B<rrdtool> create command supports
+implicit creation of the other four when HWPREDICT is specified alone and
+the final argument I<rra num> is omitted.
+
+I<rows> specifies the length of the B<RRA> prior to wrap around. Remember
+that there is a one-to-one correspondence between primary data points and
+entries in these RRAs. For the HWPREDICT CF, I<rows> should be larger than
+the I<seasonal period>. If the DEVPREDICT B<RRA> is implicity created, the
+default number of rows is the same as the HWPREDICT I<rows> argument. If the
+FAILURES B<RRA> is implicitly created, I<rows> will be set to the I<seasonal
+period> argument of the HWPREDICT B<RRA>. Of course, the B<rrdtool>
+I<resize> command is available if these defaults are not sufficient and the
+create wishes to avoid explicit creations of the other specialized function
+B<RRAs>.
+
+I<seasonal period> specifies the number of primary data points in a seasonal
+cycle. If SEASONAL and DEVSEASONAL are implicitly created, this argument for
+those B<RRAs> is set automatically to the value specified by HWPREDICT. If
+they are explicity created, the creator should verify that all three
+I<seasonal period> arguments agree.
+
+I<alpha> is the adaptation parameter of the intercept (or baseline)
+coefficient in the Holt-Winters Forecasting algorithm. See L<rrdtool> for a
+description of this algorithm. I<alpha> must lie between 0 and 1. A value
+closer to 1 means that more recent observations carry greater weight in
+predicting the baseline component of the forecast. A value closer to 0 mean
+that past history carries greater weight in predicted the baseline
+component.
+
+I<beta> is the adaption parameter of the slope (or linear trend) coefficient
+in the Holt-Winters Forecating algorihtm. I<beta> must lie between 0 and 1
+and plays the same role as I<alpha> with respect to the predicted linear
+trend.
+
+I<gamma> is the adaption parameter of the seasonal coefficients in the
+Holt-Winters Forecasting algorithm (HWPREDICT) or the adaption parameter in
+the exponential smoothing update of the seasonal deviations. It must lie
+between 0 and 1. If the SEASONAL and DEVSEASONAL B<RRAs> are created
+implicitly, they will both have the same value for I<gamma>: the value
+specified for the HWPREDICT I<alpha> argument. Note that because there is
+one seasonal coefficient (or deviation) for each time point during the
+seasonal cycle, the adaption rate is much slower than the baseline. Each
+seasonal coefficient is only updated (or adapts) when the observed value
+occurs at the offset in the seasonal cycle corresponding to that
+coefficient.
+
+If SEASONAL and DEVSEASONAL B<RRAs> are created explicity, I<gamma> need not
+be the same for both. Note that I<gamma> can also be changed via the
+B<rrdtool> I<tune> command.
+
+I<rra num> provides the links between related B<RRAs>. If HWPREDICT is
+specified alone and the other B<RRAs> created implicitly, then there is no
+need to worry about this argument. If B<RRAs> are created explicitly, then
+pay careful attention to this argument. For each B<RRA> which includes this
+argument, there is a dependency between that B<RRA> and another B<RRA>. The
+I<rra num> argument is the 1-based index in the order of B<RRA> creation
+(that is, the order they appear in the I<create> command). The dependent
+B<RRA> for each B<RRA> requiring the I<rra num> argument is listed here:
+
+=over 4
+
+=item *
+
+HWPREDICT I<rra num> is the index of the SEASONAL B<RRA>.
+
+=item * 
+
+SEASONAL I<rra num> is the index of the HWPREDICT B<RRA>.
+
+=item * 
+
+DEVPREDICT I<rra num> is the index of the DEVSEASONAL B<RRA>.
+
+=item *
+
+DEVSEASONAL I<rra num> is the index of the HWPREDICT B<RRA>.
+
+=item * 
+
+FAILURES I<rra num> is the index of the DEVSEASONAL B<RRA>.
+
+=back
+
+I<threshold> is the minimum number of violations (observed values outside
+the confidence bounds) within a window that constitutes a failure. If the
+FAILURES B<RRA> is implicitly created, the default value is 7.
+
+I<window length> is the number of time points in the window. Specify an
+integer greater than or equal to the threshold and less than or equal to 28.
+The time interval this window represents depends on the interval between
+primary data points. If the FAILURES B<RRA> is implicity created, the
+default value is 9.
+
 =head1 The HEARTBEAT and the STEP
 
 Here is an explanation by Don Baarda on the inner workings of rrdtool.
@@ -232,6 +376,46 @@ every hour (12 * 300 seconds = 1 hour), for 100 days (2400 hours). The
 third and the fourth RRA's do the same with the for the maximum and
 average temperature, respectively.
 
+=head1 EXAMPLE 2
+
+C<rrdtool create monitor.rrd --step 300 
+DS:ifOutOctets:COUNTER:1800:0:4294967295
+RRA:AVERAGE:0.5:1:2016
+RRA:HWPREDICT:1440:0.1:0.0035:288>
+
+This example is a monitor of a router interface. The first B<RRA> tracks the
+traffic flow in octects; the second B<RRA> generates the specialized
+functions B<RRAs> for aberrant behavior detection. Note that the I<rra num>
+argument of HWPREDICT is missing, so the other B<RRAs> will be implicitly be
+created with default parameter values. In this example, the forecasting
+algorithm baseline adapts quickly; in fact the most recent one hour of
+observations (each at 5 minute intervals) account for 75% of the baseline
+prediction. The linear trend forecast adapts much more slowly. Observations
+made in during the last day (at 288 observations per day) account for only
+65% of the predicted linear trend. Note: these computations rely on an
+exponential smoothing formula described in a forthcoming LISA 2000 paper.
+
+The seasonal cycle is one day (288 data points at 300 second intervals), and
+the seasonal adaption paramter will be set to 0.1. The RRD file will store 5
+days (1440 data points) of forecasts and deviation predictions before wrap
+around. The file will store 1 day (a seasonal cycle) of 0-1 indicators in
+the FAILURES B<RRA>.
+
+The same RRD file and B<RRAs> are created with the following command, which explicitly
+creates all specialized function B<RRAs>.
+
+C<rrdtool create monitor.rrd --step 300 
+DS:ifOutOctets:COUNTER:1800:0:4294967295
+RRA:AVERAGE:0.5:1:2016
+RRA:HWPREDICT:1440:0.1:0.0035:288:3
+RRA:SEASONAL:288:0.1:2
+RRA:DEVPREDICT:1440:5
+RRA:DEVSEASONAL:288:0.1:2
+RRA:FAILURES:288:7:9:5>
+
+Of course, explicit creation need not replicate implicit create, a number of arguments 
+could be changed.
+
 =head1 AUTHOR
 
 Tobias Oetiker E<lt>oetiker@ee.ethz.chE<gt>
diff --git a/doc/rrdgraph.pod b/doc/rrdgraph.pod
index 2e3a4f6..8e8038e 100644
--- a/doc/rrdgraph.pod
+++ b/doc/rrdgraph.pod
@@ -41,6 +41,7 @@ S<[B<VRULE:>I<time>B<#>I<rrggbb>[B<:>I<legend>]]>
 S<[B<LINE>{B<1>|B<2>|B<3>}B<:>I<vname>[B<#>I<rrggbb>[B<:>I<legend>]]]>
 S<[B<AREA:>I<vname>[B<#>I<rrggbb>[B<:>I<legend>]]]>
 S<[B<STACK:>I<vname>[B<#>I<rrggbb>[B<:>I<legend>]]]>
+S<[B<TICK:>I<vname>B<#>I<rrggbb>[B<:>I<axis-fraction>[B<:>I<legend>]]]>
 
 =head1 DESCRIPTION
 
@@ -472,10 +473,18 @@ B<AREA> or B<LINE?> -- you need something to stack something onto in
 the first place ;) 
 
 Note, that when you STACK onto *UNKNOWN* data, rrdtool will not draw
-any graphics ... *UNKNOWN* is not zero ... if you want it to zero
+any graphics ... *UNKNOWN* is not zero ... if you want it to be zero
 then you might want to use a CDEF argument with IF and UN functions to
 turn *UNKNOWN* into zero ...
 
+=item B<TICK:>I<vname>B<#>I<rrggbb>[B<:>I<axis-fraction>[B<:>I<legend>]]
+
+Plot a tick mark (a vertical line) for each value of I<vname> that is
+non-zero and not *UNKNOWN*. The I<axis-fraction> argument specifies the
+length of the tick mark as a fraction of the y-axis; the default value
+is 0.1 (10% of the axis). Note that the color specification is not
+optional.
+
 =back
 
 =head1 NOTES on legend arguments
@@ -587,6 +596,61 @@ Note that this example assumes that your data is in the positive half of the y-a
 otherwhise you would would have to add NEGINF in order to extend the coverage
 of the rea to whole graph.
 
+=head1 EXAMPLE 4
+
+If the specialized function B<RRAs> exist for aberrant behavior detection, they
+can be used to generate the graph of a time series with confidence bands and
+failures.
+
+   rrdtool graph example.gif \
+          DEF:obs=monitor.rrd:ifOutOctets:AVERAGE \
+          DEF:pred=monitor.rrd:ifOutOctets:HWPREDICT \
+          DEF:dev=monitor.rrd:ifOutOctets:DEVPREDICT \
+          DEF:fail=monitor.rrd:ifOutOctets:FAILURES \
+          TICK:fail#ffffa0:1.0:"Failures\: Average bits out" \
+          CDEF:scaledobs=obs,8,* \
+          CDEF:upper=pred,dev,2,*,+ \
+          CDEF:lower=pred,dev,2,*,- \
+          CDEF:scaledupper=upper,8,* \
+          CDEF:scaledlower=lower,8,* \
+          LINE2:scaledobs#0000ff:"Average bits out" \
+          LINE1:scaledupper#ff0000:"Upper Confidence Bound: Average bits out" \
+          LINE1:scaledlower#ff0000:"Lower Confidence Bound: Average bits out"
+
+This example generates a graph of the data series in blue (LINE2 with the scaledobs
+virtual data source), confidence bounds in red (scaledupper and scaledlower virtual
+data sources), and potential failures (i.e. potential aberrant aberrant behavior)
+marked by vertical yellow lines (the fail data source).
+
+The raw data comes from an AVERAGE B<RRA>, the finest resolution of the observed
+time series (one consolidated data point per primary data point). The predicted
+(or smoothed) values are stored in the HWPREDICT B<RRA>. The predicted deviations
+(think standard deviation) values are stored in the DEVPREDICT B<RRA>. Finally,
+the FAILURES B<RRA> contains indicators, with 1 denoting a potential failure.
+
+All of the data is rescaled to bits (instead of Octets) by multiplying by 8.
+The confidence bounds are computed by an offset of 2 deviations both above
+and below the predicted values (the CDEFs upper and lower). Vertical lines
+indicated potential failures are graphed via the TICK graph element, which
+converts non-zero values in an B<RRA> into tick marks. Here an axis-fraction
+argument of 1.0 means the tick marks span the entire y-axis, and hence become
+vertical lines on the graph.
+
+The choice of 2 deviations (a scaling factor) matches the default used internally
+by the FAILURES B<RRA>. If the internal value is changed (see L<rrdtune>), this
+graphing command should be changed to be consistent.
+
+=head2 A note on data reduction:
+
+The B<rrdtool> I<graph> command is designed to plot data at a specified temporal
+resolution, regardless of the actually resolution of the data in the RRD file.
+This can present a problem for the specialized consolidation functions which
+maintain a one-to-one mapping between primary data points and consolidated
+data points. If a graph insists on viewing the contents of these B<RRAs> on a
+coarser temporal scale, the I<graph> command tries to do something intelligent,
+but the confidence bands and failures no longer have the same meaning and may
+be misleading.
+
 =head1 AUTHOR
 
 Tobias Oetiker E<lt>oetiker@ee.ethz.chE<gt>
diff --git a/doc/rrdtool.pod b/doc/rrdtool.pod
index e487863..fee6d35 100644
--- a/doc/rrdtool.pod
+++ b/doc/rrdtool.pod
@@ -170,6 +170,63 @@ B<RRD>s. The graphing feature is fully configurable. Size, color and
 contents of the graph can be defined freely. Check L<rrdgraph>
 for more information on this.
 
+=item Aberrant Behavior Detection
+
+by Jake Brutlag E<lt>jakeb@corp.webtv.netE<gt>
+
+The  B<rrdtool> also provides the building blocks for near real-time
+aberrant behavior detection. These components include:
+
+=over 12
+
+=item *
+
+An algorithm for predicting the values time series one time step into the future.
+
+=item *
+
+A measure of deviation between the predicted values and the observed values.
+
+=item *
+
+A mechanism to decide if and when an observed value
+or sequence of observed values is I<too deviant> from the predicted value(s).
+
+=back
+
+Each of these components is briefly described:
+
+Holt-Winters Time Series Forecasting Algorithm is an online, or incremental, 
+algorithm that adaptively predicts future observations in a time series. It's 
+forecast is the sum of three components: a baseline (or intercept), a linear 
+trend over time (or slope), and a seasonal coefficient (a periodic effect, 
+such as a daily cycle). There is one seasonal coefficient for each time point 
+in the period (cycle). After a value is observed, each of these components is 
+updated via exponential smoothing. So the algorithm learns from past values 
+and uses them to predict the future. The rate of adaptation is governed by 
+3 parameters, alpha (intercept), beta (slope), and gamma (seasonal). The prediction 
+can also be viewed as a smoothed value for the time series.
+
+The measure of deviation is a seasonal weighted absolute deviation. The term 
+I<seasonal> means deviation is measured separately for each time point in the 
+seasonal cycle. As with Holt-Winters Forecasting, deviation is predicted using 
+the measure computed from past values (but only at that point in the seasonal cycle). 
+After the value is observed, the algorithm learns from the observed value via 
+exponential smoothing. Confidence bands for the observed time series are generated 
+by scaling the sequence of predicted deviation values (we usually think of the sequence 
+as a continuous line rather than a set of discrete points).
+
+Aberrant behavior (a potential failure) is reported whenever the number of 
+times the observed value violates the confidence bands meets or exceeds a 
+specified threshold within a specified temporal window (i.e. 5 violations 
+during the past 45 minutes with a value observed every 5 mintues).
+
+This functionality is embedded in a set of related B<RRAs>. In particular, a FAILURES
+B<RRA> logs potential failures. Presumably a front-end application to B<rrdtool> can
+utilize this B<RRA> to initiate real-time alerts if that is desired.
+
+You can find a detailed description of how to set this up in L<rrdcreate>.
+
 =back
 
 =head2 REMOTE CONTROL
diff --git a/doc/rrdtune.pod b/doc/rrdtune.pod
index badd5d5..f96d8d1 100644
--- a/doc/rrdtune.pod
+++ b/doc/rrdtune.pod
@@ -12,6 +12,13 @@ S<[B<--minimum>|B<-i> I<ds-name>:I<min>]>
 S<[B<--maximum>|B<-a> I<ds-name>:I<max>]>
 S<[B<--data-source-type>|B<-d> I<ds-name>:I<DST>]>
 S<[B<--data-source-rename>|B<-r> I<old-name>:I<new-name>]>
+S<[B<--deltapos> I<scale-value>]>
+S<[B<--deltaneg> I<scale value>]>
+S<[B<--failure-threshold> I<failure-threshold>]>
+S<[B<--window-length> I<window-length>]>
+S<[B<--alpha> I<adaption-parameter>]>
+S<[B<--beta> I<adaption-parameter>]>
+S<[B<--gamma> I<adaption-parameter>]>
 
 =head1 DESCRIPTION
 
@@ -21,11 +28,14 @@ All these tunable parameters together decide when data fed into an
 B<RRD> is to be regarded as invalid. Invalid data is entered into the 
 database as *UNKNOWN*.
 
-The main application of the B<tune> function is to relax the 
+One application of the B<tune> function is to relax the 
 validation rules on an B<RRD>. This allows to fill a new B<RRD> with
 data available in larger intervals than what you would normally want
 to permit.
 
+A second application of the B<tune> function is to set or alter parameters
+used by the specialized function B<RRAs> for aberrant behavior detection.
+
 =over 8
 
 =item I<filename>
@@ -51,13 +61,59 @@ Setting I<max> to 'U' will disable this limit.
 
 alter the type B<DST> of a data source.
 
-=item S<[B<--data-source-rename>|B<-r> I<old-name>:I<new-name>]>
+=item S<B<--data-source-rename>|B<-r> I<old-name>:I<new-name>>
 
 rename a data source
 
+=item S<B<--deltapos> I<scale-value>>
+
+Alter the deviation scaling factor for the upper bound of the confidence band
+used internally to calculate violations for the FAILURES B<RRA>. The default
+value is 2. Note that this parameter is not related to graphing confidence
+bounds, that scale factor is specified as a CDEV argument to generate a graph with
+confidence bounds. It need agree with the value used internally by the FAILURES
+B<RRA> (although common sense dictates it should).
+
+=item S<B<--deltaneg> I<scale-value>>
+
+Alter the deviation scaling factor for the lower bound of the confidence band
+used internally to calculate violations for the FAILURES B<RRA>. The default
+value is 2. As with B<--deltapos>, this argument is unrelated to the scale
+factor chosen when graphing confidence bounds.
+
+=item S<B<--failure-threshold> I<failure-threshold>>
+
+Alter the number of confidence bound violations that constitute a failure for
+purposes of the FAILURES B<RRA>. This must be an integer less than or equal to
+the window length of the FAILURES B<RRA>. This restriction is not verified by
+the tune option, so one can reset failure-threshold and window-length 
+simultaneously. Setting this option will reset the count of violations to 0.
+
+=item S<B<--window-length> I<window-length>>
+
+Alter the number of time points in the temporal window for determining failures.
+This must be an integer greater than or equal to the window length of the
+FAILURES B<RRA> and less than or equal to 28. Setting this option will reset the
+count of violations to 0.
+
+=item S<B<--alpha> I<adaption-parameter>>
+
+Alter the intercept adaptation parameter for the Holt-Winters forecasting algorithm.
+Must be between 0 and 1.
+
+=item S<B<--beta> I<adaption-parameter>>
+
+Alter the intercept adaptation parameter for the Holt-Winters forecasting algorithm.
+Must be between 0 and 1.
+
+=item S<B<--gamma> I<adaption-parameter>>
+
+Alter the seasonal coefficient and deviation adaptation parameters the SEASONAL and
+DEVSEAONAL B<RRAs>. Must be between 0 and 1.
+
 =back
 
-=head1 EXAMPLE
+=head1 EXAMPLE 1
 
 C<rrdtool tune data.rrd -h in:100000 -h out:100000 -h through:100000>
 
@@ -66,6 +122,14 @@ and 'through' to 10000 seconds which is a little over one day in data.rrd.
 This would allow to feed old data from mrtg-2.0 right into
 rrdtool without generating *UNKNOWN* entries.
 
+=head1 EXAMPLE 2
+
+C<rrdtool tune monitor.rrd --window-length 5 --failure-threshold 3>
+
+If the FAILURES B<RRA> is implicitly created, the default window-length is 9 and 
+the default failure-thresold is 7. This command now defines a failure as 3 or more
+violations in a temporal window of 5 time points.
+
 =head1 AUTHOR
 
 Tobias Oetiker <oetiker@ee.ethz.ch>
diff --git a/src/Makefile.am b/src/Makefile.am
index 875c007..33f6790 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -22,6 +22,8 @@ RRD_C_FILES =		\
 	getopt1.c	\
 	gifsize.c	\
 	parsetime.c	\
+	hash_32.c	\
+        rrd_hw.c	\
 	pngsize.c	\
 	rrd_create.c	\
 	rrd_diff.c	\
diff --git a/src/fnv.h b/src/fnv.h
new file mode 100644
index 0000000..205e5a8
--- /dev/null
+++ b/src/fnv.h
@@ -0,0 +1,102 @@
+/*
+ * fnv - Fowler/Noll/Vo- hash code
+ *
+ * @(#) $Revision$
+ * @(#) $Id$
+ * @(#) $Source$
+ *
+ ***
+ *
+ * Fowler/Noll/Vo- hash
+ *
+ * The basis of this hash algorithm was taken from an idea sent
+ * as reviewer comments to the IEEE POSIX P1003.2 committee by:
+ *
+ *      Phong Vo (http://www.research.att.com/info/kpv/)
+ *      Glenn Fowler (http://www.research.att.com/~gsf/)
+ *
+ * In a subsequent ballot round:
+ *
+ *      Landon Curt Noll (http://reality.sgi.com/chongo/)
+ *
+ * improved on their algorithm.  Some people tried this hash
+ * and found that it worked rather well.  In an EMail message
+ * to Landon, they named it the ``Fowler/Noll/Vo'' or FNV hash.
+ *
+ * FNV hashes are architected to be fast while maintaining a low
+ * collision rate. The FNV speed allows one to quickly hash lots
+ * of data while maintaining a reasonable collision rate.  See:
+ *
+ *      http://reality.sgi.com/chongo/tech/comp/fnv/
+ *
+ * for more details as well as other forms of the FNV hash.
+ *
+ ***
+ *
+ * NOTE: The FNV-0 historic hash is not recommended.  One should use
+ *	 the FNV-1 hash instead.
+ *
+ * To use the 32 bit FNV-0 historic hash, pass FNV0_32_INIT as the
+ * Fnv32_t hashval argument to fnv_32_buf() or fnv_32_str().
+ *
+ * To use the 64 bit FNV-0 historic hash, pass FNV0_64_INIT as the
+ * Fnv64_t hashval argument to fnv_64_buf() or fnv_64_str().
+ *
+ * To use the recommended 32 bit FNV-1 hash, pass FNV1_32_INIT as the
+ * Fnv32_t hashval argument to fnv_32_buf() or fnv_32_str().
+ *
+ * To use the recommended 64 bit FNV-1 hash, pass FNV1_64_INIT as the
+ * Fnv64_t hashval argument to fnv_64_buf() or fnv_64_str().
+ *
+ ***
+ *
+ * Please do not copyright this code.  This code is in the public domain.
+ *
+ * LANDON CURT NOLL DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE,
+ * INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO
+ * EVENT SHALL LANDON CURT NOLL BE LIABLE FOR ANY SPECIAL, INDIRECT OR
+ * CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF
+ * USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR
+ * OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+ * PERFORMANCE OF THIS SOFTWARE.
+ *
+ * By:
+ *	chongo <Landon Curt Noll> /\oo/\
+ *	http://reality.sgi.com/chongo/
+ *	EMail: chongo_fnv at prime dot engr dot sgi dot com
+ *
+ * Share and Enjoy!	:-)
+ */
+
+#if !defined(__FNV_H__)
+#define __FNV_H__
+
+
+/*
+ * 32 bit FNV-0 hash type
+ */
+typedef unsigned long Fnv32_t;
+
+
+/* 
+ * 32 bit FNV-0 zero initial basis
+ *
+ * This historic hash is not recommended.  One should use
+ * the FNV-1 hash and inital basis instead.
+ */
+#define FNV0_32_INIT ((Fnv32_t)0)
+
+
+/*
+ * 32 bit FNV-1 non-zero initial basis
+ *
+ * The FNV-1 initial basis is the FNV-0 hash of the following 32 octets:
+ *
+ *              chongo <Landon Curt Noll> /\../\
+ *
+ * Note that the \'s above are not back-slashing escape characters.
+ * They are literal ASCII  backslash 0x5c characters.
+ */
+#define FNV1_32_INIT ((Fnv32_t)0x811c9dc5)
+
+#endif /* __FNV_H__ */
diff --git a/src/hash_32.c b/src/hash_32.c
new file mode 100644
index 0000000..3fda164
--- /dev/null
+++ b/src/hash_32.c
@@ -0,0 +1,152 @@
+/*
+ * hash_32 - 32 bit Fowler/Noll/Vo hash code
+ *
+ *
+ ***
+ *
+ * Fowler/Noll/Vo hash
+ *
+ * The basis of this hash algorithm was taken from an idea sent
+ * as reviewer comments to the IEEE POSIX P1003.2 committee by:
+ *
+ *      Phong Vo (http://www.research.att.com/info/kpv/)
+ *      Glenn Fowler (http://www.research.att.com/~gsf/)
+ *
+ * In a subsequent ballot round:
+ *
+ *      Landon Curt Noll (http://reality.sgi.com/chongo/)
+ *
+ * improved on their algorithm.  Some people tried this hash
+ * and found that it worked rather well.  In an EMail message
+ * to Landon, they named it the ``Fowler/Noll/Vo'' or FNV hash.
+ *
+ * FNV hashes are architected to be fast while maintaining a low
+ * collision rate. The FNV speed allows one to quickly hash lots
+ * of data while maintaining a reasonable collision rate.  See:
+ *
+ *      http://reality.sgi.com/chongo/tech/comp/fnv/
+ *
+ * for more details as well as other forms of the FNV hash.
+ ***
+ *
+ * NOTE: The FNV-0 historic hash is not recommended.  One should use
+ *	 the FNV-1 hash instead.
+ *
+ * To use the 32 bit FNV-0 historic hash, pass FNV0_32_INIT as the
+ * Fnv32_t hashval argument to fnv_32_buf() or fnv_32_str().
+ *
+ * To use the recommended 32 bit FNV-1 hash, pass FNV1_32_INIT as the
+ * Fnv32_t hashval argument to fnv_32_buf() or fnv_32_str().
+ *
+ ***
+ *
+ * Please do not copyright this code.  This code is in the public domain.
+ *
+ * LANDON CURT NOLL DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE,
+ * INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO
+ * EVENT SHALL LANDON CURT NOLL BE LIABLE FOR ANY SPECIAL, INDIRECT OR
+ * CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF
+ * USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR
+ * OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+ * PERFORMANCE OF THIS SOFTWARE.
+ *
+ * By:
+ *	chongo <Landon Curt Noll> /\oo/\
+ *	http://reality.sgi.com/chongo/
+ *	EMail: chongo_fnv at prime dot engr dot sgi dot com
+ *
+ * Share and Enjoy!	:-)
+ */
+
+#include <stdlib.h>
+#include "fnv.h"
+
+
+/* 
+ * 32 bit magic FNV-0 and FNV-1 prime 
+ */
+#define FNV_32_PRIME ((Fnv32_t)0x01000193)	
+
+
+/*
+ * fnv_32_buf - perform a 32 bit Fowler/Noll/Vo hash on a buffer
+ *
+ * input:
+ *	buf	- start of buffer to hash
+ *	len	- length of buffer in octets
+ *	hval	- previous hash value or 0 if first call
+ *
+ * returns:
+ *	32 bit hash as a static hash type
+ *
+ * NOTE: To use the 32 bit FNV-0 historic hash, use FNV0_32_INIT as the hval
+ *	 argument on the first call to either fnv_32_buf() or fnv_32_str().
+ *
+ * NOTE: To use the recommended 32 bit FNV-1 hash, use FNV1_32_INIT as the hval
+ *	 argument on the first call to either fnv_32_buf() or fnv_32_str().
+ */
+Fnv32_t
+fnv_32_buf(void *buf, size_t len, Fnv32_t hval)
+{
+    unsigned char *bp = (unsigned char *)buf;	/* start of buffer */
+    unsigned char *be = bp + len;		/* beyond end of buffer */
+
+    /*
+     * FNV-1 hash each octet in the buffer
+     */
+    while (bp < be) {
+
+	/* multiply by the 32 bit FNV magic prime mod 2^64 */
+	hval *= FNV_32_PRIME;
+
+	/* xor the bottom with the current octet */
+	hval ^= (Fnv32_t)*bp++;
+    }
+
+    /* return our new hash value */
+    return hval;
+}
+
+
+/*
+ * fnv_32_str - perform a 32 bit Fowler/Noll/Vo hash on a string
+ *
+ * input:
+ *	str	- string to hash
+ *	hval	- previous hash value or 0 if first call
+ *
+ * returns:
+ *	32 bit hash as a static hash type
+ *
+ * NOTE: To use the 32 bit FNV-0 historic hash, use FNV0_32_INIT as the hval
+ *	 argument on the first call to either fnv_32_buf() or fnv_32_str().
+ *
+ * NOTE: To use the recommended 32 bit FNV-1 hash, use FNV1_32_INIT as the hval
+ *	 argument on the first call to either fnv_32_buf() or fnv_32_str().
+ */
+Fnv32_t
+fnv_32_str(char *str, Fnv32_t hval)
+{
+    unsigned char *s = (unsigned char *)str;	/* unsigned string */
+
+    /*
+     * FNV-1 hash each octet in the buffer
+     */
+    while (*s) {
+
+	/* multiply by the 32 bit FNV magic prime mod 2^64 */
+	hval *= FNV_32_PRIME;
+
+	/* xor the bottom with the current octet */
+	hval ^= (Fnv32_t)*s++;
+    }
+
+    /* return our new hash value */
+    return hval;
+}
+
+/* a wrapper function for fnv_32_str */
+unsigned long FnvHash(char *str)
+{
+  return fnv_32_str(str,FNV1_32_INIT);
+}
diff --git a/src/rrd_create.c b/src/rrd_create.c
index f118da6..d60e2e9 100644
--- a/src/rrd_create.c
+++ b/src/rrd_create.c
@@ -6,7 +6,10 @@
 
 #include "rrd_tool.h"
 
+/* prototype for FnvHash */
+unsigned long FnvHash(char *str);
 
+/* #define DEBUG */
 int
 rrd_create(int argc, char **argv) 
 {
@@ -15,7 +18,9 @@ rrd_create(int argc, char **argv)
     time_t             last_up;
     struct time_value last_up_tv;
     char *parsetime_error = NULL;
-
+    char *token;
+    unsigned short token_idx, error_flag, period=0;
+	unsigned long hashed_name;
     /* init last_up */
     last_up = time(NULL)-10;
     /* init rrd clean */
@@ -34,7 +39,8 @@ rrd_create(int argc, char **argv)
 
     /* set some defaults */
     strcpy(rrd.stat_head->cookie,RRD_COOKIE);
-    strcpy(rrd.stat_head->version,RRD_VERSION);
+	/* assume the will be version 1 compatible */
+    strcpy(rrd.stat_head->version,"0001");
     rrd.stat_head->float_cookie = FLOAT_COOKIE;
     rrd.stat_head->ds_cnt = 0; /* this will be adjusted later */
     rrd.stat_head->rra_cnt = 0; /* ditto */
@@ -103,7 +109,12 @@ rrd_create(int argc, char **argv)
 	}
     }
     rrd.live_head->last_up = last_up;
-
+	
+	/* optind points to the first non-option command line arg,
+	 * in this case, the file name. */
+	/* Compute the FNV hash value (used by SEASONAL and DEVSEASONAL
+	 * arrays. */
+	hashed_name = FnvHash(argv[optind]);
     for(i=optind+1;i<argc;i++){
 	char minstr[DS_NAM_SIZE], maxstr[DS_NAM_SIZE];	
 	int ii;
@@ -162,36 +173,217 @@ rrd_create(int argc, char **argv)
 	} else if (strncmp(argv[i],"RRA:",3)==0){
 	    size_t old_size = sizeof(rra_def_t)*(rrd.stat_head->rra_cnt);
 	    if((rrd.rra_def = rrd_realloc(rrd.rra_def,
-				      old_size+sizeof(rra_def_t)))==NULL){
-		rrd_set_error("allocating rrd.rra_def");
-		rrd_free(&rrd);
-		return(-1);	
+				      old_size+sizeof(rra_def_t)))==NULL)
+	    {
+		   rrd_set_error("allocating rrd.rra_def");
+		   rrd_free(&rrd);
+		   return(-1);	
 	    }
 	    memset(&rrd.rra_def[rrd.stat_head->rra_cnt], 0, sizeof(rra_def_t));
-	    if (sscanf(&argv[i][4],
-		       CF_NAM_FMT ":%lf:%lu:%lu",
-		       rrd.rra_def[rrd.stat_head->rra_cnt].cf_nam,
-		       &rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_cdp_xff_val].u_val,
-		       &rrd.rra_def[rrd.stat_head->rra_cnt].pdp_cnt,
-		       &rrd.rra_def[rrd.stat_head->rra_cnt].row_cnt) == 4){
-		if(cf_conv(rrd.rra_def[rrd.stat_head->rra_cnt].cf_nam) == -1){
-		    rrd_free(&rrd);
-		    return (-1);
+
+	    token = strtok(&argv[i][4],":");
+	    token_idx = error_flag = 0;
+	    while (token != NULL)
+	    {
+	      switch(token_idx)
+	      {
+	      case 0:
+		if (sscanf(token,CF_NAM_FMT,
+			   rrd.rra_def[rrd.stat_head->rra_cnt].cf_nam) != 1)
+		  rrd_set_error("Failed to parse CF name");
+		switch(cf_conv(rrd.rra_def[rrd.stat_head->rra_cnt].cf_nam))
+		{
+		case CF_HWPREDICT:
+		  /* initialize some parameters */
+		  rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_hw_alpha].u_val = 0.1;
+		  rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_hw_beta].u_val = 1.0/288;
+		  rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_dependent_rra_idx].u_cnt = 
+		    rrd.stat_head -> rra_cnt;
+		  /* need to mark the file version */
+          strcpy(rrd.stat_head->version,RRD_VERSION);
+		  break;
+		case CF_DEVSEASONAL:
+		case CF_SEASONAL:
+		  /* initialize some parameters */
+		  rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_seasonal_gamma].u_val = 0.1;
+		  /* fall through */
+		case CF_DEVPREDICT:
+		  rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_dependent_rra_idx].u_cnt = -1;
+		  break;
+		case CF_FAILURES:
+		  rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_delta_pos].u_val = 2.0;
+		  rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_delta_neg].u_val = 2.0;
+		  rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_window_len].u_cnt = 3;
+		  rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_failure_threshold].u_cnt = 2;
+		  rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_dependent_rra_idx].u_cnt = -1;
+		  break;
+		/* invalid consolidation function */
+		case -1:
+		  rrd_set_error("Unrecognized consolidation function %s",
+			 rrd.rra_def[rrd.stat_head->rra_cnt].cf_nam);
+		default:
+		  break;
 		}
-	        if (rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_cdp_xff_val].u_val<0.0 ||
-		    rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_cdp_xff_val].u_val>=1.0) {
-		    rrd_set_error("the xff must always be >= 0 and < 1");
-		    rrd_free(&rrd);
-		    return (-1);
+		/* default: 1 pdp per cdp */ 
+		rrd.rra_def[rrd.stat_head->rra_cnt].pdp_cnt = 1;
+		break;
+	      case 1:
+		switch(cf_conv(rrd.rra_def[rrd.stat_head->rra_cnt].cf_nam))
+		{
+		case CF_HWPREDICT:
+		case CF_DEVSEASONAL:
+		case CF_SEASONAL:
+		case CF_DEVPREDICT:
+		case CF_FAILURES:
+		  rrd.rra_def[rrd.stat_head->rra_cnt].row_cnt = atoi(token);
+		  break;
+		default:
+		  rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_cdp_xff_val].u_val = atof(token);
+		  if (rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_cdp_xff_val].u_val<0.0 ||
+		      rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_cdp_xff_val].u_val>=1.0)
+		    rrd_set_error("Invalid xff: must be between 0 and 1");
+		  break;
 		}
-		rrd.stat_head->rra_cnt++;	    		
-	    }
-	    else {  
-		rrd_set_error("can't parse argument '%s'",argv[i]);
+		break;
+	      case 2:
+		switch(cf_conv(rrd.rra_def[rrd.stat_head->rra_cnt].cf_nam))
+		{
+		case CF_HWPREDICT:
+		  rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_hw_alpha].u_val = atof(token);
+		  if (atof(token) <= 0.0 || atof(token) >= 1.0)
+		    rrd_set_error("Invalid alpha: must be between 0 and 1");
+		  break;
+		case CF_DEVSEASONAL:
+		case CF_SEASONAL:
+		  rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_seasonal_gamma].u_val = 
+			atof(token);
+		  if (atof(token) <= 0.0 || atof(token) >= 1.0)
+		    rrd_set_error("Invalid gamma: must be between 0 and 1");
+		  rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_seasonal_smooth_idx].u_cnt
+			= hashed_name % rrd.rra_def[rrd.stat_head->rra_cnt].row_cnt; 
+		  break;
+	    case CF_FAILURES:
+		  /* specifies the # of violations that constitutes the failure threshold */
+		  rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_failure_threshold].u_cnt =
+			atoi(token);
+		  if (atoi(token) < 1 || atoi(token) > MAX_FAILURES_WINDOW_LEN)
+			rrd_set_error("Failure threshold is out of range %d, %d",1,
+			  MAX_FAILURES_WINDOW_LEN);
+		  break;
+		case CF_DEVPREDICT:
+		  /* specifies the index (1-based) of CF_DEVSEASONAL array
+		   * associated with this CF_DEVPREDICT array. */
+		  rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_dependent_rra_idx].u_cnt =
+		    atoi(token) - 1;
+		  break;
+		default:
+		  rrd.rra_def[rrd.stat_head->rra_cnt].pdp_cnt = atoi(token);
+		  break;
+		}
+		break;
+	      case 3:
+		switch(cf_conv(rrd.rra_def[rrd.stat_head->rra_cnt].cf_nam))
+		{
+		case CF_HWPREDICT:
+		  rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_hw_beta].u_val = atof(token);
+		  if (atof(token) < 0.0 || atof(token) > 1.0)
+		    rrd_set_error("Invalid beta: must be between 0 and 1");
+		  break;
+		case CF_DEVSEASONAL:
+		case CF_SEASONAL:
+		  /* specifies the index (1-based) of CF_HWPREDICT array
+		   * associated with this CF_DEVSEASONAL or CF_SEASONAL array. 
+		   * */
+		  rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_dependent_rra_idx].u_cnt =
+		    atoi(token) - 1;
+		  break;
+		case CF_FAILURES:
+		  /* specifies the window length */
+		  rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_window_len].u_cnt =
+			atoi(token);
+		  if (atoi(token) < 1 || atoi(token) > MAX_FAILURES_WINDOW_LEN)
+			rrd_set_error("Window length is out of range %d, %d",1,
+			   MAX_FAILURES_WINDOW_LEN);
+		  /* verify that window length exceeds the failure threshold */
+		  if (rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_window_len].u_cnt <
+			  rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_failure_threshold].u_cnt)
+		    rrd_set_error("Window length is shorter than the failure threshold");
+		  break;
+		case CF_DEVPREDICT:
+		  /* shouldn't be any more arguments */
+		  rrd_set_error("Unexpected extra argument for consolidation function DEVPREDICT");
+		  break;
+		default:
+		  rrd.rra_def[rrd.stat_head->rra_cnt].row_cnt = atoi(token);
+		  break;
+		}
+		break;
+	      case 4:
+		switch(cf_conv(rrd.rra_def[rrd.stat_head->rra_cnt].cf_nam))
+		{
+		case CF_FAILURES:
+		  /* specifies the index (1-based) of CF_DEVSEASONAL array
+		   * associated with this CF_DEVFAILURES array. */
+		  rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_dependent_rra_idx].u_cnt =
+		    atoi(token) - 1;
+		  break;
+		case CF_HWPREDICT:
+		  /* length of the associated CF_SEASONAL and CF_DEVSEASONAL arrays. */
+		  period = atoi(token);
+          if (period > rrd.rra_def[rrd.stat_head->rra_cnt].row_cnt)
+		    rrd_set_error("Length of seasonal cycle exceeds length of HW prediction array");
+		  break;
+		default:
+		  /* shouldn't be any more arguments */
+		  rrd_set_error("Unexpected extra argument for consolidation function %s",
+				rrd.rra_def[rrd.stat_head->rra_cnt].cf_nam);
+                  break;
+		}
+		break;
+	      case 5:
+		/* If we are here, this must be a CF_HWPREDICT RRA.
+		 * Specifies the index (1-based) of CF_SEASONAL array
+		 * associated with this CF_HWPREDICT array. If this argument 
+		 * is missing, then the CF_SEASONAL, CF_DEVSEASONAL, CF_DEVPREDICT,
+		 * CF_FAILURES.
+		 * arrays are created automatically. */
+		rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_dependent_rra_idx].u_cnt =
+		  atoi(token) - 1;
+		break;
+	      default:
+		/* should never get here */
+		rrd_set_error("Unknown error");
+		break;
+	      } /* end switch */
+	      if (rrd_test_error())
+	      {
+		/* all errors are unrecoverable */
 		rrd_free(&rrd);
-		return (-1);		
+		return (-1);
+	      }
+	      token = strtok(NULL,":");
+	      token_idx++;
+	    } /* end while */
+#ifdef DEBUG
+	    fprintf(stderr,"Creating RRA CF: %s, dep idx %lu, current idx %lu\n",
+		    rrd.rra_def[rrd.stat_head->rra_cnt].cf_nam,
+		    rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_dependent_rra_idx].u_cnt, 
+		    rrd.stat_head -> rra_cnt);
+#endif
+	    /* should we create CF_SEASONAL, CF_DEVSEASONAL, and CF_DEVPREDICT? */
+	    if (cf_conv(rrd.rra_def[rrd.stat_head->rra_cnt].cf_nam) == CF_HWPREDICT
+		&& rrd.rra_def[rrd.stat_head->rra_cnt].par[RRA_dependent_rra_idx].u_cnt 
+		== rrd.stat_head -> rra_cnt)
+	    {
+#ifdef DEBUG
+	      fprintf(stderr,"Creating HW contingent RRAs\n");
+#endif
+	      if (create_hw_contingent_rras(&rrd,period,hashed_name) == -1) {
+		rrd_free(&rrd);
+		return -1;
+	      }
 	    }
-
+	    rrd.stat_head->rra_cnt++;	    		
 	} else {
 	    rrd_set_error("can't parse argument '%s'",argv[i]);
 	    rrd_free(&rrd);
@@ -214,6 +406,75 @@ rrd_create(int argc, char **argv)
     return rrd_create_fn(argv[optind],&rrd);
 }
 
+/* Create the CF_DEVPREDICT, CF_DEVSEASONAL, CF_SEASONAL, and CF_FAILURES RRAs
+ * associated with a CF_HWPREDICT RRA. */
+int
+create_hw_contingent_rras(rrd_t *rrd, unsigned short period, unsigned long hashed_name)
+{
+   size_t old_size;
+   rra_def_t* current_rra;
+   
+   /* save index to CF_HWPREDICT */
+   unsigned long hw_index = rrd -> stat_head -> rra_cnt;
+   /* advance the pointer */
+   (rrd -> stat_head -> rra_cnt)++;	    		
+   /* allocate the memory for the 4 contingent RRAs */
+   old_size = sizeof(rra_def_t)*(rrd -> stat_head->rra_cnt);
+   if ((rrd -> rra_def = rrd_realloc(rrd -> rra_def,
+       old_size+4*sizeof(rra_def_t)))==NULL)
+   {
+     rrd_set_error("allocating rrd.rra_def");
+     return(-1);	
+   }
+   /* clear memory */
+   memset(&(rrd -> rra_def[rrd -> stat_head->rra_cnt]), 0, 4*sizeof(rra_def_t));
+
+   /* create the CF_SEASONAL RRA */
+   current_rra = &(rrd -> rra_def[rrd -> stat_head -> rra_cnt]);
+   strcpy(current_rra -> cf_nam,"SEASONAL");
+   current_rra -> row_cnt = period;
+   current_rra -> par[RRA_seasonal_smooth_idx].u_cnt = hashed_name % period;
+   current_rra -> pdp_cnt = 1;
+   current_rra -> par[RRA_seasonal_gamma].u_val = 
+     rrd -> rra_def[hw_index].par[RRA_hw_alpha].u_val;
+   current_rra -> par[RRA_dependent_rra_idx].u_cnt = hw_index; 
+   rrd -> rra_def[hw_index].par[RRA_dependent_rra_idx].u_cnt = rrd -> stat_head -> rra_cnt;
+
+   /* create the CF_DEVSEASONAL RRA */
+   (rrd -> stat_head -> rra_cnt)++; 
+   current_rra = &(rrd -> rra_def[rrd -> stat_head -> rra_cnt]);
+   strcpy(current_rra -> cf_nam,"DEVSEASONAL");
+   current_rra -> row_cnt = period;
+   current_rra -> par[RRA_seasonal_smooth_idx].u_cnt = hashed_name % period;
+   current_rra -> pdp_cnt = 1;
+   current_rra -> par[RRA_seasonal_gamma].u_val = 
+     rrd -> rra_def[hw_index].par[RRA_hw_alpha].u_val;
+   current_rra -> par[RRA_dependent_rra_idx].u_cnt = hw_index; 
+   
+   /* create the CF_DEVPREDICT RRA */
+   (rrd -> stat_head -> rra_cnt)++; 
+   current_rra = &(rrd -> rra_def[rrd -> stat_head -> rra_cnt]);
+   strcpy(current_rra -> cf_nam,"DEVPREDICT");
+   current_rra -> row_cnt = (rrd -> rra_def[hw_index]).row_cnt;
+   current_rra -> pdp_cnt = 1;
+   current_rra -> par[RRA_dependent_rra_idx].u_cnt 
+     = hw_index + 2; /* DEVSEASONAL */
+
+   /* create the CF_FAILURES RRA */
+   (rrd -> stat_head -> rra_cnt)++; 
+   current_rra = &(rrd -> rra_def[rrd -> stat_head -> rra_cnt]);
+   strcpy(current_rra -> cf_nam,"FAILURES");
+   current_rra -> row_cnt = period; 
+   current_rra -> pdp_cnt = 1;
+   current_rra -> par[RRA_delta_pos].u_val = 2.0;
+   current_rra -> par[RRA_delta_neg].u_val = 2.0;
+   current_rra -> par[RRA_failure_threshold].u_cnt = 7;
+   current_rra -> par[RRA_window_len].u_cnt = 9;
+   current_rra -> par[RRA_dependent_rra_idx].u_cnt = 
+	 hw_index + 2; /* DEVSEASONAL */
+   return 0;
+}
+
 /* create and empty rrd file according to the specs given */
 
 int
@@ -266,21 +527,50 @@ rrd_create_fn(char *file_name, rrd_t *rrd)
 	return(-1);
     }
 
-    /* can not be zero because we don't know nothing ... */
-    rrd->cdp_prep->scratch[CDP_val].u_val = DNAN;
-    for(i=0; i < rrd->stat_head->rra_cnt; i++) {
-
-	/* startup missing pdp count */
-	rrd->cdp_prep->scratch[CDP_unkn_pdp_cnt].u_cnt = 
-	    ((rrd->live_head->last_up -
-	     rrd->pdp_prep->scratch[PDP_unkn_sec_cnt].u_cnt)
-	    % (rrd->stat_head->pdp_step 
-	       * rrd->rra_def[i].pdp_cnt)) / rrd->stat_head->pdp_step;	
 
-
-	for(ii=0; ii < rrd->stat_head->ds_cnt; ii++) {
-	    fwrite( rrd->cdp_prep,sizeof(cdp_prep_t),1,rrd_file);
-	}
+    for(i=0; i < rrd->stat_head->rra_cnt; i++) {
+       switch (cf_conv(rrd->rra_def[i].cf_nam))
+	   {
+		  case CF_HWPREDICT:
+             rrd->cdp_prep->scratch[CDP_hw_intercept].u_val = DNAN;
+             rrd->cdp_prep->scratch[CDP_hw_last_intercept].u_val = DNAN;
+             rrd->cdp_prep->scratch[CDP_hw_slope].u_val = DNAN;
+             rrd->cdp_prep->scratch[CDP_hw_last_slope].u_val = DNAN;
+             rrd->cdp_prep->scratch[CDP_null_count].u_cnt = 1; 
+             rrd->cdp_prep->scratch[CDP_last_null_count].u_cnt = 1;
+			 break;
+		  case CF_SEASONAL:
+		  case CF_DEVSEASONAL:
+             rrd->cdp_prep->scratch[CDP_hw_seasonal].u_val = DNAN;
+             rrd->cdp_prep->scratch[CDP_hw_last_seasonal].u_val = DNAN;
+             rrd->cdp_prep->scratch[CDP_init_seasonal].u_cnt = 1; 
+			 break;
+		  case CF_FAILURES:
+             /* initialize violation history to 0 */
+			 for (ii = 0; ii < MAX_CDP_PAR_EN; ii++)
+			 {
+				/* We can zero everything out, by setting u_val to the
+				 * NULL address. Each array entry in scratch is 8 bytes
+				 * (a double), but u_cnt only accessed 4 bytes (long) */
+				rrd->cdp_prep->scratch[ii].u_val = 0.0;
+			 }
+			 break;
+		  default:
+             /* can not be zero because we don't know anything ... */
+             rrd->cdp_prep->scratch[CDP_val].u_val = DNAN;
+	         /* startup missing pdp count */
+	         rrd->cdp_prep->scratch[CDP_unkn_pdp_cnt].u_cnt = 
+	         ((rrd->live_head->last_up -
+	         rrd->pdp_prep->scratch[PDP_unkn_sec_cnt].u_cnt)
+	         % (rrd->stat_head->pdp_step 
+	         * rrd->rra_def[i].pdp_cnt)) / rrd->stat_head->pdp_step;	
+		     break;
+	   }
+
+	   for(ii=0; ii < rrd->stat_head->ds_cnt; ii++) 
+	   {
+	      fwrite( rrd->cdp_prep,sizeof(cdp_prep_t),1,rrd_file);
+	   }
     }
 
     /* now, we must make sure that the rest of the rrd
@@ -292,13 +582,16 @@ rrd_create_fn(char *file_name, rrd_t *rrd)
 	fclose(rrd_file);
 	return(-1);
     }
-
-    rrd->rra_ptr->cur_row = 0;
-    for(i=0; i <rrd->stat_head->rra_cnt; i++)
-	fwrite( rrd->rra_ptr,
-		sizeof(rra_ptr_t), 1,rrd_file);
-
-
+ 
+	/* changed this initialization to be consistent with
+	 * rrd_restore. With the old value (0), the first update
+	 * would occur for cur_row = 1 because rrd_update increments
+	 * the pointer a priori. */
+    for (i=0; i < rrd->stat_head->rra_cnt; i++)
+	{
+	   rrd->rra_ptr->cur_row = rrd->rra_def[i].row_cnt - 1;
+	   fwrite( rrd->rra_ptr, sizeof(rra_ptr_t),1,rrd_file);
+	}
 
     /* write the empty data area */
     for(i=0; 
diff --git a/src/rrd_dump.c b/src/rrd_dump.c
index 95b5c52..a25fbe9 100644
--- a/src/rrd_dump.c
+++ b/src/rrd_dump.c
@@ -5,8 +5,15 @@
  *****************************************************************************
  * $Id$
  * $Log$
- * Revision 1.1  2001/02/25 22:25:05  oetiker
- * Initial revision
+ * Revision 1.2  2001/03/04 13:01:55  oetiker
+ * Aberrant Behavior Detection support. A brief overview added to rrdtool.pod.
+ * Major updates to rrd_update.c, rrd_create.c. Minor update to other core files.
+ * This is backwards compatible! But new files using the Aberrant stuff are not readable
+ * by old rrdtool versions. See http://cricket.sourceforge.net/aberrant/rrd_hw.htm
+ * -- Jake Brutlag <jakeb@corp.webtv.net>
+ *
+ * Revision 1.1.1.1  2001/02/25 22:25:05  oetiker
+ * checkin
  *
  *****************************************************************************/
 
@@ -24,14 +31,15 @@ rrd_dump(int argc, char **argv)
     long         rra_base, rra_start, rra_next;
     FILE                  *in_file;
     rrd_t             rrd;
-
+    unival value;
 
     if(rrd_open(argv[1],&in_file,&rrd, RRD_READONLY)==-1){
 	return(-1);
     }
+
     puts("<!-- Round Robin Database Dump -->");
     puts("<rrd>");
-    printf("\t<version> %s </version>\n",rrd.stat_head->version);
+    printf("\t<version> %s </version>\n",RRD_VERSION);
     printf("\t<step> %lu </step> <!-- Seconds -->\n",rrd.stat_head->pdp_step);
 #if HAVE_STRFTIME
     strftime(somestring,200,"%Y-%m-%d %H:%M:%S %Z",
@@ -83,22 +91,56 @@ rrd_dump(int argc, char **argv)
                       * sizeof(rrd_value_t));
 	printf("\t<rra>\n");
 	printf("\t\t<cf> %s </cf>\n",rrd.rra_def[i].cf_nam);
-	printf("\t\t<pdp_per_row> %lu </pdp_per_row> <!-- %lu seconds -->\n",
+	printf("\t\t<pdp_per_row> %lu </pdp_per_row> <!-- %lu seconds -->\n\n",
 	       rrd.rra_def[i].pdp_cnt, rrd.rra_def[i].pdp_cnt
 	       *rrd.stat_head->pdp_step);
-	printf("\t\t<xff> %0.10e </xff>\n\n",rrd.rra_def[i].par[RRA_cdp_xff_val].u_val);
+	/* added support for RRA parameters */
+	printf("\t\t<params>");
+	for (ii = 0; ii < MAX_RRA_PAR_EN; ii++)
+	{
+	   value = rrd.rra_def[i].par[ii];
+	   if (ii == RRA_dependent_rra_idx ||
+		   ii == RRA_seasonal_smooth_idx ||
+		   ii == RRA_failure_threshold)
+	          printf("<value> %lu </value>", value.u_cnt);
+	   else {
+	          if (isnan(value.u_val)) {
+	              printf("<value> NaN </value>");
+	          } else {
+	              printf("<value> %0.10e </value>", value.u_val);
+	          }
+	   }
+	}
+	printf("\t\t</params>\n");
 	printf("\t\t<cdp_prep>\n");
 	for(ii=0;ii<rrd.stat_head->ds_cnt;ii++){
-	    double value = rrd.cdp_prep[i*rrd.stat_head->ds_cnt+ii].scratch[CDP_val].u_val;
-	    printf("\t\t\t<ds>");	
-	    if (isnan(value)){
-	      printf("<value> NaN </value>");
-	    } else {
-	      printf("<value> %0.10e </value>", value);
-	    }
-	    printf("  <unknown_datapoints> %lu </unknown_datapoints>",
-                   rrd.cdp_prep[i*rrd.stat_head->ds_cnt+ii].scratch[CDP_unkn_pdp_cnt].u_cnt);
-	   printf("</ds>\n");	 
+#if 0
+	        double value = rrd.cdp_prep[i*rrd.stat_head->ds_cnt+ii].scratch[CDP_val].u_val;
+#endif 
+		printf("\t\t\t<ds>");
+		/* added support for exporting all CDP parameters */
+		for (iii=0; iii < MAX_CDP_PAR_EN ; iii++)
+		{
+		   value = rrd.cdp_prep[i*rrd.stat_head->ds_cnt + ii].scratch[iii];
+           /* handle integer values as a special case */
+		   if (cf_conv(rrd.rra_def[i].cf_nam) == CF_FAILURES ||
+			   iii == CDP_unkn_pdp_cnt || 
+		       iii == CDP_null_count ||
+		       iii == CDP_last_null_count)
+		     printf("<value> %lu </value>", value.u_cnt);
+		   else {	
+		     if (isnan(value.u_val)) {
+		       printf("<value> NaN </value>");
+		     } else {
+		       printf("<value> %0.10e </value>", value.u_val);
+		     }
+		   }
+		}
+#if 0 
+		printf("  <unknown_datapoints> %lu </unknown_datapoints>",
+		       rrd.cdp_prep[i*rrd.stat_head->ds_cnt+ii].scratch[CDP_unkn_pdp_cnt].u_cnt);
+#endif
+        printf("</ds>\n");	 
         }
 	printf("\t\t</cdp_prep>\n");
 
diff --git a/src/rrd_format.h b/src/rrd_format.h
index dde8bb9..3528362 100644
--- a/src/rrd_format.h
+++ b/src/rrd_format.h
@@ -19,7 +19,7 @@
  *****************************************************************************/
 
 #define RRD_COOKIE    "RRD"
-#define RRD_VERSION   "0001"
+#define RRD_VERSION   "0002"
 #define FLOAT_COOKIE  8.642135E130
 
 #if defined(WIN32)
@@ -143,7 +143,7 @@ enum ds_param_en {   DS_mrhb_cnt=0,       /* minimum required heartbeat. A
 					   * least every ds_mrhb seconds,
 					   * otherwise it is regarded dead and
 					   * will be set to UNKNOWN */             
-		     DS_min_val,	  /* the processed input of a ds must */
+		             DS_min_val,	  /* the processed input of a ds must */
                      DS_max_val };        /* be between max_val and min_val
 					   * both can be set to UNKNOWN if you
 					   * do not care. Data outside the limits
@@ -168,11 +168,67 @@ typedef struct ds_def_t {
 enum cf_en           { CF_AVERAGE=0,     /* data consolidation functions */ 
                        CF_MINIMUM, 
                        CF_MAXIMUM,
-                       CF_LAST};
-
-enum rra_par_en {   RRA_cdp_xff_val=0};   /* what part of the consolidated 
-					    datapoint may be unknown, while 
-					    still a valid entry in goes into the rra */
+                       CF_LAST,
+					   CF_HWPREDICT, 
+					   /* An array of predictions using the seasonal 
+						* Holt-Winters algorithm. Requires an RRA of type
+						* CF_SEASONAL for this data source. */
+					   CF_SEASONAL,
+					   /* An array of seasonal effects. Requires an RRA of
+						* type CF_HWPREDICT for this data source. */
+					   CF_DEVPREDICT,
+					   /* An array of deviation predictions based upon
+						* smoothed seasonal deviations. Requires an RRA of
+						* type CF_DEVSEASONAL for this data source. */
+					   CF_DEVSEASONAL,
+					   /* An array of smoothed seasonal deviations. Requires
+						* an RRA of type CF_HWPREDICT for this data source.
+						* */
+					   CF_FAILURES};
+					   /* A binary array of failure indicators: 1 indicates
+						* that the number of violations in the prescribed
+						* window exceeded the prescribed threshold. */
+
+#define MAX_RRA_PAR_EN 10
+enum rra_par_en {   RRA_cdp_xff_val=0,  /* what part of the consolidated
+                     * datapoint must be known, to produce a
+					 * valid entry in the rra */
+					RRA_hw_alpha,
+					/* exponential smoothing parameter for the intercept in
+					 * the Holt-Winters prediction algorithm. */
+					RRA_hw_beta,
+					/* exponential smoothing parameter for the slope in
+					 * the Holt-Winters prediction algorithm. */
+					RRA_dependent_rra_idx,
+					/* For CF_HWPREDICT: index of the RRA with the seasonal 
+					 * effects of the Holt-Winters algorithm (of type
+					 * CF_SEASONAL).
+					 * For CF_DEVPREDICT: index of the RRA with the seasonal
+					 * deviation predictions (of type CF_DEVSEASONAL).
+					 * For CF_SEASONAL: index of the RRA with the Holt-Winters
+					 * intercept and slope coefficient (of type CF_HWPREDICT).
+					 * For CF_DEVSEASONAL: index of the RRA with the 
+					 * Holt-Winters prediction (of type CF_HWPREDICT).
+					 * For CF_FAILURES: index of the CF_DEVSEASONAL array.
+					 * */
+					RRA_seasonal_smooth_idx,
+					/* For CF_SEASONAL and CF_DEVSEASONAL:
+					 * an integer between 0 and row_count - 1 which
+					 * is index in the seasonal cycle for applying
+					 * the period smoother. */
+				    RRA_failure_threshold,
+					/* For CF_FAILURES, number of violations within the last
+					 * window required to mark a failure. */
+                    RRA_seasonal_gamma = RRA_hw_alpha,
+					/* exponential smoothing parameter for seasonal effects.
+					 * */
+                    RRA_delta_pos = RRA_hw_alpha,
+                    RRA_delta_neg = RRA_hw_beta,
+					/* confidence bound scaling parameters for the
+					 * the FAILURES RRA. */
+                    RRA_window_len = RRA_seasonal_smooth_idx};
+					/* For CF_FAILURES, the length of the window for measuring
+					 * failures. */
 		   	
 #define CF_NAM_FMT    "%19[A-Z]"
 #define CF_NAM_SIZE   20
@@ -183,7 +239,7 @@ typedef struct rra_def_t {
     unsigned long    pdp_cnt;            /* how many primary data points are
 					  * required for a consolidated data
 					  * point?*/
-    unival           par[10];            /* index see rra_param_en */
+    unival           par[MAX_RRA_PAR_EN];            /* index see rra_param_en */
 
 } rra_def_t;
 
@@ -239,15 +295,56 @@ typedef struct pdp_prep_t{
 /****************************************************************************
  * POS 6: cdp_prep_t (* rra_cnt * ds_cnt )      data prep area for cdp values
  ****************************************************************************/
-enum cdp_par_en {  CDP_val=0,          /* the base_interval is always an
-					  * average */
-		   CDP_unkn_pdp_cnt };       /* how many unknown pdp were
-               				  * integrated. This and the cdp_xff
-					    will decide if this is going to
-					    be a UNKNOWN or a valid value */
+#define MAX_CDP_PAR_EN 10
+#define MAX_CDP_FAILURES_IDX 8 
+/* max CDP scratch entries avail to record violations for a FAILURES RRA */
+#define MAX_FAILURES_WINDOW_LEN 28
+enum cdp_par_en {  CDP_val=0,          
+                   /* the base_interval is always an
+					* average */
+		           CDP_unkn_pdp_cnt,       
+				   /* how many unknown pdp were
+               	    * integrated. This and the cdp_xff
+					* will decide if this is going to
+					* be a UNKNOWN or a valid value */
+				   CDP_hw_intercept,
+				   /* Current intercept coefficient for the Holt-Winters
+					* prediction algorithm. */
+				   CDP_hw_last_intercept,
+				   /* Last iteration intercept coefficient for the Holt-Winters
+					* prediction algorihtm. */
+				   CDP_hw_slope,
+				   /* Current slope coefficient for the Holt-Winters
+					* prediction algorithm. */
+				   CDP_hw_last_slope,
+				   /* Last iteration slope coeffient. */
+				   CDP_null_count,
+				   /* Number of sequential Unknown (DNAN) values + 1 preceding
+				    * the current prediction.
+					* */
+				   CDP_last_null_count,
+				   /* Last iteration count of Unknown (DNAN) values. */
+				   CDP_primary_val = 8,
+				   /* optimization for bulk updates: the value of the first CDP
+					* value to be written in the bulk update. */
+				   CDP_secondary_val = 9,
+				   /* optimization for bulk updates: the value of subsequent
+					* CDP values to be written in the bulk update. */
+                   CDP_hw_seasonal = CDP_hw_intercept,
+                   /* Current seasonal coefficient for the Holt-Winters
+                    * prediction algorithm. This is stored in CDP prep to avoid
+                    * redundant seek operations. */
+                   CDP_hw_last_seasonal = CDP_hw_last_intercept,
+                   /* Last iteration seasonal coeffient. */
+                   CDP_seasonal_deviation = CDP_hw_intercept,
+                   CDP_last_seasonal_deviation = CDP_hw_last_intercept,
+                   CDP_init_seasonal = CDP_null_count};
+                   /* init_seasonal is a flag which when > 0, forces smoothing updates
+                    * to occur when rra_ptr.cur_row == 0 */
 
 typedef struct cdp_prep_t{
-    unival         scratch[10];          /* contents according to cdp_par_en *
+    unival         scratch[MAX_CDP_PAR_EN];          
+										 /* contents according to cdp_par_en *
                                           * init state should be NAN */
 
 } cdp_prep_t;
@@ -283,7 +380,7 @@ typedef struct rrd_t {
  * Consolidated Data Points organized in Round Robin Archives.
  ****************************************************************************
  ****************************************************************************
- 
+
  *RRA 0
  (0,0) .................... ( ds_cnt -1 , 0)
  .
diff --git a/src/rrd_graph.c b/src/rrd_graph.c
index 136d3df..f42acaf 100644
--- a/src/rrd_graph.c
+++ b/src/rrd_graph.c
@@ -35,7 +35,7 @@ enum grc_en {GRC_CANVAS=0,GRC_BACK,GRC_SHADEA,GRC_SHADEB,
 
 
 enum gf_en {GF_PRINT=0,GF_GPRINT,GF_COMMENT,GF_HRULE,GF_VRULE,GF_LINE1,
-	    GF_LINE2,GF_LINE3,GF_AREA,GF_STACK, GF_DEF, GF_CDEF };
+	    GF_LINE2,GF_LINE3,GF_AREA,GF_STACK,GF_TICK,GF_DEF, GF_CDEF};
 
 enum op_en {OP_NUMBER=0,OP_VARIABLE,OP_INF,OP_PREV,OP_NEGINF,
 	    OP_UNKN,OP_NOW,OP_TIME,OP_LTIME,OP_ADD,OP_MOD,
@@ -346,6 +346,7 @@ enum gf_en gf_conv(char *string){
     conv_if(LINE3,GF_LINE3)
     conv_if(AREA,GF_AREA)
     conv_if(STACK,GF_STACK)
+	conv_if(TICK,GF_TICK)
     conv_if(DEF,GF_DEF)
     conv_if(CDEF,GF_CDEF)
     
@@ -687,12 +688,18 @@ reduce_data(
 		if (isnan(newval)) newval = srcptr[i*(*ds_cnt)+col];
 		else {
 		    switch (cf) {
+                        case CF_HWPREDICT:
+                        case CF_DEVSEASONAL:
+                        case CF_DEVPREDICT:
+                        case CF_SEASONAL:
 			case CF_AVERAGE:
 			    newval += srcptr[i*(*ds_cnt)+col];
 			    break;
 			case CF_MINIMUM:
 			    newval = min (newval,srcptr[i*(*ds_cnt)+col]);
 			    break;
+                        case CF_FAILURES: 
+                        /* an interval contains a failure if any subintervals contained a failure */
 			case CF_MAXIMUM:
 			    newval = max (newval,srcptr[i*(*ds_cnt)+col]);
 			    break;
@@ -704,11 +711,16 @@ reduce_data(
 	    }
 	    if (validval == 0){newval = DNAN;} else{
 		switch (cf) {
-		    case CF_AVERAGE:		
-			newval /= validval;
+                    case CF_HWPREDICT:
+                    case CF_DEVSEASONAL:
+                    case CF_DEVPREDICT:
+                    case CF_SEASONAL:
+                    case CF_AVERAGE:                
+                        newval /= validval;
 			break;
 		    case CF_MINIMUM:
-		    case CF_MAXIMUM:
+                    case CF_FAILURES:
+ 		    case CF_MAXIMUM:
 		    case CF_LAST:
 			break;
 		}
@@ -1389,6 +1401,7 @@ data_proc( image_desc_t *im ){
 	 (im->gdes[i].gf==GF_LINE2) ||
 	 (im->gdes[i].gf==GF_LINE3) ||
 	 (im->gdes[i].gf==GF_AREA) ||
+	 (im->gdes[i].gf==GF_TICK) ||
 	 (im->gdes[i].gf==GF_STACK)){
 	if((im->gdes[i].p_data = malloc((im->xsize +1)
 					* sizeof(rrd_value_t)))==NULL){
@@ -1411,6 +1424,7 @@ data_proc( image_desc_t *im ){
 	    case GF_LINE2:
 	    case GF_LINE3:
 	    case GF_AREA:
+		case GF_TICK:
 		paintval = 0.0;
 	    case GF_STACK:
 		vidx = im->gdes[ii].vidx;
@@ -1430,7 +1444,8 @@ data_proc( image_desc_t *im ){
 		if (! isnan(value)) {
 		  paintval += value;
 		  im->gdes[ii].p_data[i] = paintval;
-		  if (finite(paintval)){
+		  /* GF_TICK: the data values are not relevant for min and max */
+		  if (finite(paintval) && im->gdes[ii].gf != GF_TICK ){
   		   if (isnan(minval) || paintval <  minval)
 		     minval = paintval;
 		   if (isnan(maxval) || paintval >  maxval)
@@ -1669,6 +1684,10 @@ print_calc(image_desc_t *im, char ***prdata)
 		}
 
 		switch (im->gdes[i].cf){
+		case CF_HWPREDICT:
+		case CF_DEVPREDICT:
+		case CF_DEVSEASONAL:
+		case CF_SEASONAL:
 		case CF_AVERAGE:
 		    validsteps++;
 		    printval += im->gdes[vidx].data[ii];
@@ -1676,6 +1695,7 @@ print_calc(image_desc_t *im, char ***prdata)
 		case CF_MINIMUM:
 		    printval = min( printval, im->gdes[vidx].data[ii]);
 		    break;
+	    case CF_FAILURES:
 		case CF_MAXIMUM:
 		    printval = max( printval, im->gdes[vidx].data[ii]);
 		    break;
@@ -1683,7 +1703,7 @@ print_calc(image_desc_t *im, char ***prdata)
 		    printval = im->gdes[vidx].data[ii];
 		}
 	    }
-	    if (im->gdes[i].cf ==  CF_AVERAGE) {
+	    if (im->gdes[i].cf ==  CF_AVERAGE || im -> gdes[i].cf > CF_LAST) {
 		if (validsteps > 1) {
 		    printval = (printval / validsteps);
 		}
@@ -1738,6 +1758,7 @@ print_calc(image_desc_t *im, char ***prdata)
 	case GF_LINE2:
 	case GF_LINE3:
 	case GF_AREA:
+	case GF_TICK:
 	case GF_STACK:
 	case GF_HRULE:
 	case GF_VRULE:
@@ -2423,7 +2444,7 @@ MkLineBrush(image_desc_t *im,long cosel, enum gf_en typsel){
 
 int lazy_check(image_desc_t *im){
     FILE *fd = NULL;
-    int size = 1;
+	int size = 1;
     struct stat  gifstat;
     
     if (im->lazy == 0) return 0; /* no lazy option */
@@ -2438,11 +2459,11 @@ int lazy_check(image_desc_t *im){
       return 0; /* the file does not exist */
     switch (im->imgformat) {
     case IF_GIF:
-	size = GifSize(fd,&(im->xgif),&(im->ygif));
-	break;
+	   size = GifSize(fd,&(im->xgif),&(im->ygif));
+	   break;
     case IF_PNG:
-	size = PngSize(fd,&(im->xgif),&(im->ygif));
-	break;
+	   size = PngSize(fd,&(im->xgif),&(im->ygif));
+	   break;
     }
     fclose(fd);
     return size;
@@ -2592,7 +2613,22 @@ graph_paint(image_desc_t *im, char ***calcpr)
 	case GF_COMMENT:
 	case GF_HRULE:
 	case GF_VRULE:
-	  break;
+		break;
+	case GF_TICK:
+		for (ii = 0; ii < im->xsize; ii++)
+		{
+		   if (!isnan(im->gdes[i].p_data[ii]) && 
+			   im->gdes[i].p_data[ii] > 0.0)
+		   { 
+			  /* generate a tick */
+			  gdImageLine(gif, im -> xorigin + ii, 
+				 im -> yorigin - (im -> gdes[i].yrule * im -> ysize),
+				 im -> xorigin + ii, 
+				 im -> yorigin,
+				 im -> gdes[i].col.i);
+		   }
+		}
+		break;
 	case GF_LINE1:
 	case GF_LINE2:
 	case GF_LINE3:
@@ -3181,6 +3217,48 @@ rrd_graph(int argc, char **argv, char ***prdata, int *xsize, int *ysize)
 		return -1;
 	    }
 	    break;
+	case GF_TICK:
+	    if((scancount=sscanf(
+		&argv[i][argstart],
+		"%29[^:#]#%2x%2x%2x:%lf:%n",
+		varname,
+		&col_red,
+		&col_green,
+		&col_blue,
+		&(im.gdes[im.gdes_c-1].yrule),
+		&strstart))>=1)
+		{
+		im.gdes[im.gdes_c-1].col.red = col_red;
+		im.gdes[im.gdes_c-1].col.green = col_green;
+		im.gdes[im.gdes_c-1].col.blue = col_blue;
+		if(strstart <= 0){
+		    im.gdes[im.gdes_c-1].legend[0] = '\0';
+		} else { 
+		    scan_for_col(&argv[i][argstart+strstart],FMT_LEG_LEN,im.gdes[im.gdes_c-1].legend);
+		}
+		if((im.gdes[im.gdes_c-1].vidx=find_var(&im,varname))==-1){
+		    im_free(&im);
+		    rrd_set_error("unknown variable '%s'",varname);
+		    return -1;
+		}
+		if (im.gdes[im.gdes_c-1].yrule <= 0.0 || im.gdes[im.gdes_c-1].yrule > 1.0)
+		{
+		    im_free(&im);
+		    rrd_set_error("Tick mark scaling factor out of range");
+		    return -1;
+		}
+		if (scancount < 4)
+		   im.gdes[im.gdes_c-1].col.red = -1;		
+	    if (scancount < 5) 
+		   /* default tick marks: 10% of the y-axis */
+		   im.gdes[im.gdes_c-1].yrule = 0.1;
+
+		} else {
+		   im_free(&im);
+		   rrd_set_error("can't parse '%s'",&argv[i][argstart]);
+		   return -1;
+		} /* endif sscanf */
+		break;
 	case GF_STACK:
 	    if(linepass == 0){
 		im_free(&im);
diff --git a/src/rrd_hw.c b/src/rrd_hw.c
new file mode 100644
index 0000000..85b4fd7
--- /dev/null
+++ b/src/rrd_hw.c
@@ -0,0 +1,714 @@
+/*****************************************************************************
+ * RRDtool 1.0.21  Copyright Tobias Oetiker, 1997 - 2000
+ *****************************************************************************
+ * rrd_hw.c
+ *****************************************************************************
+ * Initial version by Jake Brutlag, WebTV Networks, 5/1/00
+ *****************************************************************************/
+
+#include "rrd_tool.h"
+
+/* #define DEBUG */
+
+unsigned long MyMod(signed long val, unsigned long mod);
+
+int update_hwpredict(rrd_t *rrd, unsigned long cdp_idx, unsigned long rra_idx, 
+                 unsigned long ds_idx, unsigned short CDP_scratch_idx);
+int update_seasonal(rrd_t *rrd, unsigned long cdp_idx, unsigned long rra_idx, 
+				 unsigned long ds_idx, unsigned short CDP_scratch_idx, 
+				 rrd_value_t *seasonal_coef);
+int update_devpredict(rrd_t *rrd, unsigned long cdp_idx, 
+				  unsigned long rra_idx, unsigned long ds_idx, unsigned short CDP_scratch_idx);
+int update_devseasonal(rrd_t *rrd, unsigned long cdp_idx, unsigned long rra_idx, 
+	               unsigned long ds_idx, unsigned short CDP_scratch_idx, 
+				   rrd_value_t *seasonal_dev);
+int update_failures(rrd_t *rrd, unsigned long cdp_idx, unsigned long rra_idx, 
+				unsigned long ds_idx, unsigned short CDP_scratch_idx);
+
+int
+update_hwpredict(rrd_t *rrd, unsigned long cdp_idx, unsigned long rra_idx, 
+                 unsigned long ds_idx, unsigned short CDP_scratch_idx)
+{
+   rrd_value_t prediction, seasonal_coef;
+   unsigned long dependent_rra_idx, seasonal_cdp_idx;
+   unival *coefs = rrd -> cdp_prep[cdp_idx].scratch;
+   rra_def_t *current_rra = &(rrd -> rra_def[rra_idx]);
+
+   /* save coefficients from current prediction */
+   coefs[CDP_hw_last_intercept].u_val = coefs[CDP_hw_intercept].u_val;
+   coefs[CDP_hw_last_slope].u_val = coefs[CDP_hw_slope].u_val;
+   coefs[CDP_last_null_count].u_cnt = coefs[CDP_null_count].u_cnt;
+
+   /* retrieve the current seasonal coef */
+   dependent_rra_idx = current_rra -> par[RRA_dependent_rra_idx].u_cnt;
+   seasonal_cdp_idx = dependent_rra_idx*(rrd -> stat_head -> ds_cnt) + ds_idx;
+   if (dependent_rra_idx < rra_idx)
+	  seasonal_coef = rrd -> cdp_prep[seasonal_cdp_idx].scratch[CDP_hw_last_seasonal].u_val;
+   else
+	  seasonal_coef = rrd -> cdp_prep[seasonal_cdp_idx].scratch[CDP_hw_seasonal].u_val;
+   
+   /* compute the prediction */
+   if (isnan(coefs[CDP_hw_intercept].u_val) || isnan(coefs[CDP_hw_slope].u_val)
+      || isnan(seasonal_coef))
+   {
+     prediction = DNAN;
+      
+     /* bootstrap initialization of slope and intercept */
+     if (isnan(coefs[CDP_hw_intercept].u_val) &&
+	 !isnan(coefs[CDP_scratch_idx].u_val)) 
+     {
+#ifdef DEBUG
+       fprintf(stderr,"Initialization of slope/intercept\n");
+#endif
+       coefs[CDP_hw_intercept].u_val = coefs[CDP_scratch_idx].u_val;
+       coefs[CDP_hw_last_intercept].u_val = coefs[CDP_scratch_idx].u_val;
+       /* initialize the slope to 0 */
+       coefs[CDP_hw_slope].u_val = 0.0;
+       coefs[CDP_hw_last_slope].u_val = 0.0;
+       /* initialize null count to 1 */
+       coefs[CDP_null_count].u_cnt = 1;
+       coefs[CDP_last_null_count].u_cnt = 1;
+     }
+     /* if seasonal coefficient is NA, then don't update intercept, slope */
+   } else {
+     prediction = coefs[CDP_hw_intercept].u_val + 
+       (coefs[CDP_hw_slope].u_val)*(coefs[CDP_null_count].u_cnt)
+       + seasonal_coef;
+#ifdef DEBUG
+     fprintf(stderr,"computed prediction: %f\n",prediction);
+#endif
+     if (isnan(coefs[CDP_scratch_idx].u_val))
+     {
+       /* NA value, no updates of intercept, slope;
+	    * increment the null count */
+       (coefs[CDP_null_count].u_cnt)++;
+     } else {
+#ifdef DEBUG
+       fprintf(stderr,"Updating intercept, slope\n");
+#endif
+       /* update the intercept */
+       coefs[CDP_hw_intercept].u_val = (current_rra -> par[RRA_hw_alpha].u_val)*
+	 	(coefs[CDP_scratch_idx].u_val - seasonal_coef) +
+	 	(1 - current_rra -> par[RRA_hw_alpha].u_val)*(coefs[CDP_hw_intercept].u_val
+	 	+ (coefs[CDP_hw_slope].u_val)*(coefs[CDP_null_count].u_cnt));
+       /* update the slope */
+       coefs[CDP_hw_slope].u_val = (current_rra -> par[RRA_hw_beta].u_val)*
+	 	(coefs[CDP_hw_intercept].u_val - coefs[CDP_hw_last_intercept].u_val) +
+	 	(1 - current_rra -> par[RRA_hw_beta].u_val)*(coefs[CDP_hw_slope].u_val);
+       /* reset the null count */
+       coefs[CDP_null_count].u_cnt = 1;
+     }
+   }
+
+   /* store the prediction for writing */
+   coefs[CDP_scratch_idx].u_val = prediction;
+   return 0;
+}
+
+int
+lookup_seasonal(rrd_t *rrd, unsigned long rra_idx, unsigned long rra_start,
+				FILE *rrd_file, unsigned long offset, rrd_value_t **seasonal_coef)
+{
+   unsigned long pos_tmp;
+   /* rra_ptr[].cur_row points to the rra row to be written; this function
+	* reads cur_row + offset */
+   unsigned long row_idx = rrd -> rra_ptr[rra_idx].cur_row + offset;
+   /* handle wrap around */
+   if (row_idx >= rrd -> rra_def[rra_idx].row_cnt)
+     row_idx = row_idx % (rrd -> rra_def[rra_idx].row_cnt);
+
+   /* rra_start points to the appropriate rra block in the file */
+   /* compute the pointer to the appropriate location in the file */
+   pos_tmp = rra_start + (row_idx)*(rrd -> stat_head -> ds_cnt)*sizeof(rrd_value_t);
+
+   /* allocate memory if need be */
+   if (*seasonal_coef == NULL)
+	  *seasonal_coef = 
+	     (rrd_value_t *) malloc((rrd -> stat_head -> ds_cnt)*sizeof(rrd_value_t));
+   if (*seasonal_coef == NULL) {
+	  rrd_set_error("memory allocation failure: seasonal coef");
+	  return -1;
+   }
+
+   if (!fseek(rrd_file,pos_tmp,SEEK_SET))
+   {
+      if (fread(*seasonal_coef,sizeof(rrd_value_t),rrd->stat_head->ds_cnt,rrd_file)
+		  == rrd -> stat_head -> ds_cnt)
+	  {
+		 /* success! */
+         /* we can safely ignore the rule requiring a seek operation between read
+          * and write, because this read moves the file pointer to somewhere
+          * in the file other than the next write location.
+          * */
+		 return 0;
+	  } else {
+	     rrd_set_error("read operation failed in lookup_seasonal(): %lu\n",pos_tmp);
+	  }
+   } else {
+	  rrd_set_error("seek operation failed in lookup_seasonal(): %lu\n",pos_tmp);
+   }
+   
+   return -1;
+}
+
+int
+update_seasonal(rrd_t *rrd, unsigned long cdp_idx, unsigned long rra_idx, 
+				 unsigned long ds_idx, unsigned short CDP_scratch_idx, rrd_value_t *seasonal_coef)
+{
+/* TODO: extract common if subblocks in the wake of I/O optimization */
+   rrd_value_t intercept, seasonal;
+   rra_def_t *current_rra = &(rrd -> rra_def[rra_idx]);
+   rra_def_t *hw_rra = &(rrd -> rra_def[current_rra -> par[RRA_dependent_rra_idx].u_cnt]);
+   /* obtain cdp_prep index for HWPREDICT */
+   unsigned long hw_cdp_idx = (current_rra -> par[RRA_dependent_rra_idx].u_cnt)
+      * (rrd -> stat_head -> ds_cnt) + ds_idx;
+   unival *coefs = rrd -> cdp_prep[hw_cdp_idx].scratch;
+
+   /* update seasonal coefficient in cdp prep areas */
+   seasonal = rrd -> cdp_prep[cdp_idx].scratch[CDP_hw_seasonal].u_val;
+   rrd -> cdp_prep[cdp_idx].scratch[CDP_hw_last_seasonal].u_val = seasonal;
+   rrd -> cdp_prep[cdp_idx].scratch[CDP_hw_seasonal].u_val =
+	  seasonal_coef[ds_idx];
+
+   /* update seasonal value for disk */
+   if (current_rra -> par[RRA_dependent_rra_idx].u_cnt < rra_idx)
+	  /* associated HWPREDICT has already been updated */
+	  /* check for possible NA values */
+      if (isnan(rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val))
+	  {
+		 /* no update, store the old value unchanged,
+		  * doesn't matter if it is NA */
+	     rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val = seasonal;
+	  } else if (isnan(coefs[CDP_hw_last_intercept].u_val) 
+		     || isnan(coefs[CDP_hw_last_slope].u_val))
+	  {
+		 /* this should never happen, as HWPREDICT was already updated */
+		 rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val= DNAN;
+	  } else if (isnan(seasonal))
+	  {
+		 /* initialization: intercept is not currently being updated */
+#ifdef DEBUG
+		 fprintf(stderr,"Initialization of seasonal coef %lu\n",
+			rrd -> rra_ptr[rra_idx].cur_row);
+#endif
+		 rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val 
+			-= coefs[CDP_hw_last_intercept].u_val; 
+	  } else {
+	         intercept = coefs[CDP_hw_intercept].u_val;
+#ifdef DEBUG
+		 fprintf(stderr,
+			"Updating seasonal, params: gamma %f, new intercept %f, old seasonal %f\n",
+			current_rra -> par[RRA_seasonal_gamma].u_val,
+			intercept, seasonal);
+#endif
+		 rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val =
+			(current_rra -> par[RRA_seasonal_gamma].u_val)*
+			(rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val - intercept) +
+			(1 - current_rra -> par[RRA_seasonal_gamma].u_val)*seasonal;
+	  }
+   else {
+	  /* SEASONAL array is updated first, which means the new intercept
+	   * hasn't be computed; so we compute it here. */
+
+	  /* check for possible NA values */
+      if (isnan(rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val))
+	  {
+		 /* no update, simple store the old value unchanged,
+		  * doesn't matter if it is NA */
+		 rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val =  seasonal;
+	  } else if (isnan(coefs[CDP_hw_intercept].u_val) 
+		     || isnan(coefs[CDP_hw_slope].u_val))
+	  {
+		 /* Initialization of slope and intercept will occur.
+		  * force seasonal coefficient to 0. */
+		 rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val= 0.0;
+	  } else if (isnan(seasonal))
+	  {
+		 /* initialization: intercept will not be updated
+		  * CDP_hw_intercept = CDP_hw_last_intercept; just need to 
+		  * subtract this baseline value. */
+#ifdef DEBUG
+		 fprintf(stderr,"Initialization of seasonal coef %lu\n",
+			rrd -> rra_ptr[rra_idx].cur_row);
+#endif
+		 rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val -= coefs[CDP_hw_intercept].u_val; 
+	  } else {
+	         /* Note that we must get CDP_scratch_idx from SEASONAL array, as CDP_scratch_idx
+	          * for HWPREDICT array will be DNAN. */
+	     intercept = (hw_rra -> par[RRA_hw_alpha].u_val)*
+		    (rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val - seasonal)
+		    + (1 - hw_rra -> par[RRA_hw_alpha].u_val)*(coefs[CDP_hw_intercept].u_val
+		    + (coefs[CDP_hw_slope].u_val)*(coefs[CDP_null_count].u_cnt));
+		 rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val =
+		   (current_rra -> par[RRA_seasonal_gamma].u_val)*
+		   (rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val - intercept) +
+		   (1 - current_rra -> par[RRA_seasonal_gamma].u_val)*seasonal;
+	  }
+   }
+#ifdef DEBUG
+   fprintf(stderr,"seasonal coefficient set= %f\n",
+         rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val);
+#endif
+   return 0;
+}
+
+int
+update_devpredict(rrd_t *rrd, unsigned long cdp_idx, 
+				  unsigned long rra_idx, unsigned long ds_idx, unsigned short CDP_scratch_idx)
+{
+   /* there really isn't any "update" here; the only reason this information
+    * is stored separately from DEVSEASONAL is to preserve deviation predictions
+    * for a longer duration than one seasonal cycle. */
+   unsigned long seasonal_cdp_idx = (rrd -> rra_def[rra_idx].par[RRA_dependent_rra_idx].u_cnt)
+      * (rrd -> stat_head -> ds_cnt) + ds_idx;
+
+   if (rrd -> rra_def[rra_idx].par[RRA_dependent_rra_idx].u_cnt < rra_idx)
+   {
+	  /* associated DEVSEASONAL array already updated */
+	  rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val
+		 = rrd -> cdp_prep[seasonal_cdp_idx].scratch[CDP_last_seasonal_deviation].u_val;
+   } else {
+	  /* associated DEVSEASONAL not yet updated */
+	  rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val
+		 = rrd -> cdp_prep[seasonal_cdp_idx].scratch[CDP_seasonal_deviation].u_val;
+   }
+   return 0;
+}
+
+int
+update_devseasonal(rrd_t *rrd, unsigned long cdp_idx, unsigned long rra_idx, 
+	               unsigned long ds_idx, unsigned short CDP_scratch_idx, 
+				   rrd_value_t *seasonal_dev)
+{
+   rrd_value_t prediction = 0, seasonal_coef = DNAN;
+   rra_def_t *current_rra = &(rrd -> rra_def[rra_idx]);
+   /* obtain cdp_prep index for HWPREDICT */
+   unsigned long hw_rra_idx = current_rra -> par[RRA_dependent_rra_idx].u_cnt;
+   unsigned long hw_cdp_idx = hw_rra_idx * (rrd -> stat_head -> ds_cnt) + ds_idx;
+   unsigned long seasonal_cdp_idx;
+   unival *coefs = rrd -> cdp_prep[hw_cdp_idx].scratch;
+ 
+   rrd -> cdp_prep[cdp_idx].scratch[CDP_last_seasonal_deviation].u_val =
+	  rrd -> cdp_prep[cdp_idx].scratch[CDP_seasonal_deviation].u_val;
+   /* retrieve the next seasonal deviation value, could be NA */
+   rrd -> cdp_prep[cdp_idx].scratch[CDP_seasonal_deviation].u_val =
+	  seasonal_dev[ds_idx];
+
+   /* retrieve the current seasonal_coef (not to be confused with the
+	* current seasonal deviation). Could make this more readable by introducing
+	* some wrapper functions. */
+   seasonal_cdp_idx = (rrd -> rra_def[hw_rra_idx].par[RRA_dependent_rra_idx].u_cnt)
+	  *(rrd -> stat_head -> ds_cnt) + ds_idx;
+   if (rrd -> rra_def[hw_rra_idx].par[RRA_dependent_rra_idx].u_cnt < rra_idx)
+	  /* SEASONAL array already updated */
+	  seasonal_coef = rrd -> cdp_prep[seasonal_cdp_idx].scratch[CDP_hw_last_seasonal].u_val;
+   else
+	  /* SEASONAL array not yet updated */
+	  seasonal_coef = rrd -> cdp_prep[seasonal_cdp_idx].scratch[CDP_hw_seasonal].u_val;
+   
+   /* compute the abs value of the difference between the prediction and
+	* observed value */
+   if (hw_rra_idx < rra_idx)
+   {
+	  /* associated HWPREDICT has already been updated */
+	  if (isnan(coefs[CDP_hw_last_intercept].u_val) ||
+	      isnan(coefs[CDP_hw_last_slope].u_val) ||
+	      isnan(seasonal_coef))
+	  {
+		 /* one of the prediction values is uinitialized */
+		 rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val = DNAN;
+		 return 0;
+	  } else {
+	     prediction = coefs[CDP_hw_last_intercept].u_val + 
+		   (coefs[CDP_hw_last_slope].u_val)*(coefs[CDP_last_null_count].u_cnt)
+		   + seasonal_coef;
+	  }
+   } else {
+	  /* associated HWPREDICT has NOT been updated */
+	  if (isnan(coefs[CDP_hw_intercept].u_val) ||
+	      isnan(coefs[CDP_hw_slope].u_val) ||
+	      isnan(seasonal_coef))
+	  {
+		 /* one of the prediction values is uinitialized */
+		 rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val = DNAN;
+		 return 0;
+	  } else {
+	     prediction = coefs[CDP_hw_intercept].u_val + 
+		   (coefs[CDP_hw_slope].u_val)*(coefs[CDP_null_count].u_cnt) 
+		   + seasonal_coef;
+	  }
+   }
+
+   if (isnan(rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val))
+   {
+      /* no update, store existing value unchanged, doesn't
+	   * matter if it is NA */
+	  rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val =
+		 rrd -> cdp_prep[cdp_idx].scratch[CDP_last_seasonal_deviation].u_val;
+   } else if (isnan(rrd -> cdp_prep[cdp_idx].scratch[CDP_last_seasonal_deviation].u_val))
+   {
+	  /* initialization */
+#ifdef DEBUG
+	  fprintf(stderr,"Initialization of seasonal deviation\n");
+#endif
+	  rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val =
+	     abs(prediction - rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val);
+   } else {
+	  /* exponential smoothing update */
+	  rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val =
+	    (rrd -> rra_def[rra_idx].par[RRA_seasonal_gamma].u_val)*
+	    abs(prediction - rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val)
+	    + (1 -  rrd -> rra_def[rra_idx].par[RRA_seasonal_gamma].u_val)*
+	    (rrd -> cdp_prep[cdp_idx].scratch[CDP_last_seasonal_deviation].u_val);
+   }
+   return 0;
+}
+
+/* Check for a failure based on a threshold # of violations within the specified
+ * window. */
+int 
+update_failures(rrd_t *rrd, unsigned long cdp_idx, unsigned long rra_idx, 
+				unsigned long ds_idx, unsigned short CDP_scratch_idx)
+{
+   /* detection of a violation depends on 3 RRAs:
+	* HWPREDICT, SEASONAL, and DEVSEASONAL */
+   rra_def_t *current_rra = &(rrd -> rra_def[rra_idx]);
+   unsigned long dev_rra_idx = current_rra -> par[RRA_dependent_rra_idx].u_cnt;
+   rra_def_t *dev_rra = &(rrd -> rra_def[dev_rra_idx]);
+   unsigned long hw_rra_idx = dev_rra -> par[RRA_dependent_rra_idx].u_cnt;
+   rra_def_t *hw_rra =  &(rrd -> rra_def[hw_rra_idx]);
+   unsigned long seasonal_rra_idx = hw_rra -> par[RRA_dependent_rra_idx].u_cnt;
+   unsigned long temp_cdp_idx;
+   rrd_value_t deviation = DNAN;
+   rrd_value_t seasonal_coef = DNAN;
+   rrd_value_t prediction = DNAN;
+   char violation = 0; 
+   unsigned short violation_cnt = 0, i;
+   char *violations_array;
+
+   /* usual checks to determine the order of the RRAs */
+   temp_cdp_idx = dev_rra_idx * (rrd -> stat_head -> ds_cnt) + ds_idx;
+   if (rra_idx < seasonal_rra_idx)
+   {
+	  /* DEVSEASONAL not yet updated */
+	  deviation = rrd -> cdp_prep[temp_cdp_idx].scratch[CDP_seasonal_deviation].u_val;
+   } else {
+	  /* DEVSEASONAL already updated */
+	  deviation = rrd -> cdp_prep[temp_cdp_idx].scratch[CDP_last_seasonal_deviation].u_val;
+   }
+   if (!isnan(deviation)) {
+
+   temp_cdp_idx = seasonal_rra_idx * (rrd -> stat_head -> ds_cnt) + ds_idx;
+   if (rra_idx < seasonal_rra_idx)
+   {
+	  /* SEASONAL not yet updated */
+	  seasonal_coef = rrd -> cdp_prep[temp_cdp_idx].scratch[CDP_hw_seasonal].u_val;
+   } else {
+	  /* SEASONAL already updated */
+	  seasonal_coef = rrd -> cdp_prep[temp_cdp_idx].scratch[CDP_hw_last_seasonal].u_val;
+   }
+   /* in this code block, we know seasonal coef is not DNAN, because deviation is not
+	* null */
+
+   temp_cdp_idx = hw_rra_idx * (rrd -> stat_head -> ds_cnt) + ds_idx;
+   if (rra_idx < hw_rra_idx)
+   {
+	  /* HWPREDICT not yet updated */
+	  prediction = rrd -> cdp_prep[temp_cdp_idx].scratch[CDP_hw_intercept].u_val + 
+	     (rrd -> cdp_prep[temp_cdp_idx].scratch[CDP_hw_slope].u_val)
+		 *(rrd -> cdp_prep[temp_cdp_idx].scratch[CDP_null_count].u_cnt)
+		 + seasonal_coef;
+   } else {
+	  /* HWPREDICT already updated */
+	  prediction = rrd -> cdp_prep[temp_cdp_idx].scratch[CDP_hw_last_intercept].u_val + 
+	     (rrd -> cdp_prep[temp_cdp_idx].scratch[CDP_hw_last_slope].u_val)
+		 *(rrd -> cdp_prep[temp_cdp_idx].scratch[CDP_last_null_count].u_cnt)
+		 + seasonal_coef;
+   }
+
+   /* determine if the observed value is a violation */
+   if (!isnan(rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val))
+   {
+	  if (rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val > prediction + 
+		 (current_rra -> par[RRA_delta_pos].u_val)*deviation
+	     || rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val < prediction - 
+		 (current_rra -> par[RRA_delta_neg].u_val)*deviation)
+		 violation = 1;
+   } else {
+	  violation = 1; /* count DNAN values as violations */
+   }
+
+   }
+
+   /* determine if a failure has occurred and update the failure array */
+   violation_cnt = violation;
+   /* WARNING: this cast makes XML files non-portable across platforms,
+	* because an array of longs on disk is treated as an array of chars
+	* in memory. */
+   violations_array = (char *) ((void *) rrd -> cdp_prep[cdp_idx].scratch);
+   for (i = current_rra -> par[RRA_window_len].u_cnt; i > 1; i--)
+   {
+	  /* shift */
+	  violations_array[i-1] = violations_array[i-2]; 
+	  violation_cnt += violations_array[i-1];
+   }
+   violations_array[0] = violation;
+
+   if (violation_cnt < current_rra -> par[RRA_failure_threshold].u_cnt)
+	  /* not a failure */
+	  rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val = 0.0;
+   else
+	  rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val = 1.0;
+
+   return (rrd-> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val);
+}
+
+/* For the specified CDP prep area and the FAILURES RRA,
+ * erase all history of past violations.
+ */
+void
+erase_violations(rrd_t *rrd, unsigned long cdp_idx, unsigned long rra_idx)
+{
+   unsigned short i;
+   char *violations_array;
+   /* check that rra_idx is a CF_FAILURES array */
+   if (cf_conv(rrd -> rra_def[rra_idx].cf_nam) != CF_FAILURES)
+   {
+#ifdef DEBUG
+	  fprintf(stderr,"erase_violations called for non-FAILURES RRA: %s\n",
+	     rrd -> rra_def[rra_idx].cf);
+#endif
+      return;
+   }
+
+#ifdef DEBUG
+   fprintf(stderr,"scratch buffer before erase:\n");
+   for (i = 0; i < MAX_CDP_PAR_EN; i++)
+   {
+	  fprintf(stderr,"%lu ", rrd -> cdp_prep[cdp_idx].scratch[i].u_cnt);
+   }
+   fprintf(stderr,"\n");
+#endif
+
+   /* WARNING: this cast makes XML files non-portable across platforms,
+	* because an array of longs on disk is treated as an array of chars
+	* in memory. */
+   violations_array = (char *) ((void *) rrd -> cdp_prep[cdp_idx].scratch);
+   /* erase everything in the part of the CDP scratch array that will be
+	* used to store violations for the current window */
+   for (i = rrd -> rra_def[rra_idx].par[RRA_window_len].u_cnt; i > 0; i--)
+   {
+	  violations_array[i-1] = 0;
+   }
+#ifdef DEBUG
+   fprintf(stderr,"scratch buffer after erase:\n");
+   for (i = 0; i < MAX_CDP_PAR_EN; i++)
+   {
+	  fprintf(stderr,"%lu ", rrd -> cdp_prep[cdp_idx].scratch[i].u_cnt);
+   }
+   fprintf(stderr,"\n");
+#endif
+}
+
+/* Smooth a periodic array with a moving average: equal weights and
+ * length = 5% of the period. */
+int
+apply_smoother(rrd_t *rrd, unsigned long rra_idx, unsigned long rra_start,
+               FILE *rrd_file)
+{
+   unsigned long i, j, k;
+   unsigned long totalbytes;
+   rrd_value_t *rrd_values;
+   unsigned long row_length = rrd -> stat_head -> ds_cnt;
+   unsigned long row_count = rrd -> rra_def[rra_idx].row_cnt;
+   unsigned long offset;
+   FIFOqueue **buffers;
+   rrd_value_t *working_average;
+
+   offset = floor(0.025*row_count);
+   if (offset == 0) return 0; /* no smoothing */
+
+   /* allocate memory */
+   totalbytes = sizeof(rrd_value_t)*row_length*row_count;
+   rrd_values = (rrd_value_t *) malloc(totalbytes);
+   if (rrd_values == NULL)
+   {
+	  rrd_set_error("apply smoother: memory allocation failure");
+	  return -1;
+   }
+
+   /* rra_start is at the beginning of this rra */
+   if (fseek(rrd_file,rra_start,SEEK_SET))
+   {
+	  rrd_set_error("seek to rra %d failed", rra_start);
+	  free(rrd_values);
+	  return -1;
+   }
+   fflush(rrd_file);
+   /* could read all data in a single block, but we need to
+    * check for NA values */
+   for (i = 0; i < row_count; ++i)
+   {
+	  for (j = 0; j < row_length; ++j)
+	  {
+		 fread(&(rrd_values[i*row_length + j]),sizeof(rrd_value_t),1,rrd_file);
+		 /* should check fread for errors... */
+		 if (isnan(rrd_values[i*row_length + j])) {
+			/* can't apply smoothing, still uninitialized values */
+#ifdef DEBUG
+			fprintf(stderr,"apply_smoother: NA detected in seasonal array: %ld %ld\n",i,j);
+#endif
+			free(rrd_values);
+			return 0;
+		 }
+	  }
+   }
+
+   /* allocate queues, one for each data source */
+   buffers = (FIFOqueue **) malloc(sizeof(FIFOqueue *)*row_length);
+   for (i = 0; i < row_length; ++i)
+   {
+      queue_alloc(&(buffers[i]),2*offset + 1);
+   }
+   /* need working average initialized to 0 */
+   working_average = (rrd_value_t *) calloc(row_length,sizeof(rrd_value_t));
+
+   /* compute sums of the first 2*offset terms */ 
+   for (i = 0; i < 2*offset; ++i)
+   {
+	  k = MyMod(i - offset,row_count);
+	  for (j = 0; j < row_length; ++j)
+	  {
+		 queue_push(buffers[j],rrd_values[k*row_length + j]);
+		 working_average[j] += rrd_values[k*row_length + j];
+	  }
+   }
+
+   /* compute moving averages */
+   for (i = offset; i < row_count + offset; ++i)
+   {
+	  for (j = 0; j < row_length; ++j)
+	  {
+	     k = MyMod(i,row_count);
+	     /* add a term to the sum */
+	     working_average[j] += rrd_values[k*row_length + j];
+	     queue_push(buffers[j],rrd_values[k*row_length + j]);
+
+	     /* reset k to be the center of the window */
+	     k = MyMod(i - offset,row_count);
+	     /* overwrite rdd_values entry, the old value is already
+	      * saved in buffers */
+	     rrd_values[k*row_length + j] = working_average[j]/(2*offset + 1);
+
+	     /* remove a term from the sum */
+	     working_average[j] -= queue_pop(buffers[j]);
+	  }	
+   } 
+
+   for (i = 0; i < row_length; ++i)
+   {
+	  queue_dealloc(buffers[i]);
+   }
+   free(buffers);
+   free(working_average);
+
+   /* flush updated values to disk */
+   fflush(rrd_file);
+   if (fseek(rrd_file,rra_start,SEEK_SET))
+   {
+	  rrd_set_error("apply_smoother: seek to pos %d failed", rra_start);
+	  free(rrd_values);
+	  return -1;
+   }
+   /* write as a single block */
+   if (fwrite(rrd_values,sizeof(rrd_value_t),row_length*row_count,rrd_file)
+       != row_length*row_count)
+   {
+	  rrd_set_error("apply_smoother: write failed to %lu",rra_start);
+	  free(rrd_values);
+	  return -1;
+   }
+
+   fflush(rrd_file);
+   free(rrd_values);
+   return 0;
+}
+
+int
+update_aberrant_CF(rrd_t *rrd, rrd_value_t pdp_val, enum cf_en current_cf, 
+		  unsigned long cdp_idx, unsigned long rra_idx, unsigned long ds_idx, 
+		  unsigned short CDP_scratch_idx, rrd_value_t *seasonal_coef)
+{
+   rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val = pdp_val;
+   switch (current_cf) {
+   case CF_AVERAGE:
+   default:
+	 return 0;
+   case CF_HWPREDICT:
+     return update_hwpredict(rrd,cdp_idx,rra_idx,ds_idx,CDP_scratch_idx);
+   case CF_DEVPREDICT:
+	 return update_devpredict(rrd,cdp_idx,rra_idx,ds_idx,CDP_scratch_idx);
+   case CF_SEASONAL:
+     return update_seasonal(rrd,cdp_idx,rra_idx,ds_idx,CDP_scratch_idx,seasonal_coef);
+   case CF_DEVSEASONAL:
+     return update_devseasonal(rrd,cdp_idx,rra_idx,ds_idx,CDP_scratch_idx,seasonal_coef);
+   case CF_FAILURES:
+     return update_failures(rrd,cdp_idx,rra_idx,ds_idx,CDP_scratch_idx);
+   }
+   return -1;
+}
+
+unsigned long MyMod(signed long val, unsigned long mod)
+{
+   unsigned long new_val;
+   if (val < 0)
+     new_val = ((unsigned long) abs(val)) % mod;
+   else
+     new_val = (val % mod);
+   
+   if (val < 0) 
+     return (mod - new_val);
+   else
+     return (new_val);
+}
+
+/* a standard fixed-capacity FIF0 queue implementation
+ * No overflow checking is performed. */
+int queue_alloc(FIFOqueue **q,int capacity)
+{
+   *q = (FIFOqueue *) malloc(sizeof(FIFOqueue));
+   if (*q == NULL) return -1;
+   (*q) -> queue = (rrd_value_t *) malloc(sizeof(rrd_value_t)*capacity);
+   if ((*q) -> queue == NULL)
+   {
+	  free(*q);
+	  return -1;
+   }
+   (*q) -> capacity = capacity;
+   (*q) -> head = capacity;
+   (*q) -> tail = 0;
+   return 0;
+}
+
+int queue_isempty(FIFOqueue *q)
+{
+   return (q -> head % q -> capacity == q -> tail);
+}
+
+void queue_push(FIFOqueue *q, rrd_value_t value)
+{
+   q -> queue[(q -> tail)++] = value;
+   q -> tail = q -> tail % q -> capacity;
+}
+
+rrd_value_t queue_pop(FIFOqueue *q)
+{
+   q -> head = q -> head % q -> capacity;
+   return q -> queue[(q -> head)++];
+}
+
+void queue_dealloc(FIFOqueue *q)
+{
+   free(q -> queue);
+   free(q);
+}
diff --git a/src/rrd_info.c b/src/rrd_info.c
index 37c43db..9f030e7 100644
--- a/src/rrd_info.c
+++ b/src/rrd_info.c
@@ -67,6 +67,7 @@ rrd_info(int argc, char **argv) {
     rrd_t        rrd;
     info_t       *data,*cd;
     infoval      info;
+	enum cf_en   current_cf;
 
     if(rrd_open(argv[1],&in_file,&rrd, RRD_READONLY)==-1){
 	return(NULL);
@@ -113,6 +114,7 @@ rrd_info(int argc, char **argv) {
     for(i=0;i<rrd.stat_head->rra_cnt;i++){
 	info.u_str=rrd.rra_def[i].cf_nam;
 	cd=push(cd,sprintf_alloc("rra[%d].cf",         i),  RD_I_STR,   info);
+	current_cf = cf_conv(rrd.rra_def[i].cf_nam);
 
 	info.u_cnt=rrd.rra_def[i].row_cnt;
 	cd=push(cd,sprintf_alloc("rra[%d].rows",i),  RD_I_CNT,   info);
@@ -120,23 +122,69 @@ rrd_info(int argc, char **argv) {
 	info.u_cnt=rrd.rra_def[i].pdp_cnt;
 	cd=push(cd,sprintf_alloc("rra[%d].pdp_per_row",i),  RD_I_CNT,   info);
 
-        info.u_val=rrd.rra_def[i].par[RRA_cdp_xff_val].u_val;
-        cd=push(cd,sprintf_alloc("rra[%d].xff",i),  RD_I_VAL,   info);
-        
+	switch(current_cf)
+	{
+	   case CF_HWPREDICT:
+		  info.u_val=rrd.rra_def[i].par[RRA_hw_alpha].u_val;
+		  cd=push(cd,sprintf_alloc("rra[%d].alpha",i),RD_I_VAL,info);
+		  info.u_val=rrd.rra_def[i].par[RRA_hw_beta].u_val;
+		  cd=push(cd,sprintf_alloc("rra[%d].beta",i),RD_I_VAL,info);
+		  break;
+	   case CF_SEASONAL:
+	   case CF_DEVSEASONAL:
+		  info.u_val=rrd.rra_def[i].par[RRA_seasonal_gamma].u_val;
+		  cd=push(cd,sprintf_alloc("rra[%d].gamma",i),RD_I_VAL,info);
+		  break;
+	   case CF_FAILURES:
+		  info.u_val=rrd.rra_def[i].par[RRA_delta_pos].u_val;
+		  cd=push(cd,sprintf_alloc("rra[%d].delta_pos",i),RD_I_VAL,info);
+		  info.u_val=rrd.rra_def[i].par[RRA_delta_neg].u_val;
+		  cd=push(cd,sprintf_alloc("rra[%d].delta_neg",i),RD_I_VAL,info);
+		  info.u_cnt=rrd.rra_def[i].par[RRA_failure_threshold].u_cnt;
+		  cd=push(cd,sprintf_alloc("rra[%d].failure_threshold",i),RD_I_CNT,info);
+		  info.u_cnt=rrd.rra_def[i].par[RRA_window_len].u_cnt;
+		  cd=push(cd,sprintf_alloc("rra[%d].window_length",i),RD_I_CNT,info);
+		  break;
+	   case CF_DEVPREDICT:
+		  break;
+	   default:
+		  info.u_val=rrd.rra_def[i].par[RRA_cdp_xff_val].u_val;
+		  cd=push(cd,sprintf_alloc("rra[%d].xff",i),RD_I_VAL,info);
+		  break;
+	}
+
 	for(ii=0;ii<rrd.stat_head->ds_cnt;ii++){
 	    info.u_val=rrd.cdp_prep[i*rrd.stat_head->ds_cnt+ii].scratch[CDP_val].u_val;
 	    cd=push(cd,sprintf_alloc("rra[%d].cdp_prep[%d].value",i,ii), RD_I_VAL, info);
-
+        switch(current_cf)
+		{
+		case CF_HWPREDICT:
+	    info.u_val=rrd.cdp_prep[i*rrd.stat_head->ds_cnt+ii].scratch[CDP_hw_intercept].u_val;
+	    cd=push(cd,sprintf_alloc("rra[%d].cdp_prep[%d].intercept",i,ii), RD_I_VAL, info);
+	    info.u_val=rrd.cdp_prep[i*rrd.stat_head->ds_cnt+ii].scratch[CDP_hw_slope].u_val;
+	    cd=push(cd,sprintf_alloc("rra[%d].cdp_prep[%d].slope",i,ii), RD_I_VAL, info);
+	    info.u_cnt=rrd.cdp_prep[i*rrd.stat_head->ds_cnt+ii].scratch[CDP_null_count].u_cnt;
+	    cd=push(cd,sprintf_alloc("rra[%d].cdp_prep[%d].NaN_count",i,ii), RD_I_CNT, info);
+		   break;
+		case CF_SEASONAL:
+	    info.u_val=rrd.cdp_prep[i*rrd.stat_head->ds_cnt+ii].scratch[CDP_hw_seasonal].u_val;
+	    cd=push(cd,sprintf_alloc("rra[%d].cdp_prep[%d].seasonal",i,ii), RD_I_VAL, info);
+		   break;
+		case CF_DEVSEASONAL:
+	    info.u_val=rrd.cdp_prep[i*rrd.stat_head->ds_cnt+ii].scratch[CDP_seasonal_deviation].u_val;
+	    cd=push(cd,sprintf_alloc("rra[%d].cdp_prep[%d].deviation",i,ii), RD_I_VAL, info);
+		   break;
+		case CF_DEVPREDICT:
+		case CF_FAILURES:
+		   break;
+		default:
 	    info.u_cnt=rrd.cdp_prep[i*rrd.stat_head->ds_cnt+ii].scratch[CDP_unkn_pdp_cnt].u_cnt;
 	    cd=push(cd,sprintf_alloc("rra[%d].cdp_prep[%d].unknown_datapoints",i,ii), RD_I_CNT, info);
+		   break;
         }
     }
-    rrd_free(&rrd);
+	}
+	rrd_free(&rrd);
     return(data);
 
 }
-
-
-
-
-
diff --git a/src/rrd_open.c b/src/rrd_open.c
index 1a3496d..7b546f5 100644
--- a/src/rrd_open.c
+++ b/src/rrd_open.c
@@ -5,6 +5,13 @@
  *****************************************************************************
  * $Id$
  * $Log$
+ * Revision 1.3  2001/03/04 13:01:55  oetiker
+ * Aberrant Behavior Detection support. A brief overview added to rrdtool.pod.
+ * Major updates to rrd_update.c, rrd_create.c. Minor update to other core files.
+ * This is backwards compatible! But new files using the Aberrant stuff are not readable
+ * by old rrdtool versions. See http://cricket.sourceforge.net/aberrant/rrd_hw.htm
+ * -- Jake Brutlag <jakeb@corp.webtv.net>
+ *
  * Revision 1.2  2001/03/04 10:29:20  oetiker
  * fixed filedescriptor leak
  * -- Mike Franusich <mike@franusich.com>
@@ -45,6 +52,15 @@ rrd_open(char *file_name, FILE **in_file, rrd_t *rrd, int rdwr)
 	rrd_set_error("opening '%s': %s",file_name, strerror(errno));
 	return (-1);
     }
+/*
+	if (rdwr == RRD_READWRITE)
+	{
+	   if (setvbuf((*in_file),NULL,_IONBF,2)) {
+		  rrd_set_error("failed to disable the stream buffer\n");
+		  return (-1);
+	   }
+	}
+*/
     
 #define MYFREAD(MYVAR,MYVART,MYCNT) \
     if ((MYVAR = malloc(sizeof(MYVART) * MYCNT)) == NULL) {\
@@ -63,7 +79,7 @@ rrd_open(char *file_name, FILE **in_file, rrd_t *rrd, int rdwr)
 	    fclose(*in_file);
 	    return(-1);}
 
-	if (strncmp(rrd->stat_head->version,RRD_VERSION,5) != 0){
+        if (atoi(rrd->stat_head->version) > atoi(RRD_VERSION)){
 	    rrd_set_error("can't handle RRD file version %s",
 			rrd->stat_head->version);
 	    free(rrd->stat_head);
diff --git a/src/rrd_restore.c b/src/rrd_restore.c
index 4e2b6c3..f1d1fad 100644
--- a/src/rrd_restore.c
+++ b/src/rrd_restore.c
@@ -99,7 +99,7 @@ int read_tag(char **buf, char *tag, char *format, void *value){
 int xml2rrd(char* buf, rrd_t* rrd, char rc){
   /* pass 1 identify number of RRAs  */
   char *ptr,*ptr2,*ptr3; /* walks thought the buffer */
-  long rows=0,mempool=0,i=0;
+  long rows=0,mempool=0,i=0,ii;
   xml_lc(buf); /* lets lowercase all active parts of the xml */
   ptr=buf;
   ptr2=buf;
@@ -115,6 +115,14 @@ int xml2rrd(char* buf, rrd_t* rrd, char rc){
 
   strcpy(rrd->stat_head->cookie,RRD_COOKIE);
   read_tag(&ptr,"version","%4[0-9]",rrd->stat_head->version);
+  /* added primitive version checking */
+  if (atoi(rrd -> stat_head -> version) < 2)
+  {
+    rrd_set_error("Incompatible file version, detected version %s, required version %s\n",
+		  rrd -> stat_head -> version, RRD_VERSION);
+    free(rrd -> stat_head);
+    return -1;
+  }
   rrd->stat_head->float_cookie = FLOAT_COOKIE;
   rrd->stat_head->ds_cnt = 0;
   rrd->stat_head->rra_cnt = 0;
@@ -187,21 +195,47 @@ int xml2rrd(char* buf, rrd_t* rrd, char rc){
       if(cf_conv(rrd->rra_def[rrd->stat_head->rra_cnt-1].cf_nam) == -1) return -1;
 
       read_tag(&ptr2,"pdp_per_row","%lu",&(rrd->rra_def[rrd->stat_head->rra_cnt-1].pdp_cnt));
-      read_tag(&ptr2,"xff","%lf",&(rrd->rra_def[rrd->stat_head->rra_cnt-1].par[RRA_cdp_xff_val].u_val));
-      if(rrd->rra_def[rrd->stat_head->rra_cnt-1].par[RRA_cdp_xff_val].u_val > 1 ||
-         rrd->rra_def[rrd->stat_head->rra_cnt-1].par[RRA_cdp_xff_val].u_val < 0)
-          return -1;
-      
+      /* add support to read RRA parameters */
+      eat_tag(&ptr2, "params");
+      for (i = 0; i < MAX_RRA_PAR_EN; i++)
+      {
+	if (i == RRA_dependent_rra_idx || 
+		i == RRA_seasonal_smooth_idx ||
+		i == RRA_failure_threshold)
+	  read_tag(&ptr2, "value","%u",
+		   &(rrd->rra_def[rrd->stat_head->rra_cnt-1].par[i].u_cnt));
+	else
+	  read_tag(&ptr2, "value","%lf",
+		   &(rrd->rra_def[rrd->stat_head->rra_cnt-1].par[i].u_val));
+      }
+      eat_tag(&ptr2, "/params");
       eat_tag(&ptr2,"cdp_prep");
-      for(i=0;i<rrd->stat_head->ds_cnt;i++){
-	   eat_tag(&ptr2,"ds");
-	   read_tag(&ptr2,"value","%lf",&(rrd->cdp_prep[rrd->stat_head->ds_cnt*
-						       (rrd->stat_head->rra_cnt-1)
-						       +i].scratch[CDP_val].u_val));
-	   read_tag(&ptr2,"unknown_datapoints","%lu",&(rrd->cdp_prep[rrd->stat_head->ds_cnt
-						      *(rrd->stat_head->rra_cnt-1)
-						      +i].scratch[CDP_unkn_pdp_cnt].u_cnt));
-	   eat_tag(&ptr2,"/ds");
+      for(i=0;i<rrd->stat_head->ds_cnt;i++)
+      {
+	eat_tag(&ptr2,"ds");
+	/* add suport to read CDP parameters */
+	for (ii = 0; ii < MAX_CDP_PAR_EN; ii++)
+	{
+      /* handle integer values as a special case */
+      if (cf_conv(rrd->rra_def[rrd->stat_head->rra_cnt-1].cf_nam) == CF_FAILURES ||
+		  ii == CDP_unkn_pdp_cnt || 
+		  ii == CDP_null_count ||
+	      ii == CDP_last_null_count)
+	  {
+	    read_tag(&ptr2,"value","%lu",
+		     &(rrd->cdp_prep[rrd->stat_head->ds_cnt*(rrd->stat_head->rra_cnt-1)
+		     +i].scratch[ii].u_cnt));
+	  } else {
+	    read_tag(&ptr2,"value","%lf",&(rrd->cdp_prep[rrd->stat_head->ds_cnt*
+		     (rrd->stat_head->rra_cnt-1) +i].scratch[ii].u_val));
+	  }
+
+#if 0
+	  read_tag(&ptr2,"unknown_datapoints","%lu",&(rrd->cdp_prep[rrd->stat_head->ds_cnt
+		   *(rrd->stat_head->rra_cnt-1) +i].scratch[CDP_unkn_pdp_cnt].u_cnt));
+#endif
+	} /* end for */
+	eat_tag(&ptr2,"/ds");
       }
       eat_tag(&ptr2,"/cdp_prep");
       rrd->rra_def[rrd->stat_head->rra_cnt-1].row_cnt=0;
@@ -257,6 +291,9 @@ int xml2rrd(char* buf, rrd_t* rrd, char rc){
   }
 
   for(i=0; i <rrd->stat_head->rra_cnt; i++) {
+	  /* last row in the xml file is the most recent; as
+	   * rrd_update increments the current row pointer, set cur_row
+	   * here to the last row. */
       rrd->rra_ptr[i].cur_row = rrd->rra_def[i].row_cnt-1;
   }
   if (ptr==NULL)
@@ -379,3 +416,20 @@ rrd_restore(int argc, char **argv)
     rrd_free(&rrd);    
     return 0;
 }
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/src/rrd_tool.h b/src/rrd_tool.h
index 789f072..2795665 100644
--- a/src/rrd_tool.h
+++ b/src/rrd_tool.h
@@ -5,8 +5,15 @@
  *****************************************************************************
  * $Id$
  * $Log$
- * Revision 1.1  2001/02/25 22:25:06  oetiker
- * Initial revision
+ * Revision 1.2  2001/03/04 13:01:55  oetiker
+ * Aberrant Behavior Detection support. A brief overview added to rrdtool.pod.
+ * Major updates to rrd_update.c, rrd_create.c. Minor update to other core files.
+ * This is backwards compatible! But new files using the Aberrant stuff are not readable
+ * by old rrdtool versions. See http://cricket.sourceforge.net/aberrant/rrd_hw.htm
+ * -- Jake Brutlag <jakeb@corp.webtv.net>
+ *
+ * Revision 1.1.1.1  2001/02/25 22:25:06  oetiker
+ * checkin
  *
  *****************************************************************************/
 #ifdef  __cplusplus
@@ -153,11 +160,36 @@ enum dst_en dst_conv(char *string);
 long ds_match(rrd_t *rrd,char *ds_nam);
 double rrd_diff(char *a, char *b);
 
+/* functions added for aberrant behavior detection.
+ * implemented for the most part in rrd_hw.c */
+int update_aberrant_CF(rrd_t *rrd, rrd_value_t pdp_val, enum cf_en current_cf,
+   unsigned long cdp_idx, unsigned long rra_idx, unsigned long ds_idx,
+   unsigned short CDP_scratch_idx, rrd_value_t *seasonal_coef);
+int create_hw_contingent_rras(rrd_t *rrd, unsigned short period, 
+   unsigned long hashed_name);
+int lookup_seasonal(rrd_t *rrd, unsigned long rra_idx, unsigned long rra_start,
+   FILE *rrd_file, unsigned long offset, rrd_value_t **seasonal_coef);
+void erase_violations(rrd_t *rrd, unsigned long cdp_idx, unsigned long rra_idx);
+int apply_smoother(rrd_t *rrd, unsigned long rra_idx, unsigned long rra_start,
+   FILE *rrd_file);
+
+/* a standard fixed-capacity FIFO queue implementation */
+typedef struct FIFOqueue {
+   rrd_value_t *queue;
+   int capacity, head, tail;
+} FIFOqueue;
+
+int queue_alloc(FIFOqueue **q,int capacity);
+void queue_dealloc(FIFOqueue *q);
+void queue_push(FIFOqueue *q, rrd_value_t value);
+int queue_isempty(FIFOqueue *q);
+rrd_value_t queue_pop(FIFOqueue *q);
+
+#define BURNIN_CYCLES 3
+
 #endif
 
 
 #ifdef  __cplusplus
 }
 #endif
-
-
diff --git a/src/rrd_tune.c b/src/rrd_tune.c
index d04a725..b45e5db 100644
--- a/src/rrd_tune.c
+++ b/src/rrd_tune.c
@@ -5,13 +5,24 @@
  *****************************************************************************
  * $Id$
  * $Log$
- * Revision 1.1  2001/02/25 22:25:06  oetiker
- * Initial revision
+ * Revision 1.2  2001/03/04 13:01:55  oetiker
+ * Aberrant Behavior Detection support. A brief overview added to rrdtool.pod.
+ * Major updates to rrd_update.c, rrd_create.c. Minor update to other core files.
+ * This is backwards compatible! But new files using the Aberrant stuff are not readable
+ * by old rrdtool versions. See http://cricket.sourceforge.net/aberrant/rrd_hw.htm
+ * -- Jake Brutlag <jakeb@corp.webtv.net>
+ *
+ * Revision 1.1.1.1  2001/02/25 22:25:06  oetiker
+ * checkin
  *
  *****************************************************************************/
 
 #include "rrd_tool.h"
 
+int set_hwarg(rrd_t *rrd,enum cf_en cf,enum rra_par_en rra_par,char *arg);
+int set_deltaarg(rrd_t *rrd,enum rra_par_en rra_par,char *arg);
+int set_windowarg(rrd_t *rrd,enum rra_par_en,char *arg);
+
 int
 rrd_tune(int argc, char **argv)    
 {   
@@ -41,11 +52,19 @@ rrd_tune(int argc, char **argv)
 	    {"maximum",          required_argument, 0, 'a'},
 	    {"data-source-type", required_argument, 0, 'd'},
 	    {"data-source-rename", required_argument, 0, 'r'},
+	    /* added parameter tuning options for aberrant behavior detection */
+		{"deltapos",required_argument,0,'p'},
+		{"deltaneg",required_argument,0,'n'},
+		{"window-length",required_argument,0,'w'},
+		{"failure-threshold",required_argument,0,'f'},
+	    {"alpha",required_argument,0,'x'},
+	    {"beta",required_argument,0,'y'},
+	    {"gamma",required_argument,0,'z'},
 	    {0,0,0,0}
 	};
 	int option_index = 0;
 	int opt;
-	opt = getopt_long(argc, argv, "h:i:a:d:r:", 
+	opt = getopt_long(argc, argv, "h:i:a:d:r:p:n:w:f:x:y:z:", 
 			  long_options, &option_index);
 	if (opt == EOF)
 	    break;
@@ -145,6 +164,48 @@ rrd_tune(int argc, char **argv)
 	    strncpy(rrd.ds_def[ds].ds_nam,ds_new,DS_NAM_SIZE-1);
 	    rrd.ds_def[ds].ds_nam[DS_NAM_SIZE-1]='\0';
 	    break;
+    case 'p':
+		if (set_deltaarg(&rrd,RRA_delta_pos,optarg)) {
+		   rrd_free(&rrd);
+		   return -1;
+		}
+		break;
+	case 'n':
+		if (set_deltaarg(&rrd,RRA_delta_neg,optarg)) {
+		   rrd_free(&rrd);
+		   return -1;
+		}
+		break;
+	case 'f':
+		if (set_windowarg(&rrd,RRA_failure_threshold,optarg)) {
+		   rrd_free(&rrd);
+		   return -1;
+		}
+		break;
+	case 'w':
+		if (set_windowarg(&rrd,RRA_window_len,optarg)) {
+		   rrd_free(&rrd);
+		   return -1;
+		}
+		break;
+	case 'x':
+		if (set_hwarg(&rrd,CF_HWPREDICT,RRA_hw_alpha,optarg)) {
+		   rrd_free(&rrd);
+		   return -1;
+		}
+		break;
+	case 'y':
+		if (set_hwarg(&rrd,CF_HWPREDICT,RRA_hw_beta,optarg)) {
+		   rrd_free(&rrd);
+		   return -1;
+		}
+		break;
+	case 'z':
+		if (set_hwarg(&rrd,CF_SEASONAL,RRA_seasonal_gamma,optarg)) {
+		   rrd_free(&rrd);
+		   return -1;
+		}
+		break;
 	case '?':
             if (optopt != 0)
                 rrd_set_error("unknown option '%c'", optopt);
@@ -162,6 +223,9 @@ rrd_tune(int argc, char **argv)
 	       sizeof(stat_head_t),1, rrd_file);
 	fwrite(rrd.ds_def,
 	       sizeof(ds_def_t), rrd.stat_head->ds_cnt, rrd_file);
+	/* need to write rra_defs for RRA parameter changes */
+	fwrite(rrd.rra_def, sizeof(rra_def_t), rrd.stat_head->rra_cnt,
+		   rrd_file);
     } else {
 	int i;
 	for(i=0;i< rrd.stat_head->ds_cnt;i++)
@@ -177,3 +241,114 @@ rrd_tune(int argc, char **argv)
     return 0;
 }
 
+int set_hwarg(rrd_t *rrd,enum cf_en cf,enum rra_par_en rra_par,char *arg)
+{
+   double param;
+   unsigned long i;
+   signed short rra_idx = -1, devseasonal_idx = -1;
+   /* read the value */
+   param = atof(arg);
+   if (param <= 0.0 || param >= 1.0)
+   {
+	  rrd_set_error("Holt-Winters parameter must be between 0 and 1");
+	  return -1;
+   }
+   /* does the appropriate RRA exist?  */
+   for (i =  0; i < rrd -> stat_head -> rra_cnt; ++i)
+   {
+	  if (cf_conv(rrd -> rra_def[i].cf_nam) == cf)
+	  {
+		 rra_idx = i;
+	  }
+	  /* find the DEVSEASONAL index separately, as it is optional */
+	  if (cf_conv(rrd -> rra_def[i].cf_nam) == CF_DEVSEASONAL)
+	  {
+		 devseasonal_idx = i;
+	  }
+   }
+   if (rra_idx == -1) 
+   {
+	  rrd_set_error("Holt-Winters RRA does not exist in this RRD");
+	  return -1;
+   }
+   
+   /* set the value */
+   rrd -> rra_def[rra_idx].par[rra_par].u_val = param;
+   if (devseasonal_idx > -1)
+	  rrd -> rra_def[devseasonal_idx].par[rra_par].u_val = param;
+   return 0;
+}
+
+int set_deltaarg(rrd_t *rrd,enum rra_par_en rra_par,char *arg)
+{
+   rrd_value_t param;
+   unsigned long i;
+   signed short rra_idx = -1;
+
+   param = atof(arg);
+   if (param < 0.1)
+   {
+	  rrd_set_error("Parameter specified is too small");
+	  return -1;
+   }
+   /* does the appropriate RRA exist?  */
+   for (i = 0; i < rrd -> stat_head -> rra_cnt; ++i)
+   {
+	  if (cf_conv(rrd -> rra_def[i].cf_nam) == CF_FAILURES) 
+	  {
+		 rra_idx = i;
+		 break;
+	  }
+   }
+   if (rra_idx == -1) 
+   {
+	  rrd_set_error("Failures RRA does not exist in this RRD");
+	  return -1;
+   }
+
+   /* set the value */
+   rrd -> rra_def[rra_idx].par[rra_par].u_val = param;
+   return 0;
+}
+
+int set_windowarg(rrd_t *rrd,enum rra_par_en rra_par,char *arg)
+{
+   unsigned long param;
+   unsigned long i, cdp_idx;
+   signed short rra_idx = -1;
+   /* read the value */
+   param = atoi(arg);
+   /* there are 4 chars to a long, reserve on CDP entry for future
+	* use. */
+   if (param < 1 || param > MAX_FAILURES_WINDOW_LEN)
+   {
+	  rrd_set_error("Parameter must be between %d and %d",
+		 1, MAX_CDP_PAR_EN - 1);
+	  return -1;
+   }
+   /* does the appropriate RRA exist?  */
+   for (i = 0; i < rrd -> stat_head -> rra_cnt; ++i)
+   {
+	  if (cf_conv(rrd -> rra_def[i].cf_nam) == CF_FAILURES) 
+	  {
+		 rra_idx = i;
+		 break;
+	  }
+   }
+   if (rra_idx == -1) 
+   {
+	  rrd_set_error("Failures RRA does not exist in this RRD");
+	  return -1;
+   }
+   
+   /* set the value */
+   rrd -> rra_def[rra_idx].par[rra_par].u_cnt = param;
+
+   /* erase existing violations */
+   for (i = 0; i < rrd -> stat_head -> ds_cnt; i++)
+   {
+	  cdp_idx = rra_idx * (rrd -> stat_head -> ds_cnt) + i;
+	  erase_violations(rrd,cdp_idx,rra_idx);
+   }
+   return 0;
+}
diff --git a/src/rrd_update.c b/src/rrd_update.c
index 1e5a83e..08ecb49 100644
--- a/src/rrd_update.c
+++ b/src/rrd_update.c
@@ -5,6 +5,13 @@
  *****************************************************************************
  * $Id$
  * $Log$
+ * Revision 1.3  2001/03/04 13:01:55  oetiker
+ * Aberrant Behavior Detection support. A brief overview added to rrdtool.pod.
+ * Major updates to rrd_update.c, rrd_create.c. Minor update to other core files.
+ * This is backwards compatible! But new files using the Aberrant stuff are not readable
+ * by old rrdtool versions. See http://cricket.sourceforge.net/aberrant/rrd_hw.htm
+ * -- Jake Brutlag <jakeb@corp.webtv.net>
+ *
  * Revision 1.2  2001/03/04 11:14:25  oetiker
  * added at-style-time@value:value syntax to rrd_update
  * --  Dave Bodenstab <imdave@mcs.net>
@@ -24,12 +31,15 @@
  #include <io.h>
 #endif
 
-
 /* Prototypes */
 int LockRRD(FILE *rrd_file);
-
+void write_RRA_row (rrd_t *rrd, unsigned long rra_idx, unsigned long *rra_current,
+    unsigned short CDP_scratch_idx, FILE *rrd_file);
+ 
 /*#define DEBUG */
 
+#define IFDNAN(X,Y) (isnan(X) ? (Y) : (X));
+
 
 #ifdef STANDALONE
 int 
@@ -56,7 +66,8 @@ rrd_update(int argc, char **argv)
 {
 
     int              arg_i = 2;
-    long             i,ii,iii;
+    short            j;
+    long             i,ii,iii=1;
 
     unsigned long    rra_begin;          /* byte pointer to the rra
 					  * area in the rrd file.  this
@@ -81,8 +92,6 @@ rrd_update(int argc, char **argv)
 					  * was last updated */
     unsigned long    occu_pdp_age;       /* how long ago was the last
 					  * pdp_step time */
-    unsigned long    pdp_st;             /* helper for cdp_prep 
-					  * processing */
     rrd_value_t      *pdp_new;           /* prepare the incoming data
 					  * to be added the the
 					  * existing entry */
@@ -98,9 +107,23 @@ rrd_update(int argc, char **argv)
     rrd_t            rrd;
     time_t           current_time = time(NULL);
     char             **updvals;
-    int              wrote_to_file = 0;
-    char             *template = NULL;          
-
+    int              schedule_smooth = 0;
+    char             *template = NULL;   
+	rrd_value_t      *seasonal_coef = NULL, *last_seasonal_coef = NULL;
+					 /* a vector of future Holt-Winters seasonal coefs */
+    unsigned long    elapsed_pdp_st;
+					 /* number of elapsed PDP steps since last update */
+    unsigned long    *rra_step_cnt = NULL;
+					 /* number of rows to be updated in an RRA for a data
+					  * value. */
+    unsigned long    start_pdp_offset;
+					 /* number of PDP steps since the last update that
+					  * are assigned to the first CDP to be generated
+					  * since the last update. */
+    unsigned short   scratch_idx;
+					 /* index into the CDP scratch array */
+    enum cf_en       current_cf;
+					 /* numeric id of the current consolidation function */
 
     while (1) {
 	static struct option long_options[] =
@@ -123,7 +146,7 @@ rrd_update(int argc, char **argv)
 
 	case '?':
 	    rrd_set_error("unknown option '%s'",argv[optind-1]);
-            rrd_free(&rrd);
+            rrd_free(&rrd);            
 	    return(-1);
 	}
     }
@@ -158,7 +181,7 @@ rrd_update(int argc, char **argv)
       rrd_free(&rrd);
       fclose(rrd_file);
       return(-1);   
-    }
+    } 
 
     if((updvals = malloc( sizeof(char*) * (rrd.stat_head->ds_cnt+1)))==NULL){
 	rrd_set_error("allocating updvals pointer array");
@@ -248,7 +271,7 @@ rrd_update(int argc, char **argv)
 	enum {atstyle, normal} timesyntax;
 	struct time_value ds_tv;
         if (stepper == NULL){
-                rrd_set_error("faild duplication argv entry");
+                rrd_set_error("failed duplication argv entry");
                 free(updvals);
                 free(pdp_temp);  
                 free(tmpl_idx);
@@ -329,7 +352,7 @@ rrd_update(int argc, char **argv)
 	}
 	
 	
-	/* seek to the beginning of the rrd's */
+	/* seek to the beginning of the rra's */
 	if (rra_current != rra_begin) {
 	    if(fseek(rrd_file, rra_begin, SEEK_SET) != 0) {
 		rrd_set_error("seek error in rrd");
@@ -532,183 +555,439 @@ rrd_update(int argc, char **argv)
 #endif
 	    }
 
-
-	    /* now we have to integrate this data into the cdp_prep areas */
-	    /* going through the round robin archives */
-	    for(i = 0;
-		i < rrd.stat_head->rra_cnt;
-		i++){
-		enum cf_en current_cf = cf_conv(rrd.rra_def[i].cf_nam);
-		/* going through all pdp_st moments which have occurred 
-		 * since the last run */
-		for(pdp_st  = proc_pdp_st+rrd.stat_head->pdp_step; 
-		    pdp_st <= occu_pdp_st; 
-		    pdp_st += rrd.stat_head->pdp_step){
-
+		/* compute the number of elapsed pdp_st moments */
+		elapsed_pdp_st = (occu_pdp_st - proc_pdp_st) / rrd.stat_head -> pdp_step;
 #ifdef DEBUG
-		    fprintf(stderr,"RRA %lu STEP %lu\n",i,pdp_st);
+		fprintf(stderr,"elapsed PDP steps: %lu\n", elapsed_pdp_st);
 #endif
+		if (rra_step_cnt == NULL)
+		{
+		   rra_step_cnt = (unsigned long *) 
+			  malloc((rrd.stat_head->rra_cnt)* sizeof(unsigned long));
+		}
 
-		    if((pdp_st %
-			(rrd.rra_def[i].pdp_cnt*rrd.stat_head->pdp_step)) == 0){
+	    for(i = 0, rra_start = rra_begin;
+		i < rrd.stat_head->rra_cnt;
+	    rra_start += rrd.rra_def[i].row_cnt * rrd.stat_head -> ds_cnt * sizeof(rrd_value_t),
+		i++)
+		{
+		current_cf = cf_conv(rrd.rra_def[i].cf_nam);
+		start_pdp_offset = rrd.rra_def[i].pdp_cnt -
+		   (proc_pdp_st / rrd.stat_head -> pdp_step) % rrd.rra_def[i].pdp_cnt;
+        if (start_pdp_offset <= elapsed_pdp_st) {
+           rra_step_cnt[i] = (elapsed_pdp_st - start_pdp_offset) / 
+		      rrd.rra_def[i].pdp_cnt + 1;
+	    } else {
+		   rra_step_cnt[i] = 0;
+		}
 
-			/* later on the cdp_prep values will be transferred to
-			 * the rra.  we want to be in the right place. */
-			rrd.rra_ptr[i].cur_row++;
-			if (rrd.rra_ptr[i].cur_row >= rrd.rra_def[i].row_cnt)
-			    /* oops ... we have to wrap the beast ... */
-			    rrd.rra_ptr[i].cur_row=0;			
+		if (current_cf == CF_SEASONAL || current_cf == CF_DEVSEASONAL) 
+		{
+		   /* If this is a bulk update, we need to skip ahead in the seasonal
+			* arrays so that they will be correct for the next observed value;
+			* note that for the bulk update itself, no update will occur to
+			* DEVSEASONAL or SEASONAL; futhermore, HWPREDICT and DEVPREDICT will
+			* be set to DNAN. */
+           if (rra_step_cnt[i] > 2) 
+		   {
+			  /* skip update by resetting rra_step_cnt[i],
+			   * note that this is not data source specific; this is due
+			   * to the bulk update, not a DNAN value for the specific data
+			   * source. */
+			  rra_step_cnt[i] = 0;
+              lookup_seasonal(&rrd,i,rra_start,rrd_file,elapsed_pdp_st, 
+			     &last_seasonal_coef);
+		      lookup_seasonal(&rrd,i,rra_start,rrd_file,elapsed_pdp_st + 1,
+			     &seasonal_coef);
+		   }
+		
+		  /* periodically run a smoother for seasonal effects */
+		  /* Need to use first cdp parameter buffer to track
+		   * burnin (burnin requires a specific smoothing schedule).
+		   * The CDP_init_seasonal parameter is really an RRA level,
+		   * not a data source within RRA level parameter, but the rra_def
+		   * is read only for rrd_update (not flushed to disk). */
+		  iii = i*(rrd.stat_head -> ds_cnt);
+		  if (rrd.cdp_prep[iii].scratch[CDP_init_seasonal].u_cnt 
+			  <= BURNIN_CYCLES)
+		  {
+		     if (rrd.rra_ptr[i].cur_row + elapsed_pdp_st 
+				 > rrd.rra_def[i].row_cnt - 1) {
+			   /* mark off one of the burnin cycles */
+			   ++(rrd.cdp_prep[iii].scratch[CDP_init_seasonal].u_cnt);
+		       schedule_smooth = 1;
+			 }  
+		  } else {
+			 /* someone has no doubt invented a trick to deal with this
+			  * wrap around, but at least this code is clear. */
+			 if (rrd.rra_def[i].par[RRA_seasonal_smooth_idx].u_cnt >
+			     rrd.rra_ptr[i].cur_row)
+			 {
+				 /* here elapsed_pdp_st = rra_step_cnt[i] because of 1-1
+				  * mapping between PDP and CDP */
+				 if (rrd.rra_ptr[i].cur_row + elapsed_pdp_st
+					>= rrd.rra_def[i].par[RRA_seasonal_smooth_idx].u_cnt)
+				 {
 #ifdef DEBUG
-			fprintf(stderr,"  -- RRA Preseek %ld\n",ftell(rrd_file));
+					fprintf(stderr,
+					"schedule_smooth 1: cur_row %lu, elapsed_pdp_st %lu, smooth idx %lu\n",
+                    rrd.rra_ptr[i].cur_row, elapsed_pdp_st, 
+					rrd.rra_def[i].par[RRA_seasonal_smooth_idx].u_cnt);
 #endif
-			/* determine if a seek is even needed. */
-			rra_pos_tmp = rra_start +
-				rrd.stat_head->ds_cnt*rrd.rra_ptr[i].cur_row*sizeof(rrd_value_t);
-			if(rra_pos_tmp != rra_current) {
-			    if(fseek(rrd_file, rra_pos_tmp, SEEK_SET) != 0){
-				rrd_set_error("seek error in rrd");
-				break;
-			    }
-			    rra_current = rra_pos_tmp;
-			}
+					schedule_smooth = 1;
+				 }
+             } else {
+				 /* can't rely on negative numbers because we are working with
+				  * unsigned values */
+				 /* Don't need modulus here. If we've wrapped more than once, only
+				  * one smooth is executed at the end. */
+				 if (rrd.rra_ptr[i].cur_row + elapsed_pdp_st >= rrd.rra_def[i].row_cnt
+					&& rrd.rra_ptr[i].cur_row + elapsed_pdp_st - rrd.rra_def[i].row_cnt
+					>= rrd.rra_def[i].par[RRA_seasonal_smooth_idx].u_cnt)
+				 {
 #ifdef DEBUG
-			fprintf(stderr,"  -- RRA Postseek %ld\n",ftell(rrd_file));
+					fprintf(stderr,
+					"schedule_smooth 2: cur_row %lu, elapsed_pdp_st %lu, smooth idx %lu\n",
+                    rrd.rra_ptr[i].cur_row, elapsed_pdp_st, 
+					rrd.rra_def[i].par[RRA_seasonal_smooth_idx].u_cnt);
 #endif
-		    }
+					schedule_smooth = 1;
+				 }
+			 }
+		  }
+
+	      rra_current = ftell(rrd_file); 
+		} /* if cf is DEVSEASONAL or SEASONAL */
 
+        if (rrd_test_error()) break;
+
+		    /* update CDP_PREP areas */
+		    /* loop over data soures within each RRA */
 		    for(ii = 0;
 			ii < rrd.stat_head->ds_cnt;
-			ii++){
+			ii++)
+			{
+			
+			/* iii indexes the CDP prep area for this data source within the RRA */
 			iii=i*rrd.stat_head->ds_cnt+ii;
-		    
-			/* the contents of cdp_prep[].scratch[CDP_val].u_val depends
-			 * on the consolidation function ! */
-		    
-			if (isnan(pdp_temp[ii])){    /* pdp is unknown */
-			    rrd.cdp_prep[iii].scratch[CDP_unkn_pdp_cnt].u_cnt++;
+
+			if (rrd.rra_def[i].pdp_cnt > 1) {
+			  
+			   if (rra_step_cnt[i] > 0) {
+			   /* If we are in this block, as least 1 CDP value will be written to
+				* disk, this is the CDP_primary_val entry. If more than 1 value needs
+				* to be written, then the "fill in" value is the CDP_secondary_val
+				* entry. */
+				  if (isnan(pdp_temp[ii]))
+                  {
+					 rrd.cdp_prep[iii].scratch[CDP_unkn_pdp_cnt].u_cnt += start_pdp_offset;
+					 rrd.cdp_prep[iii].scratch[CDP_secondary_val].u_val = DNAN;
+				  } else {
+					 /* CDP_secondary value is the RRA "fill in" value for intermediary
+					  * CDP data entries. No matter the CF, the value is the same because
+					  * the average, max, min, and last of a list of identical values is
+					  * the same, namely, the value itself. */
+					 rrd.cdp_prep[iii].scratch[CDP_secondary_val].u_val = pdp_temp[ii];
+				  }
+                     
+				  if (rrd.cdp_prep[iii].scratch[CDP_unkn_pdp_cnt].u_cnt
+				      > rrd.rra_def[i].pdp_cnt*
+				      rrd.rra_def[i].par[RRA_cdp_xff_val].u_val)
+				  {
+					 rrd.cdp_prep[iii].scratch[CDP_primary_val].u_val = DNAN;
+					 /* initialize carry over */
+					 if (current_cf == CF_AVERAGE) {
+						   if (isnan(pdp_temp[ii])) { 
+							  rrd.cdp_prep[iii].scratch[CDP_val].u_val = DNAN;
+						   } else {
+							  rrd.cdp_prep[iii].scratch[CDP_val].u_val = pdp_temp[ii] *
+								 ((elapsed_pdp_st - start_pdp_offset) % rrd.rra_def[i].pdp_cnt);
+						   }
+					 } else {
+						rrd.cdp_prep[iii].scratch[CDP_val].u_val = pdp_temp[ii];
+					 }
+				  } else {
+					 rrd_value_t cum_val, cur_val; 
+				     switch (current_cf) {
+						case CF_AVERAGE:
+						  cum_val = IFDNAN(rrd.cdp_prep[iii].scratch[CDP_val].u_val, 0.0);
+						  cur_val = IFDNAN(pdp_temp[ii],0.0);
+                          rrd.cdp_prep[iii].scratch[CDP_primary_val].u_val =
+					       (cum_val + cur_val) /
+				           (rrd.rra_def[i].pdp_cnt
+					       -rrd.cdp_prep[iii].scratch[CDP_unkn_pdp_cnt].u_cnt);
+						   /* initialize carry over value */
+						   if (isnan(pdp_temp[ii])) { 
+							  rrd.cdp_prep[iii].scratch[CDP_val].u_val = DNAN;
+						   } else {
+							  rrd.cdp_prep[iii].scratch[CDP_val].u_val = pdp_temp[ii] *
+								 ((elapsed_pdp_st - start_pdp_offset) % rrd.rra_def[i].pdp_cnt);
+						   }
+						   break;
+						case CF_MAXIMUM:
+						  cum_val = IFDNAN(rrd.cdp_prep[iii].scratch[CDP_val].u_val, -DINF);
+						  cur_val = IFDNAN(pdp_temp[ii],-DINF);
 #ifdef DEBUG
-			    fprintf(stderr,"  ** UNKNOWN ADD %lu\n",
-				    rrd.cdp_prep[iii].scratch[CDP_unkn_pdp_cnt].u_cnt);
+						  if (isnan(rrd.cdp_prep[iii].scratch[CDP_val].u_val) &&
+							  isnan(pdp_temp[ii])) {
+						     fprintf(stderr,
+								"RRA %lu, DS %lu, both CDP_val and pdp_temp are DNAN!",
+								i,ii);
+							 exit(-1);
+						  }
 #endif
-			} else {
-			    if (isnan(rrd.cdp_prep[iii].scratch[CDP_val].u_val)){
-				/* cdp_prep is unknown when it does not
-				 * yet contain data. It can not be zero for
-				 * things like mim and max consolidation
-				 * functions */
+						  if (cur_val > cum_val)
+							 rrd.cdp_prep[iii].scratch[CDP_primary_val].u_val = cur_val;
+						  else
+							 rrd.cdp_prep[iii].scratch[CDP_primary_val].u_val = cum_val;
+						  /* initialize carry over value */
+						  rrd.cdp_prep[iii].scratch[CDP_val].u_val = pdp_temp[ii];
+						  break;
+						case CF_MINIMUM:
+						  cum_val = IFDNAN(rrd.cdp_prep[iii].scratch[CDP_val].u_val, DINF);
+						  cur_val = IFDNAN(pdp_temp[ii],DINF);
 #ifdef DEBUG
-				fprintf(stderr,"  ** INIT CDP %e\n", pdp_temp[ii]);
+						  if (isnan(rrd.cdp_prep[iii].scratch[CDP_val].u_val) &&
+							  isnan(pdp_temp[ii])) {
+						     fprintf(stderr,
+								"RRA %lu, DS %lu, both CDP_val and pdp_temp are DNAN!",
+								i,ii);
+							 exit(-1);
+						  }
 #endif
-				rrd.cdp_prep[iii].scratch[CDP_val].u_val = pdp_temp[ii];
-			    }
-			    else {
-				switch (current_cf){
-				    case CF_AVERAGE:				
-					rrd.cdp_prep[iii].scratch[CDP_val].u_val+=pdp_temp[ii];
+						  if (cur_val < cum_val)
+							 rrd.cdp_prep[iii].scratch[CDP_primary_val].u_val = cur_val;
+						  else
+							 rrd.cdp_prep[iii].scratch[CDP_primary_val].u_val = cum_val;
+						  /* initialize carry over value */
+						  rrd.cdp_prep[iii].scratch[CDP_val].u_val = pdp_temp[ii];
+						  break;
+						case CF_LAST:
+						default:
+						   rrd.cdp_prep[iii].scratch[CDP_primary_val].u_val = pdp_temp[ii];
+						   /* initialize carry over value */
+						   rrd.cdp_prep[iii].scratch[CDP_val].u_val = pdp_temp[ii];
+						break;
+					 }
+				  } /* endif meets xff value requirement for a valid value */
+				  /* initialize carry over CDP_unkn_pdp_cnt, this must after CDP_primary_val
+				   * is set because CDP_unkn_pdp_cnt is required to compute that value. */
+				  if (isnan(pdp_temp[ii]))
+					 rrd.cdp_prep[iii].scratch[CDP_unkn_pdp_cnt].u_cnt = 
+						(elapsed_pdp_st - start_pdp_offset) % rrd.rra_def[i].pdp_cnt;
+				  else
+					 rrd.cdp_prep[iii].scratch[CDP_unkn_pdp_cnt].u_cnt = 0;
+               } else  /* rra_step_cnt[i]  == 0 */
+			   {
 #ifdef DEBUG
-					fprintf(stderr,"  ** AVERAGE %e\n", 
-						rrd.cdp_prep[iii].scratch[CDP_val].u_val);
+				  if (isnan(rrd.cdp_prep[iii].scratch[CDP_val].u_val)) {
+				  fprintf(stderr,"schedule CDP_val update, RRA %lu DS %lu, DNAN\n",
+					 i,ii);
+				  } else {
+				  fprintf(stderr,"schedule CDP_val update, RRA %lu DS %lu, %10.2f\n",
+					 i,ii,rrd.cdp_prep[iii].scratch[CDP_val].u_val);
+				  }
 #endif
-					break;    
-				    case CF_MINIMUM:
-					if (pdp_temp[ii] < rrd.cdp_prep[iii].scratch[CDP_val].u_val)
+				  if (isnan(pdp_temp[ii])) {
+			     	 rrd.cdp_prep[iii].scratch[CDP_unkn_pdp_cnt].u_cnt += elapsed_pdp_st;
+				  } else if (isnan(rrd.cdp_prep[iii].scratch[CDP_val].u_val))
+				  {
+					 if (current_cf == CF_AVERAGE) {
+					    rrd.cdp_prep[iii].scratch[CDP_val].u_val = pdp_temp[ii] *
+						   elapsed_pdp_st;
+					 } else {
 					    rrd.cdp_prep[iii].scratch[CDP_val].u_val = pdp_temp[ii];
+					 }
 #ifdef DEBUG
-					fprintf(stderr,"  ** MINIMUM %e\n", 
-						rrd.cdp_prep[iii].scratch[CDP_val].u_val);
+					 fprintf(stderr,"Initialize CDP_val for RRA %lu DS %lu: %10.2f\n",
+					    i,ii,rrd.cdp_prep[iii].scratch[CDP_val].u_val);
 #endif
-					break;
-				    case CF_MAXIMUM:
-					if (pdp_temp[ii] > rrd.cdp_prep[iii].scratch[CDP_val].u_val)
-					    rrd.cdp_prep[iii].scratch[CDP_val].u_val = pdp_temp[ii];
+				  } else {
+					 switch (current_cf) {
+					 case CF_AVERAGE:
+					    rrd.cdp_prep[iii].scratch[CDP_val].u_val += pdp_temp[ii] *
+						   elapsed_pdp_st;
+						break;
+					 case CF_MINIMUM:
+						if (pdp_temp[ii] < rrd.cdp_prep[iii].scratch[CDP_val].u_val)
+						   rrd.cdp_prep[iii].scratch[CDP_val].u_val = pdp_temp[ii];
+						break; 
+					 case CF_MAXIMUM:
+						if (pdp_temp[ii] > rrd.cdp_prep[iii].scratch[CDP_val].u_val)
+						   rrd.cdp_prep[iii].scratch[CDP_val].u_val = pdp_temp[ii];
+						break; 
+					 case CF_LAST:
+					 default:
+						rrd.cdp_prep[iii].scratch[CDP_val].u_val = pdp_temp[ii];
+						break;
+					 }
+				  }
+			   }
+			} else { /* rrd.rra_def[i].pdp_cnt == 1 */
+			   if (elapsed_pdp_st > 2)
+			   {
+				   switch (current_cf) {
+				   case CF_AVERAGE:
+				   default:
+			          rrd.cdp_prep[iii].scratch[CDP_primary_val].u_val=pdp_temp[ii];
+			          rrd.cdp_prep[iii].scratch[CDP_secondary_val].u_val=pdp_temp[ii];
+					  break;
+                   case CF_SEASONAL:
+				   case CF_DEVSEASONAL:
+					  /* need to update cached seasonal values, so they are consistent
+					   * with the bulk update */
+                      /* WARNING: code relies on the fact that CDP_hw_last_seasonal and
+					   * CDP_last_deviation are the same. */
+                      rrd.cdp_prep[iii].scratch[CDP_hw_last_seasonal].u_val =
+						 last_seasonal_coef[ii];
+					  rrd.cdp_prep[iii].scratch[CDP_hw_seasonal].u_val =
+						 seasonal_coef[ii];
+					  break;
+                   case CF_HWPREDICT:
+					  /* need to update the null_count and last_null_count.
+					   * even do this for non-DNAN pdp_temp because the
+					   * algorithm is not learning from batch updates. */
+					  rrd.cdp_prep[iii].scratch[CDP_null_count].u_cnt += 
+						 elapsed_pdp_st;
+					  rrd.cdp_prep[iii].scratch[CDP_last_null_count].u_cnt += 
+						 elapsed_pdp_st - 1;
+					  /* fall through */
+				   case CF_DEVPREDICT:
+			          rrd.cdp_prep[iii].scratch[CDP_primary_val].u_val = DNAN;
+			          rrd.cdp_prep[iii].scratch[CDP_secondary_val].u_val = DNAN;
+					  break;
+                   case CF_FAILURES:
+					  /* do not count missed bulk values as failures */
+			          rrd.cdp_prep[iii].scratch[CDP_primary_val].u_val = 0;
+			          rrd.cdp_prep[iii].scratch[CDP_secondary_val].u_val = 0;
+					  /* need to reset violations buffer.
+					   * could do this more carefully, but for now, just
+					   * assume a bulk update wipes away all violations. */
+                      erase_violations(&rrd, iii, i);
+					  break;
+				   }
+			   } 
+			} /* endif rrd.rra_def[i].pdp_cnt == 1 */
+
+			if (rrd_test_error()) break;
+
+			} /* endif data sources loop */
+        } /* end RRA Loop */
+
+		/* this loop is only entered if elapsed_pdp_st < 3 */
+		for (j = elapsed_pdp_st, scratch_idx = CDP_primary_val; 
+			 j > 0 && j < 3; j--, scratch_idx = CDP_secondary_val)
+		{
+	       for(i = 0, rra_start = rra_begin;
+		   i < rrd.stat_head->rra_cnt;
+	       rra_start += rrd.rra_def[i].row_cnt * rrd.stat_head -> ds_cnt * sizeof(rrd_value_t),
+		   i++)
+		   {
+			  if (rrd.rra_def[i].pdp_cnt > 1) continue;
+
+	          current_cf = cf_conv(rrd.rra_def[i].cf_nam);
+			  if (current_cf == CF_SEASONAL || current_cf == CF_DEVSEASONAL)
+			  {
+		         lookup_seasonal(&rrd,i,rra_start,rrd_file,
+				    elapsed_pdp_st + (scratch_idx == CDP_primary_val ? 1 : 2),
+			        &seasonal_coef);
+			  }
+			  if (rrd_test_error()) break;
+		      /* loop over data soures within each RRA */
+		      for(ii = 0;
+			  ii < rrd.stat_head->ds_cnt;
+			  ii++)
+			  {
+			     update_aberrant_CF(&rrd,pdp_temp[ii],current_cf,
+					i*(rrd.stat_head->ds_cnt) + ii,i,ii,
+				    scratch_idx, seasonal_coef);
+			  }
+           } /* end RRA Loop */
+		   if (rrd_test_error()) break;
+	    } /* end elapsed_pdp_st loop */
+
+		if (rrd_test_error()) break;
+
+		/* Ready to write to disk */
+		/* Move sequentially through the file, writing one RRA at a time.
+		 * Note this architecture divorces the computation of CDP with
+		 * flushing updated RRA entries to disk. */
+	    for(i = 0, rra_start = rra_begin;
+		i < rrd.stat_head->rra_cnt;
+	    rra_start += rrd.rra_def[i].row_cnt * rrd.stat_head -> ds_cnt * sizeof(rrd_value_t),
+		i++) {
+		/* is there anything to write for this RRA? If not, continue. */
+        if (rra_step_cnt[i] == 0) continue;
+
+		/* write the first row */
 #ifdef DEBUG
-					fprintf(stderr,"  ** MAXIMUM %e\n", 
-						rrd.cdp_prep[iii].scratch[CDP_val].u_val);
+        fprintf(stderr,"  -- RRA Preseek %ld\n",ftell(rrd_file));
 #endif
-					break;
-				    case CF_LAST:
-					rrd.cdp_prep[iii].scratch[CDP_val].u_val=pdp_temp[ii];
+	    rrd.rra_ptr[i].cur_row++;
+	    if (rrd.rra_ptr[i].cur_row >= rrd.rra_def[i].row_cnt)
+		   rrd.rra_ptr[i].cur_row = 0; /* wrap around */
+		/* positition on the first row */
+		rra_pos_tmp = rra_start +
+		   (rrd.stat_head->ds_cnt)*(rrd.rra_ptr[i].cur_row)*sizeof(rrd_value_t);
+		if(rra_pos_tmp != rra_current) {
+		   if(fseek(rrd_file, rra_pos_tmp, SEEK_SET) != 0){
+		      rrd_set_error("seek error in rrd");
+		      break;
+		   }
+		   rra_current = rra_pos_tmp;
+		}
 #ifdef DEBUG
-					fprintf(stderr,"  ** LAST %e\n", 
-						rrd.cdp_prep[iii].scratch[CDP_val].u_val);
+	    fprintf(stderr,"  -- RRA Postseek %ld\n",ftell(rrd_file));
 #endif
-					break;    
-				    default:
-					rrd_set_error("Unknown cf %s",
-						      rrd.rra_def[i].cf_nam);
-					break;
-				}
-			    }
-			}
-
-
-			/* is the data in the cdp_prep ready to go into
-			 * its rra ? */
-			if((pdp_st % 
-			    (rrd.rra_def[i].pdp_cnt*rrd.stat_head->pdp_step)) == 0){
-
-			    /* prepare cdp_pref for its transition to the rra. */
-			    if (rrd.cdp_prep[iii].scratch[CDP_unkn_pdp_cnt].u_cnt 
-				> rrd.rra_def[i].pdp_cnt*
-				rrd.rra_def[i].par[RRA_cdp_xff_val].u_val)
-				/* to much of the cdp_prep is unknown ... */
-				rrd.cdp_prep[iii].scratch[CDP_val].u_val = DNAN;
-			    else if (current_cf == CF_AVERAGE){
-				/* for a real average we have to divide
-				 * the sum we built earlier on. While ignoring
-				 * the unknown pdps */
-				rrd.cdp_prep[iii].scratch[CDP_val].u_val 
-					/= (rrd.rra_def[i].pdp_cnt
-					    -rrd.cdp_prep[iii].scratch[CDP_unkn_pdp_cnt].u_cnt);
-			    }
-			    /* we can write straight away, because we are
-			     * already in the right place ... */
-
+		scratch_idx = CDP_primary_val;
+		write_RRA_row(&rrd, i, &rra_current, scratch_idx, rrd_file);
+		if (rrd_test_error()) break;
+
+		/* write other rows of the bulk update, if any */
+		scratch_idx = CDP_secondary_val;
+		for ( ; rra_step_cnt[i] > 1; 
+		     rra_step_cnt[i]--, rrd.rra_ptr[i].cur_row++)
+		{
+		   if (rrd.rra_ptr[i].cur_row == rrd.rra_def[i].row_cnt)
+		   {
 #ifdef DEBUG
-			    fprintf(stderr,"  -- RRA WRITE VALUE %e, at %ld\n",
-				    rrd.cdp_prep[iii].scratch[CDP_val].u_val,ftell(rrd_file));
+              fprintf(stderr,"Wraparound for RRA %s, %lu updates left\n",
+			  rrd.rra_def[i].cf_nam, rra_step_cnt[i] - 1);
 #endif
-
-			    if(fwrite(&(rrd.cdp_prep[iii].scratch[CDP_val].u_val),
-				      sizeof(rrd_value_t),1,rrd_file) != 1){
-				rrd_set_error("writing rrd");
-				break;
-			    }
-			    rra_current += sizeof(rrd_value_t);
-			    wrote_to_file = 1;
-
+			  /* wrap */
+			  rrd.rra_ptr[i].cur_row = 0;
+			  /* seek back to beginning of current rra */
+		      if (fseek(rrd_file, rra_start, SEEK_SET) != 0)
+			  {
+		         rrd_set_error("seek error in rrd");
+		         break;
+			  }
 #ifdef DEBUG
-			    fprintf(stderr,"  -- RRA WROTE new at %ld\n",ftell(rrd_file));
+	          fprintf(stderr,"  -- Wraparound Postseek %ld\n",ftell(rrd_file));
 #endif
-
-			    /* make cdp_prep ready for the next run */
-			    rrd.cdp_prep[iii].scratch[CDP_val].u_val = DNAN;
-			    rrd.cdp_prep[iii].scratch[CDP_unkn_pdp_cnt].u_cnt = 0;
-			}
-		    }
-		    /* break out of this loop if error_string has been set */
-		    if (rrd_test_error())
-			break;
+			  rra_current = rra_start;
+		   }
+		   write_RRA_row(&rrd, i, &rra_current, scratch_idx, rrd_file);
 		}
-		/* break out of this loop if error_string has been set */
+		
 		if (rrd_test_error())
-		    break;
-		/* to be able to position correctly in the next rra w move
-		 * the rra_start pointer on to the next rra */
-		rra_start += rrd.rra_def[i].row_cnt
-			*rrd.stat_head->ds_cnt*sizeof(rrd_value_t);
+		  break;
+		} /* RRA LOOP */
 
-	    }
 	    /* break out of the argument parsing loop if error_string is set */
 	    if (rrd_test_error()){
-		free(step_start);
-		break;
-	    }
-	}
+		   free(step_start);
+		   break;
+	    } 
+	    
+	} /* endif a pdp_st has occurred */ 
 	rrd.live_head->last_up = current_time;
 	free(step_start);
-    }
+    } /* function argument loop */
 
+    if (seasonal_coef != NULL) free(seasonal_coef);
+    if (last_seasonal_coef != NULL) free(last_seasonal_coef);
+	if (rra_step_cnt != NULL) free(rra_step_cnt);
 
     /* if we got here and if there is an error and if the file has not been
      * written to, then close things up and return. */
@@ -803,6 +1082,36 @@ rrd_update(int argc, char **argv)
 	return(-1);
     }
 
+    /* calling the smoothing code here guarantees at most
+ 	 * one smoothing operation per rrd_update call. Unfortunately,
+	 * it is possible with bulk updates, or a long-delayed update
+	 * for smoothing to occur off-schedule. This really isn't
+	 * critical except during the burning cycles. */
+	if (schedule_smooth)
+	{
+#ifndef WIN32
+	  rrd_file = fopen(argv[optind],"r+");
+#else
+	  rrd_file = fopen(argv[optind],"rb+");
+#endif
+	  rra_start = rra_begin;
+	  for (i = 0; i < rrd.stat_head -> rra_cnt; ++i)
+	  {
+	    if (cf_conv(rrd.rra_def[i].cf_nam) == CF_DEVSEASONAL ||
+		cf_conv(rrd.rra_def[i].cf_nam) == CF_SEASONAL)
+	    {
+#ifdef DEBUG
+	      fprintf(stderr,"Running smoother for rra %ld\n",i);
+#endif
+	      apply_smoother(&rrd,i,rra_start,rrd_file);
+	      if (rrd_test_error())
+		break;
+	    }
+	    rra_start += rrd.rra_def[i].row_cnt
+	      *rrd.stat_head->ds_cnt*sizeof(rrd_value_t);
+	  }
+	  fclose(rrd_file);
+	}
     rrd_free(&rrd);
     free(updvals);
     free(tmpl_idx);
@@ -847,3 +1156,30 @@ LockRRD(FILE *rrdfile)
 
     return(stat);
 }
+
+
+void
+write_RRA_row (rrd_t *rrd, unsigned long rra_idx, unsigned long *rra_current,
+   unsigned short CDP_scratch_idx, FILE *rrd_file)
+{
+   unsigned long ds_idx, cdp_idx;
+
+   for (ds_idx = 0; ds_idx < rrd -> stat_head -> ds_cnt; ds_idx++)
+   {
+      /* compute the cdp index */
+      cdp_idx =rra_idx * (rrd -> stat_head->ds_cnt) + ds_idx;
+#ifdef DEBUG
+	  fprintf(stderr,"  -- RRA WRITE VALUE %e, at %ld CF:%s\n",
+	     rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val,ftell(rrd_file),
+	     rrd -> rra_def[rra_idx].cf_nam);
+#endif
+
+	  if(fwrite(&(rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val),
+		 sizeof(rrd_value_t),1,rrd_file) != 1)
+	  { 
+	     rrd_set_error("writing rrd");
+		 return;
+	  }
+	  *rra_current += sizeof(rrd_value_t);
+	}
+}
diff --git a/src/rrdupdate.c b/src/rrdupdate.c
new file mode 100644
index 0000000..cbc99a6
--- /dev/null
+++ b/src/rrdupdate.c
@@ -0,0 +1,1184 @@
+/*****************************************************************************
+ * RRDtool 1.0.33  Copyright Tobias Oetiker, 1997 - 2000
+ *****************************************************************************
+ * rrd_update.c  RRD Update Function
+ *****************************************************************************
+ * $Id$
+ * $Log$
+ * Revision 1.3  2001/03/04 13:01:56  oetiker
+ * Aberrant Behavior Detection support. A brief overview added to rrdtool.pod.
+ * Major updates to rrd_update.c, rrd_create.c. Minor update to other core files.
+ * This is backwards compatible! But new files using the Aberrant stuff are not readable
+ * by old rrdtool versions. See http://cricket.sourceforge.net/aberrant/rrd_hw.htm
+ * -- Jake Brutlag <jakeb@corp.webtv.net>
+ *
+ * Revision 1.2  2001/03/04 11:14:25  oetiker
+ * added at-style-time@value:value syntax to rrd_update
+ * --  Dave Bodenstab <imdave@mcs.net>
+ * Revision 1.1  2001/02/25 22:25:06  oetiker
+ * Initial revision
+ *
+ *****************************************************************************/
+
+#include "rrd_tool.h"
+#include <sys/types.h>
+#include <fcntl.h>
+
+#ifdef WIN32
+ #include <sys/locking.h>
+ #include <sys/stat.h>
+ #include <io.h>
+#endif
+
+/* Prototypes */
+int LockRRD(FILE *rrd_file);
+void write_RRA_row (rrd_t *rrd, unsigned long rra_idx, unsigned long *rra_current,
+    unsigned short CDP_scratch_idx, FILE *rrd_file);
+ 
+/*#define DEBUG */
+
+#define IFDNAN(X,Y) (isnan(X) ? (Y) : (X));
+
+
+#ifdef STANDALONE
+int 
+main(int argc, char **argv){
+        rrd_update(argc,argv);
+        if (rrd_test_error()) {
+                printf("RRDtool 1.0.33  Copyright 1997-2000 by Tobias Oetiker <tobi@oetiker.ch>\n\n"
+                        "Usage: rrdupdate filename\n"
+                        "\t\t\t[--template|-t ds-name:ds-name:...]\n"
+                        "\t\t\ttime|N:value[:value...]\n\n"
+                        "\t\t\tat-time@value[:value...]\n\n"
+                        "\t\t\t[ time:value[:value...] ..]\n\n");
+                                   
+                printf("ERROR: %s\n",rrd_get_error());
+                rrd_clear_error();                                                            
+                return 1;
+        }
+        return 0;
+}
+#endif
+
+int
+rrd_update(int argc, char **argv)
+{
+
+    int              arg_i = 2;
+    short            j;
+    long             i,ii,iii=1;
+
+    unsigned long    rra_begin;          /* byte pointer to the rra
+					  * area in the rrd file.  this
+					  * pointer never changes value */
+    unsigned long    rra_start;          /* byte pointer to the rra
+					  * area in the rrd file.  this
+					  * pointer changes as each rrd is
+					  * processed. */
+    unsigned long    rra_current;        /* byte pointer to the current write
+					  * spot in the rrd file. */
+    unsigned long    rra_pos_tmp;        /* temporary byte pointer. */
+    unsigned long    interval,
+	pre_int,post_int;                /* interval between this and
+					  * the last run */
+    unsigned long    proc_pdp_st;        /* which pdp_st was the last
+					  * to be processed */
+    unsigned long    occu_pdp_st;        /* when was the pdp_st
+					  * before the last update
+					  * time */
+    unsigned long    proc_pdp_age;       /* how old was the data in
+					  * the pdp prep area when it
+					  * was last updated */
+    unsigned long    occu_pdp_age;       /* how long ago was the last
+					  * pdp_step time */
+    rrd_value_t      *pdp_new;           /* prepare the incoming data
+					  * to be added the the
+					  * existing entry */
+    rrd_value_t      *pdp_temp;          /* prepare the pdp values 
+					  * to be added the the
+					  * cdp values */
+
+    long             *tmpl_idx;          /* index representing the settings
+					    transported by the template index */
+    long             tmpl_cnt = 2;       /* time and data */
+
+    FILE             *rrd_file;
+    rrd_t            rrd;
+    time_t           current_time = time(NULL);
+    char             **updvals;
+    int              schedule_smooth = 0;
+    char             *template = NULL;   
+	rrd_value_t      *seasonal_coef = NULL, *last_seasonal_coef = NULL;
+					 /* a vector of future Holt-Winters seasonal coefs */
+    unsigned long    elapsed_pdp_st;
+					 /* number of elapsed PDP steps since last update */
+    unsigned long    *rra_step_cnt = NULL;
+					 /* number of rows to be updated in an RRA for a data
+					  * value. */
+    unsigned long    start_pdp_offset;
+					 /* number of PDP steps since the last update that
+					  * are assigned to the first CDP to be generated
+					  * since the last update. */
+    unsigned short   scratch_idx;
+					 /* index into the CDP scratch array */
+    enum cf_en       current_cf;
+					 /* numeric id of the current consolidation function */
+
+    while (1) {
+	static struct option long_options[] =
+	{
+	    {"template",      required_argument, 0, 't'},
+	    {0,0,0,0}
+	};
+	int option_index = 0;
+	int opt;
+	opt = getopt_long(argc, argv, "t:", 
+			  long_options, &option_index);
+	
+	if (opt == EOF)
+	  break;
+	
+	switch(opt) {
+	case 't':
+	    template = optarg;
+	    break;
+
+	case '?':
+	    rrd_set_error("unknown option '%s'",argv[optind-1]);
+            rrd_free(&rrd);            
+	    return(-1);
+	}
+    }
+
+    /* need at least 2 arguments: filename, data. */
+    if (argc-optind < 2) {
+	rrd_set_error("Not enough arguments");
+	return -1;
+    }
+
+    if(rrd_open(argv[optind],&rrd_file,&rrd, RRD_READWRITE)==-1){
+	return -1;
+    }
+    rra_current = rra_start = rra_begin = ftell(rrd_file);
+    /* This is defined in the ANSI C standard, section 7.9.5.3:
+
+        When a file is opened with udpate mode ('+' as the second
+        or third character in the ... list of mode argument
+        variables), both input and ouptut may be performed on the
+        associated stream.  However, ...  input may not be directly
+        followed by output without an intervening call to a file
+        positioning function, unless the input oepration encounters
+        end-of-file. */
+    fseek(rrd_file, 0, SEEK_CUR);
+
+    
+    /* get exclusive lock to whole file.
+     * lock gets removed when we close the file.
+     */
+    if (LockRRD(rrd_file) != 0) {
+      rrd_set_error("could not lock RRD");
+      rrd_free(&rrd);
+      fclose(rrd_file);
+      return(-1);   
+    } 
+
+    if((updvals = malloc( sizeof(char*) * (rrd.stat_head->ds_cnt+1)))==NULL){
+	rrd_set_error("allocating updvals pointer array");
+	rrd_free(&rrd);
+        fclose(rrd_file);
+	return(-1);
+    }
+
+    if ((pdp_temp = malloc(sizeof(rrd_value_t)
+			   *rrd.stat_head->ds_cnt))==NULL){
+	rrd_set_error("allocating pdp_temp ...");
+	free(updvals);
+	rrd_free(&rrd);
+        fclose(rrd_file);
+	return(-1);
+    }
+
+    if ((tmpl_idx = malloc(sizeof(unsigned long)
+			   *(rrd.stat_head->ds_cnt+1)))==NULL){
+	rrd_set_error("allocating tmpl_idx ...");
+	free(pdp_temp);
+	free(updvals);
+	rrd_free(&rrd);
+        fclose(rrd_file);
+	return(-1);
+    }
+    /* initialize template redirector */
+    /* default config
+       tmpl_idx[0] -> 0; (time)
+       tmpl_idx[1] -> 1; (DS 0)
+       tmpl_idx[2] -> 2; (DS 1)
+       tmpl_idx[3] -> 3; (DS 2)
+       ... */
+    for (i=0;i<=rrd.stat_head->ds_cnt;i++) tmpl_idx[i]=i;
+    tmpl_cnt=rrd.stat_head->ds_cnt+1;
+    if (template) {
+	char *dsname;
+	int tmpl_len;
+	dsname = template;
+	tmpl_cnt = 1; /* the first entry is the time */
+	tmpl_len = strlen(template);
+	for(i=0;i<=tmpl_len ;i++) {
+	    if (template[i] == ':' || template[i] == '\0') {
+		template[i] = '\0';
+		if (tmpl_cnt>rrd.stat_head->ds_cnt){
+  		    rrd_set_error("Template contains more DS definitions than RRD");
+		    free(updvals); free(pdp_temp);
+		    free(tmpl_idx); rrd_free(&rrd);
+		    fclose(rrd_file); return(-1);
+		}
+		if ((tmpl_idx[tmpl_cnt++] = ds_match(&rrd,dsname)) == -1){
+  		    rrd_set_error("unknown DS name '%s'",dsname);
+		    free(updvals); free(pdp_temp);
+		    free(tmpl_idx); rrd_free(&rrd);
+		    fclose(rrd_file); return(-1);
+		} else {
+		  /* the first element is always the time */
+		  tmpl_idx[tmpl_cnt-1]++; 
+		  /* go to the next entry on the template */
+		  dsname = &template[i+1];
+                  /* fix the damage we did before */
+                  if (i<tmpl_len) {
+                     template[i]=':';
+                  } 
+
+		}
+	    }	    
+	}
+    }
+    if ((pdp_new = malloc(sizeof(rrd_value_t)
+			  *rrd.stat_head->ds_cnt))==NULL){
+	rrd_set_error("allocating pdp_new ...");
+	free(updvals);
+	free(pdp_temp);
+	free(tmpl_idx);
+	rrd_free(&rrd);
+        fclose(rrd_file);
+	return(-1);
+    }
+
+    /* loop through the arguments. */
+    for(arg_i=optind+1; arg_i<argc;arg_i++) {
+	char *stepper = malloc((strlen(argv[arg_i])+1)*sizeof(char));
+        char *step_start = stepper;
+	char *p;
+	char *parsetime_error = NULL;
+	enum {atstyle, normal} timesyntax;
+	struct time_value ds_tv;
+        if (stepper == NULL){
+                rrd_set_error("failed duplication argv entry");
+                free(updvals);
+                free(pdp_temp);  
+                free(tmpl_idx);
+                rrd_free(&rrd);
+                fclose(rrd_file);
+                return(-1);
+         }
+	/* initialize all ds input to unknown except the first one
+           which has always got to be set */
+	for(ii=1;ii<=rrd.stat_head->ds_cnt;ii++) updvals[ii] = "U";
+	strcpy(stepper,argv[arg_i]);
+	updvals[0]=stepper;
+	/* separate all ds elements; first must be examined separately
+	   due to alternate time syntax */
+	if ((p=strchr(stepper,'@'))!=NULL) {
+	    timesyntax = atstyle;
+	    *p = '\0';
+	    stepper = p+1;
+	} else if ((p=strchr(stepper,':'))!=NULL) {
+	    timesyntax = normal;
+	    *p = '\0';
+	    stepper = p+1;
+	} else {
+	    rrd_set_error("expected timestamp not found in data source from %s:...",
+			  argv[arg_i]);
+	    free(step_start);
+	    break;
+	}
+	ii=1;
+	updvals[tmpl_idx[ii]] = stepper;
+	while (*stepper) {
+	    if (*stepper == ':') {
+		*stepper = '\0';
+		ii++;
+		if (ii<tmpl_cnt){		    
+		    updvals[tmpl_idx[ii]] = stepper+1;
+		}
+	    }
+	    stepper++;
+	}
+
+	if (ii != tmpl_cnt-1) {
+	    rrd_set_error("expected %lu data source readings (got %lu) from %s:...",
+			  tmpl_cnt-1, ii, argv[arg_i]);
+	    free(step_start);
+	    break;
+	}
+	
+        /* get the time from the reading ... handle N */
+	if (timesyntax == atstyle) {
+            if ((parsetime_error = parsetime(updvals[0], &ds_tv))) {
+                rrd_set_error("ds time: %s: %s", updvals[0], parsetime_error );
+		free(step_start);
+		break;
+	    }
+	    if (ds_tv.type == RELATIVE_TO_END_TIME ||
+		ds_tv.type == RELATIVE_TO_START_TIME) {
+		rrd_set_error("specifying time relative to the 'start' "
+                              "or 'end' makes no sense here: %s",
+			      updvals[0]);
+		free(step_start);
+		break;
+	    }
+
+	    current_time = mktime(&ds_tv.tm) + ds_tv.offset;
+	} else if (strcmp(updvals[0],"N")==0){
+	    current_time = time(NULL);
+	} else {
+	    current_time = atol(updvals[0]);
+	}
+	
+	if(current_time <= rrd.live_head->last_up){
+	    rrd_set_error("illegal attempt to update using time %ld when "
+			  "last update time is %ld (minimum one second step)",
+			  current_time, rrd.live_head->last_up);
+	    free(step_start);
+	    break;
+	}
+	
+	
+	/* seek to the beginning of the rra's */
+	if (rra_current != rra_begin) {
+	    if(fseek(rrd_file, rra_begin, SEEK_SET) != 0) {
+		rrd_set_error("seek error in rrd");
+		free(step_start);
+		break;
+	    }
+	    rra_current = rra_begin;
+	}
+	rra_start = rra_begin;
+
+	/* when was the current pdp started */
+	proc_pdp_age = rrd.live_head->last_up % rrd.stat_head->pdp_step;
+	proc_pdp_st = rrd.live_head->last_up - proc_pdp_age;
+
+	/* when did the last pdp_st occur */
+	occu_pdp_age = current_time % rrd.stat_head->pdp_step;
+	occu_pdp_st = current_time - occu_pdp_age;
+	interval = current_time - rrd.live_head->last_up;
+    
+	if (occu_pdp_st > proc_pdp_st){
+	    /* OK we passed the pdp_st moment*/
+	    pre_int =  occu_pdp_st - rrd.live_head->last_up; /* how much of the input data
+							      * occurred before the latest
+							      * pdp_st moment*/
+	    post_int = occu_pdp_age;			     /* how much after it */
+	} else {
+	    pre_int = interval;
+	    post_int = 0;
+	}
+
+#ifdef DEBUG
+	printf(
+	       "proc_pdp_age %lu\t"
+	       "proc_pdp_st %lu\t" 
+	       "occu_pfp_age %lu\t" 
+	       "occu_pdp_st %lu\t"
+	       "int %lu\t"
+	       "pre_int %lu\t"
+	       "post_int %lu\n", proc_pdp_age, proc_pdp_st, 
+		occu_pdp_age, occu_pdp_st,
+	       interval, pre_int, post_int);
+#endif
+    
+	/* process the data sources and update the pdp_prep 
+	 * area accordingly */
+	for(i=0;i<rrd.stat_head->ds_cnt;i++){
+	    enum dst_en dst_idx;
+	    dst_idx= dst_conv(rrd.ds_def[i].dst);
+	    if((updvals[i+1][0] != 'U') &&
+	       rrd.ds_def[i].par[DS_mrhb_cnt].u_cnt >= interval) {
+	       double rate = DNAN;
+	       /* the data source type defines how to process the data */
+		/* pdp_temp contains rate * time ... eg the bytes
+		 * transferred during the interval. Doing it this way saves
+		 * a lot of math operations */
+		
+
+		switch(dst_idx){
+		case DST_COUNTER:
+		case DST_DERIVE:
+		    if(rrd.pdp_prep[i].last_ds[0] != 'U'){
+		       pdp_new[i]= rrd_diff(updvals[i+1],rrd.pdp_prep[i].last_ds);
+		       if(dst_idx == DST_COUNTER) {
+			  /* simple overflow catcher sugestet by andres kroonmaa */
+			  /* this will fail terribly for non 32 or 64 bit counters ... */
+			  /* are there any others in SNMP land ? */
+			  if (pdp_new[i] < (double)0.0 ) 
+			    pdp_new[i] += (double)4294967296.0 ;  /* 2^32 */
+			  if (pdp_new[i] < (double)0.0 ) 
+			    pdp_new[i] += (double)18446744069414584320.0; /* 2^64-2^32 */;
+		       }
+		       rate = pdp_new[i] / interval;
+		    }
+		   else {
+		     pdp_new[i]= DNAN;		
+		   }
+		   break;
+		case DST_ABSOLUTE:
+		    pdp_new[i]= atof(updvals[i+1]);
+		    rate = pdp_new[i] / interval;		  
+		    break;
+		case DST_GAUGE:
+		    pdp_new[i] = atof(updvals[i+1]) * interval;
+		    rate = pdp_new[i] / interval;		   
+		    break;
+		default:
+		    rrd_set_error("rrd contains unknown DS type : '%s'",
+				  rrd.ds_def[i].dst);
+		    break;
+		}
+		/* break out of this for loop if the error string is set */
+		if (rrd_test_error()){
+		    break;
+		}
+	       /* make sure pdp_temp is neither too large or too small
+		* if any of these occur it becomes unknown ...
+		* sorry folks ... */
+	       if ( ! isnan(rate) && 
+	            (( ! isnan(rrd.ds_def[i].par[DS_max_val].u_val) &&
+	                 rate > rrd.ds_def[i].par[DS_max_val].u_val ) ||     
+	            ( ! isnan(rrd.ds_def[i].par[DS_min_val].u_val) &&
+	                rate < rrd.ds_def[i].par[DS_min_val].u_val ))){
+		  pdp_new[i] = DNAN;
+	       }	       
+	    } else {
+		/* no news is news all the same */
+		pdp_new[i] = DNAN;
+	    }
+	    
+	    /* make a copy of the command line argument for the next run */
+#ifdef DEBUG
+	    fprintf(stderr,
+		    "prep ds[%lu]\t"
+		    "last_arg '%s'\t"
+		    "this_arg '%s'\t"
+		    "pdp_new %10.2f\n",
+		    i,
+		    rrd.pdp_prep[i].last_ds,
+		    updvals[i+1], pdp_new[i]);
+#endif
+	    if(dst_idx == DST_COUNTER || dst_idx == DST_DERIVE){
+		strncpy(rrd.pdp_prep[i].last_ds,
+			updvals[i+1],LAST_DS_LEN-1);
+		rrd.pdp_prep[i].last_ds[LAST_DS_LEN-1]='\0';
+	    }
+	}
+	/* break out of the argument parsing loop if the error_string is set */
+	if (rrd_test_error()){
+	    free(step_start);
+	    break;
+	}
+	/* has a pdp_st moment occurred since the last run ? */
+
+	if (proc_pdp_st == occu_pdp_st){
+	    /* no we have not passed a pdp_st moment. therefore update is simple */
+
+	    for(i=0;i<rrd.stat_head->ds_cnt;i++){
+		if(isnan(pdp_new[i]))
+		    rrd.pdp_prep[i].scratch[PDP_unkn_sec_cnt].u_cnt += interval;
+		else
+		    rrd.pdp_prep[i].scratch[PDP_val].u_val+= pdp_new[i];
+#ifdef DEBUG
+		fprintf(stderr,
+			"NO PDP  ds[%lu]\t"
+			"value %10.2f\t"
+			"unkn_sec %5lu\n",
+			i,
+			rrd.pdp_prep[i].scratch[PDP_val].u_val,
+			rrd.pdp_prep[i].scratch[PDP_unkn_sec_cnt].u_cnt);
+#endif
+	    }	
+	} else {
+	    /* an pdp_st has occurred. */
+
+	    /* in pdp_prep[].scratch[PDP_val].u_val we have collected rate*seconds which 
+	     * occurred up to the last run. 	   
+	    pdp_new[] contains rate*seconds from the latest run.
+	    pdp_temp[] will contain the rate for cdp */
+
+
+	    for(i=0;i<rrd.stat_head->ds_cnt;i++){
+		/* update pdp_prep to the current pdp_st */
+		if(isnan(pdp_new[i]))
+		    rrd.pdp_prep[i].scratch[PDP_unkn_sec_cnt].u_cnt += pre_int;
+		else
+		    rrd.pdp_prep[i].scratch[PDP_val].u_val += 
+			pdp_new[i]/(double)interval*(double)pre_int;
+
+		/* if too much of the pdp_prep is unknown we dump it */
+		if ((rrd.pdp_prep[i].scratch[PDP_unkn_sec_cnt].u_cnt 
+		     > rrd.ds_def[i].par[DS_mrhb_cnt].u_cnt) ||
+		    (occu_pdp_st-proc_pdp_st <= 
+		     rrd.pdp_prep[i].scratch[PDP_unkn_sec_cnt].u_cnt)) {
+		    pdp_temp[i] = DNAN;
+		} else {
+		    pdp_temp[i] = rrd.pdp_prep[i].scratch[PDP_val].u_val
+			/ (double)( occu_pdp_st
+				   - proc_pdp_st
+				   - rrd.pdp_prep[i].scratch[PDP_unkn_sec_cnt].u_cnt);
+		}
+		/* make pdp_prep ready for the next run */
+		if(isnan(pdp_new[i])){
+		    rrd.pdp_prep[i].scratch[PDP_unkn_sec_cnt].u_cnt = post_int;
+		    rrd.pdp_prep[i].scratch[PDP_val].u_val = 0.0;
+		} else {
+		    rrd.pdp_prep[i].scratch[PDP_unkn_sec_cnt].u_cnt = 0;
+		    rrd.pdp_prep[i].scratch[PDP_val].u_val = 
+			pdp_new[i]/(double)interval*(double)post_int;
+		}
+
+#ifdef DEBUG
+		fprintf(stderr,
+			"PDP UPD ds[%lu]\t"
+			"pdp_temp %10.2f\t"
+			"new_prep %10.2f\t"
+			"new_unkn_sec %5lu\n",
+			i, pdp_temp[i],
+			rrd.pdp_prep[i].scratch[PDP_val].u_val,
+			rrd.pdp_prep[i].scratch[PDP_unkn_sec_cnt].u_cnt);
+#endif
+	    }
+
+		/* compute the number of elapsed pdp_st moments */
+		elapsed_pdp_st = (occu_pdp_st - proc_pdp_st) / rrd.stat_head -> pdp_step;
+#ifdef DEBUG
+		fprintf(stderr,"elapsed PDP steps: %lu\n", elapsed_pdp_st);
+#endif
+		if (rra_step_cnt == NULL)
+		{
+		   rra_step_cnt = (unsigned long *) 
+			  malloc((rrd.stat_head->rra_cnt)* sizeof(unsigned long));
+		}
+
+	    for(i = 0, rra_start = rra_begin;
+		i < rrd.stat_head->rra_cnt;
+	    rra_start += rrd.rra_def[i].row_cnt * rrd.stat_head -> ds_cnt * sizeof(rrd_value_t),
+		i++)
+		{
+		current_cf = cf_conv(rrd.rra_def[i].cf_nam);
+		start_pdp_offset = rrd.rra_def[i].pdp_cnt -
+		   (proc_pdp_st / rrd.stat_head -> pdp_step) % rrd.rra_def[i].pdp_cnt;
+        if (start_pdp_offset <= elapsed_pdp_st) {
+           rra_step_cnt[i] = (elapsed_pdp_st - start_pdp_offset) / 
+		      rrd.rra_def[i].pdp_cnt + 1;
+	    } else {
+		   rra_step_cnt[i] = 0;
+		}
+
+		if (current_cf == CF_SEASONAL || current_cf == CF_DEVSEASONAL) 
+		{
+		   /* If this is a bulk update, we need to skip ahead in the seasonal
+			* arrays so that they will be correct for the next observed value;
+			* note that for the bulk update itself, no update will occur to
+			* DEVSEASONAL or SEASONAL; futhermore, HWPREDICT and DEVPREDICT will
+			* be set to DNAN. */
+           if (rra_step_cnt[i] > 2) 
+		   {
+			  /* skip update by resetting rra_step_cnt[i],
+			   * note that this is not data source specific; this is due
+			   * to the bulk update, not a DNAN value for the specific data
+			   * source. */
+			  rra_step_cnt[i] = 0;
+              lookup_seasonal(&rrd,i,rra_start,rrd_file,elapsed_pdp_st, 
+			     &last_seasonal_coef);
+		      lookup_seasonal(&rrd,i,rra_start,rrd_file,elapsed_pdp_st + 1,
+			     &seasonal_coef);
+		   }
+		
+		  /* periodically run a smoother for seasonal effects */
+		  /* Need to use first cdp parameter buffer to track
+		   * burnin (burnin requires a specific smoothing schedule).
+		   * The CDP_init_seasonal parameter is really an RRA level,
+		   * not a data source within RRA level parameter, but the rra_def
+		   * is read only for rrd_update (not flushed to disk). */
+		  iii = i*(rrd.stat_head -> ds_cnt);
+		  if (rrd.cdp_prep[iii].scratch[CDP_init_seasonal].u_cnt 
+			  <= BURNIN_CYCLES)
+		  {
+		     if (rrd.rra_ptr[i].cur_row + elapsed_pdp_st 
+				 > rrd.rra_def[i].row_cnt - 1) {
+			   /* mark off one of the burnin cycles */
+			   ++(rrd.cdp_prep[iii].scratch[CDP_init_seasonal].u_cnt);
+		       schedule_smooth = 1;
+			 }  
+		  } else {
+			 /* someone has no doubt invented a trick to deal with this
+			  * wrap around, but at least this code is clear. */
+			 if (rrd.rra_def[i].par[RRA_seasonal_smooth_idx].u_cnt >
+			     rrd.rra_ptr[i].cur_row)
+			 {
+				 /* here elapsed_pdp_st = rra_step_cnt[i] because of 1-1
+				  * mapping between PDP and CDP */
+				 if (rrd.rra_ptr[i].cur_row + elapsed_pdp_st
+					>= rrd.rra_def[i].par[RRA_seasonal_smooth_idx].u_cnt)
+				 {
+#ifdef DEBUG
+					fprintf(stderr,
+					"schedule_smooth 1: cur_row %lu, elapsed_pdp_st %lu, smooth idx %lu\n",
+                    rrd.rra_ptr[i].cur_row, elapsed_pdp_st, 
+					rrd.rra_def[i].par[RRA_seasonal_smooth_idx].u_cnt);
+#endif
+					schedule_smooth = 1;
+				 }
+             } else {
+				 /* can't rely on negative numbers because we are working with
+				  * unsigned values */
+				 /* Don't need modulus here. If we've wrapped more than once, only
+				  * one smooth is executed at the end. */
+				 if (rrd.rra_ptr[i].cur_row + elapsed_pdp_st >= rrd.rra_def[i].row_cnt
+					&& rrd.rra_ptr[i].cur_row + elapsed_pdp_st - rrd.rra_def[i].row_cnt
+					>= rrd.rra_def[i].par[RRA_seasonal_smooth_idx].u_cnt)
+				 {
+#ifdef DEBUG
+					fprintf(stderr,
+					"schedule_smooth 2: cur_row %lu, elapsed_pdp_st %lu, smooth idx %lu\n",
+                    rrd.rra_ptr[i].cur_row, elapsed_pdp_st, 
+					rrd.rra_def[i].par[RRA_seasonal_smooth_idx].u_cnt);
+#endif
+					schedule_smooth = 1;
+				 }
+			 }
+		  }
+
+	      rra_current = ftell(rrd_file); 
+		} /* if cf is DEVSEASONAL or SEASONAL */
+
+        if (rrd_test_error()) break;
+
+		    /* update CDP_PREP areas */
+		    /* loop over data soures within each RRA */
+		    for(ii = 0;
+			ii < rrd.stat_head->ds_cnt;
+			ii++)
+			{
+			
+			/* iii indexes the CDP prep area for this data source within the RRA */
+			iii=i*rrd.stat_head->ds_cnt+ii;
+
+			if (rrd.rra_def[i].pdp_cnt > 1) {
+			  
+			   if (rra_step_cnt[i] > 0) {
+			   /* If we are in this block, as least 1 CDP value will be written to
+				* disk, this is the CDP_primary_val entry. If more than 1 value needs
+				* to be written, then the "fill in" value is the CDP_secondary_val
+				* entry. */
+				  if (isnan(pdp_temp[ii]))
+                  {
+					 rrd.cdp_prep[iii].scratch[CDP_unkn_pdp_cnt].u_cnt += start_pdp_offset;
+					 rrd.cdp_prep[iii].scratch[CDP_secondary_val].u_val = DNAN;
+				  } else {
+					 /* CDP_secondary value is the RRA "fill in" value for intermediary
+					  * CDP data entries. No matter the CF, the value is the same because
+					  * the average, max, min, and last of a list of identical values is
+					  * the same, namely, the value itself. */
+					 rrd.cdp_prep[iii].scratch[CDP_secondary_val].u_val = pdp_temp[ii];
+				  }
+                     
+				  if (rrd.cdp_prep[iii].scratch[CDP_unkn_pdp_cnt].u_cnt
+				      > rrd.rra_def[i].pdp_cnt*
+				      rrd.rra_def[i].par[RRA_cdp_xff_val].u_val)
+				  {
+					 rrd.cdp_prep[iii].scratch[CDP_primary_val].u_val = DNAN;
+					 /* initialize carry over */
+					 if (current_cf == CF_AVERAGE) {
+						   if (isnan(pdp_temp[ii])) { 
+							  rrd.cdp_prep[iii].scratch[CDP_val].u_val = DNAN;
+						   } else {
+							  rrd.cdp_prep[iii].scratch[CDP_val].u_val = pdp_temp[ii] *
+								 ((elapsed_pdp_st - start_pdp_offset) % rrd.rra_def[i].pdp_cnt);
+						   }
+					 } else {
+						rrd.cdp_prep[iii].scratch[CDP_val].u_val = pdp_temp[ii];
+					 }
+				  } else {
+					 rrd_value_t cum_val, cur_val; 
+				     switch (current_cf) {
+						case CF_AVERAGE:
+						  cum_val = IFDNAN(rrd.cdp_prep[iii].scratch[CDP_val].u_val, 0.0);
+						  cur_val = IFDNAN(pdp_temp[ii],0.0);
+                          rrd.cdp_prep[iii].scratch[CDP_primary_val].u_val =
+					       (cum_val + cur_val) /
+				           (rrd.rra_def[i].pdp_cnt
+					       -rrd.cdp_prep[iii].scratch[CDP_unkn_pdp_cnt].u_cnt);
+						   /* initialize carry over value */
+						   if (isnan(pdp_temp[ii])) { 
+							  rrd.cdp_prep[iii].scratch[CDP_val].u_val = DNAN;
+						   } else {
+							  rrd.cdp_prep[iii].scratch[CDP_val].u_val = pdp_temp[ii] *
+								 ((elapsed_pdp_st - start_pdp_offset) % rrd.rra_def[i].pdp_cnt);
+						   }
+						   break;
+						case CF_MAXIMUM:
+						  cum_val = IFDNAN(rrd.cdp_prep[iii].scratch[CDP_val].u_val, -DINF);
+						  cur_val = IFDNAN(pdp_temp[ii],-DINF);
+#ifdef DEBUG
+						  if (isnan(rrd.cdp_prep[iii].scratch[CDP_val].u_val) &&
+							  isnan(pdp_temp[ii])) {
+						     fprintf(stderr,
+								"RRA %lu, DS %lu, both CDP_val and pdp_temp are DNAN!",
+								i,ii);
+							 exit(-1);
+						  }
+#endif
+						  if (cur_val > cum_val)
+							 rrd.cdp_prep[iii].scratch[CDP_primary_val].u_val = cur_val;
+						  else
+							 rrd.cdp_prep[iii].scratch[CDP_primary_val].u_val = cum_val;
+						  /* initialize carry over value */
+						  rrd.cdp_prep[iii].scratch[CDP_val].u_val = pdp_temp[ii];
+						  break;
+						case CF_MINIMUM:
+						  cum_val = IFDNAN(rrd.cdp_prep[iii].scratch[CDP_val].u_val, DINF);
+						  cur_val = IFDNAN(pdp_temp[ii],DINF);
+#ifdef DEBUG
+						  if (isnan(rrd.cdp_prep[iii].scratch[CDP_val].u_val) &&
+							  isnan(pdp_temp[ii])) {
+						     fprintf(stderr,
+								"RRA %lu, DS %lu, both CDP_val and pdp_temp are DNAN!",
+								i,ii);
+							 exit(-1);
+						  }
+#endif
+						  if (cur_val < cum_val)
+							 rrd.cdp_prep[iii].scratch[CDP_primary_val].u_val = cur_val;
+						  else
+							 rrd.cdp_prep[iii].scratch[CDP_primary_val].u_val = cum_val;
+						  /* initialize carry over value */
+						  rrd.cdp_prep[iii].scratch[CDP_val].u_val = pdp_temp[ii];
+						  break;
+						case CF_LAST:
+						default:
+						   rrd.cdp_prep[iii].scratch[CDP_primary_val].u_val = pdp_temp[ii];
+						   /* initialize carry over value */
+						   rrd.cdp_prep[iii].scratch[CDP_val].u_val = pdp_temp[ii];
+						break;
+					 }
+				  } /* endif meets xff value requirement for a valid value */
+				  /* initialize carry over CDP_unkn_pdp_cnt, this must after CDP_primary_val
+				   * is set because CDP_unkn_pdp_cnt is required to compute that value. */
+				  if (isnan(pdp_temp[ii]))
+					 rrd.cdp_prep[iii].scratch[CDP_unkn_pdp_cnt].u_cnt = 
+						(elapsed_pdp_st - start_pdp_offset) % rrd.rra_def[i].pdp_cnt;
+				  else
+					 rrd.cdp_prep[iii].scratch[CDP_unkn_pdp_cnt].u_cnt = 0;
+               } else  /* rra_step_cnt[i]  == 0 */
+			   {
+#ifdef DEBUG
+				  if (isnan(rrd.cdp_prep[iii].scratch[CDP_val].u_val)) {
+				  fprintf(stderr,"schedule CDP_val update, RRA %lu DS %lu, DNAN\n",
+					 i,ii);
+				  } else {
+				  fprintf(stderr,"schedule CDP_val update, RRA %lu DS %lu, %10.2f\n",
+					 i,ii,rrd.cdp_prep[iii].scratch[CDP_val].u_val);
+				  }
+#endif
+				  if (isnan(pdp_temp[ii])) {
+			     	 rrd.cdp_prep[iii].scratch[CDP_unkn_pdp_cnt].u_cnt += elapsed_pdp_st;
+				  } else if (isnan(rrd.cdp_prep[iii].scratch[CDP_val].u_val))
+				  {
+					 if (current_cf == CF_AVERAGE) {
+					    rrd.cdp_prep[iii].scratch[CDP_val].u_val = pdp_temp[ii] *
+						   elapsed_pdp_st;
+					 } else {
+					    rrd.cdp_prep[iii].scratch[CDP_val].u_val = pdp_temp[ii];
+					 }
+#ifdef DEBUG
+					 fprintf(stderr,"Initialize CDP_val for RRA %lu DS %lu: %10.2f\n",
+					    i,ii,rrd.cdp_prep[iii].scratch[CDP_val].u_val);
+#endif
+				  } else {
+					 switch (current_cf) {
+					 case CF_AVERAGE:
+					    rrd.cdp_prep[iii].scratch[CDP_val].u_val += pdp_temp[ii] *
+						   elapsed_pdp_st;
+						break;
+					 case CF_MINIMUM:
+						if (pdp_temp[ii] < rrd.cdp_prep[iii].scratch[CDP_val].u_val)
+						   rrd.cdp_prep[iii].scratch[CDP_val].u_val = pdp_temp[ii];
+						break; 
+					 case CF_MAXIMUM:
+						if (pdp_temp[ii] > rrd.cdp_prep[iii].scratch[CDP_val].u_val)
+						   rrd.cdp_prep[iii].scratch[CDP_val].u_val = pdp_temp[ii];
+						break; 
+					 case CF_LAST:
+					 default:
+						rrd.cdp_prep[iii].scratch[CDP_val].u_val = pdp_temp[ii];
+						break;
+					 }
+				  }
+			   }
+			} else { /* rrd.rra_def[i].pdp_cnt == 1 */
+			   if (elapsed_pdp_st > 2)
+			   {
+				   switch (current_cf) {
+				   case CF_AVERAGE:
+				   default:
+			          rrd.cdp_prep[iii].scratch[CDP_primary_val].u_val=pdp_temp[ii];
+			          rrd.cdp_prep[iii].scratch[CDP_secondary_val].u_val=pdp_temp[ii];
+					  break;
+                   case CF_SEASONAL:
+				   case CF_DEVSEASONAL:
+					  /* need to update cached seasonal values, so they are consistent
+					   * with the bulk update */
+                      /* WARNING: code relies on the fact that CDP_hw_last_seasonal and
+					   * CDP_last_deviation are the same. */
+                      rrd.cdp_prep[iii].scratch[CDP_hw_last_seasonal].u_val =
+						 last_seasonal_coef[ii];
+					  rrd.cdp_prep[iii].scratch[CDP_hw_seasonal].u_val =
+						 seasonal_coef[ii];
+					  break;
+                   case CF_HWPREDICT:
+					  /* need to update the null_count and last_null_count.
+					   * even do this for non-DNAN pdp_temp because the
+					   * algorithm is not learning from batch updates. */
+					  rrd.cdp_prep[iii].scratch[CDP_null_count].u_cnt += 
+						 elapsed_pdp_st;
+					  rrd.cdp_prep[iii].scratch[CDP_last_null_count].u_cnt += 
+						 elapsed_pdp_st - 1;
+					  /* fall through */
+				   case CF_DEVPREDICT:
+			          rrd.cdp_prep[iii].scratch[CDP_primary_val].u_val = DNAN;
+			          rrd.cdp_prep[iii].scratch[CDP_secondary_val].u_val = DNAN;
+					  break;
+                   case CF_FAILURES:
+					  /* do not count missed bulk values as failures */
+			          rrd.cdp_prep[iii].scratch[CDP_primary_val].u_val = 0;
+			          rrd.cdp_prep[iii].scratch[CDP_secondary_val].u_val = 0;
+					  /* need to reset violations buffer.
+					   * could do this more carefully, but for now, just
+					   * assume a bulk update wipes away all violations. */
+                      erase_violations(&rrd, iii, i);
+					  break;
+				   }
+			   } 
+			} /* endif rrd.rra_def[i].pdp_cnt == 1 */
+
+			if (rrd_test_error()) break;
+
+			} /* endif data sources loop */
+        } /* end RRA Loop */
+
+		/* this loop is only entered if elapsed_pdp_st < 3 */
+		for (j = elapsed_pdp_st, scratch_idx = CDP_primary_val; 
+			 j > 0 && j < 3; j--, scratch_idx = CDP_secondary_val)
+		{
+	       for(i = 0, rra_start = rra_begin;
+		   i < rrd.stat_head->rra_cnt;
+	       rra_start += rrd.rra_def[i].row_cnt * rrd.stat_head -> ds_cnt * sizeof(rrd_value_t),
+		   i++)
+		   {
+			  if (rrd.rra_def[i].pdp_cnt > 1) continue;
+
+	          current_cf = cf_conv(rrd.rra_def[i].cf_nam);
+			  if (current_cf == CF_SEASONAL || current_cf == CF_DEVSEASONAL)
+			  {
+		         lookup_seasonal(&rrd,i,rra_start,rrd_file,
+				    elapsed_pdp_st + (scratch_idx == CDP_primary_val ? 1 : 2),
+			        &seasonal_coef);
+			  }
+			  if (rrd_test_error()) break;
+		      /* loop over data soures within each RRA */
+		      for(ii = 0;
+			  ii < rrd.stat_head->ds_cnt;
+			  ii++)
+			  {
+			     update_aberrant_CF(&rrd,pdp_temp[ii],current_cf,
+					i*(rrd.stat_head->ds_cnt) + ii,i,ii,
+				    scratch_idx, seasonal_coef);
+			  }
+           } /* end RRA Loop */
+		   if (rrd_test_error()) break;
+	    } /* end elapsed_pdp_st loop */
+
+		if (rrd_test_error()) break;
+
+		/* Ready to write to disk */
+		/* Move sequentially through the file, writing one RRA at a time.
+		 * Note this architecture divorces the computation of CDP with
+		 * flushing updated RRA entries to disk. */
+	    for(i = 0, rra_start = rra_begin;
+		i < rrd.stat_head->rra_cnt;
+	    rra_start += rrd.rra_def[i].row_cnt * rrd.stat_head -> ds_cnt * sizeof(rrd_value_t),
+		i++) {
+		/* is there anything to write for this RRA? If not, continue. */
+        if (rra_step_cnt[i] == 0) continue;
+
+		/* write the first row */
+#ifdef DEBUG
+        fprintf(stderr,"  -- RRA Preseek %ld\n",ftell(rrd_file));
+#endif
+	    rrd.rra_ptr[i].cur_row++;
+	    if (rrd.rra_ptr[i].cur_row >= rrd.rra_def[i].row_cnt)
+		   rrd.rra_ptr[i].cur_row = 0; /* wrap around */
+		/* positition on the first row */
+		rra_pos_tmp = rra_start +
+		   (rrd.stat_head->ds_cnt)*(rrd.rra_ptr[i].cur_row)*sizeof(rrd_value_t);
+		if(rra_pos_tmp != rra_current) {
+		   if(fseek(rrd_file, rra_pos_tmp, SEEK_SET) != 0){
+		      rrd_set_error("seek error in rrd");
+		      break;
+		   }
+		   rra_current = rra_pos_tmp;
+		}
+#ifdef DEBUG
+	    fprintf(stderr,"  -- RRA Postseek %ld\n",ftell(rrd_file));
+#endif
+		scratch_idx = CDP_primary_val;
+		write_RRA_row(&rrd, i, &rra_current, scratch_idx, rrd_file);
+		if (rrd_test_error()) break;
+
+		/* write other rows of the bulk update, if any */
+		scratch_idx = CDP_secondary_val;
+		for ( ; rra_step_cnt[i] > 1; 
+		     rra_step_cnt[i]--, rrd.rra_ptr[i].cur_row++)
+		{
+		   if (rrd.rra_ptr[i].cur_row == rrd.rra_def[i].row_cnt)
+		   {
+#ifdef DEBUG
+              fprintf(stderr,"Wraparound for RRA %s, %lu updates left\n",
+			  rrd.rra_def[i].cf_nam, rra_step_cnt[i] - 1);
+#endif
+			  /* wrap */
+			  rrd.rra_ptr[i].cur_row = 0;
+			  /* seek back to beginning of current rra */
+		      if (fseek(rrd_file, rra_start, SEEK_SET) != 0)
+			  {
+		         rrd_set_error("seek error in rrd");
+		         break;
+			  }
+#ifdef DEBUG
+	          fprintf(stderr,"  -- Wraparound Postseek %ld\n",ftell(rrd_file));
+#endif
+			  rra_current = rra_start;
+		   }
+		   write_RRA_row(&rrd, i, &rra_current, scratch_idx, rrd_file);
+		}
+		
+		if (rrd_test_error())
+		  break;
+		} /* RRA LOOP */
+
+	    /* break out of the argument parsing loop if error_string is set */
+	    if (rrd_test_error()){
+		   free(step_start);
+		   break;
+	    } 
+	    
+	} /* endif a pdp_st has occurred */ 
+	rrd.live_head->last_up = current_time;
+	free(step_start);
+    } /* function argument loop */
+
+    if (seasonal_coef != NULL) free(seasonal_coef);
+    if (last_seasonal_coef != NULL) free(last_seasonal_coef);
+	if (rra_step_cnt != NULL) free(rra_step_cnt);
+
+    /* if we got here and if there is an error and if the file has not been
+     * written to, then close things up and return. */
+    if (rrd_test_error()) {
+	free(updvals);
+	free(tmpl_idx);
+	rrd_free(&rrd);
+	free(pdp_temp);
+	free(pdp_new);
+        fclose(rrd_file);
+	return(-1);
+    }
+
+    /* aargh ... that was tough ... so many loops ... anyway, its done.
+     * we just need to write back the live header portion now*/
+
+    if (fseek(rrd_file, (sizeof(stat_head_t)
+			 + sizeof(ds_def_t)*rrd.stat_head->ds_cnt 
+			 + sizeof(rra_def_t)*rrd.stat_head->rra_cnt),
+	      SEEK_SET) != 0) {
+	rrd_set_error("seek rrd for live header writeback");
+	free(updvals);
+	free(tmpl_idx);
+	rrd_free(&rrd);
+	free(pdp_temp);
+	free(pdp_new);
+        fclose(rrd_file);
+	return(-1);
+    }
+
+    if(fwrite( rrd.live_head,
+	       sizeof(live_head_t), 1, rrd_file) != 1){
+	rrd_set_error("fwrite live_head to rrd");
+	free(updvals);
+	rrd_free(&rrd);
+	free(tmpl_idx);
+	free(pdp_temp);
+	free(pdp_new);
+        fclose(rrd_file);
+	return(-1);
+    }
+
+    if(fwrite( rrd.pdp_prep,
+	       sizeof(pdp_prep_t),
+	       rrd.stat_head->ds_cnt, rrd_file) != rrd.stat_head->ds_cnt){
+	rrd_set_error("ftwrite pdp_prep to rrd");
+	free(updvals);
+	rrd_free(&rrd);
+	free(tmpl_idx);
+	free(pdp_temp);
+	free(pdp_new);
+        fclose(rrd_file);
+	return(-1);
+    }
+
+    if(fwrite( rrd.cdp_prep,
+	       sizeof(cdp_prep_t),
+	       rrd.stat_head->rra_cnt *rrd.stat_head->ds_cnt, rrd_file) 
+       != rrd.stat_head->rra_cnt *rrd.stat_head->ds_cnt){
+
+	rrd_set_error("ftwrite cdp_prep to rrd");
+	free(updvals);
+	free(tmpl_idx);
+	rrd_free(&rrd);
+	free(pdp_temp);
+	free(pdp_new);
+        fclose(rrd_file);
+	return(-1);
+    }
+
+    if(fwrite( rrd.rra_ptr,
+	       sizeof(rra_ptr_t), 
+	       rrd.stat_head->rra_cnt,rrd_file) != rrd.stat_head->rra_cnt){
+	rrd_set_error("fwrite rra_ptr to rrd");
+	free(updvals);
+	free(tmpl_idx);
+	rrd_free(&rrd);
+	free(pdp_temp);
+	free(pdp_new);
+        fclose(rrd_file);
+	return(-1);
+    }
+
+    /* OK now close the files and free the memory */
+    if(fclose(rrd_file) != 0){
+	rrd_set_error("closing rrd");
+	free(updvals);
+	free(tmpl_idx);
+	rrd_free(&rrd);
+	free(pdp_temp);
+	free(pdp_new);
+	return(-1);
+    }
+
+    /* calling the smoothing code here guarantees at most
+ 	 * one smoothing operation per rrd_update call. Unfortunately,
+	 * it is possible with bulk updates, or a long-delayed update
+	 * for smoothing to occur off-schedule. This really isn't
+	 * critical except during the burning cycles. */
+	if (schedule_smooth)
+	{
+#ifndef WIN32
+	  rrd_file = fopen(argv[optind],"r+");
+#else
+	  rrd_file = fopen(argv[optind],"rb+");
+#endif
+	  rra_start = rra_begin;
+	  for (i = 0; i < rrd.stat_head -> rra_cnt; ++i)
+	  {
+	    if (cf_conv(rrd.rra_def[i].cf_nam) == CF_DEVSEASONAL ||
+		cf_conv(rrd.rra_def[i].cf_nam) == CF_SEASONAL)
+	    {
+#ifdef DEBUG
+	      fprintf(stderr,"Running smoother for rra %ld\n",i);
+#endif
+	      apply_smoother(&rrd,i,rra_start,rrd_file);
+	      if (rrd_test_error())
+		break;
+	    }
+	    rra_start += rrd.rra_def[i].row_cnt
+	      *rrd.stat_head->ds_cnt*sizeof(rrd_value_t);
+	  }
+	  fclose(rrd_file);
+	}
+    rrd_free(&rrd);
+    free(updvals);
+    free(tmpl_idx);
+    free(pdp_new);
+    free(pdp_temp);
+    return(0);
+}
+
+/*
+ * get exclusive lock to whole file.
+ * lock gets removed when we close the file
+ *
+ * returns 0 on success
+ */
+int
+LockRRD(FILE *rrdfile)
+{
+    int	rrd_fd;		/* File descriptor for RRD */
+    int			stat;
+
+    rrd_fd = fileno(rrdfile);
+
+	{
+#ifndef WIN32    
+		struct flock	lock;
+    lock.l_type = F_WRLCK;    /* exclusive write lock */
+    lock.l_len = 0;	      /* whole file */
+    lock.l_start = 0;	      /* start of file */
+    lock.l_whence = SEEK_SET;   /* end of file */
+
+    stat = fcntl(rrd_fd, F_SETLK, &lock);
+#else
+		struct _stat st;
+
+		if ( _fstat( rrd_fd, &st ) == 0 ) {
+			stat = _locking ( rrd_fd, _LK_NBLCK, st.st_size );
+		} else {
+			stat = -1;
+		}
+#endif
+	}
+
+    return(stat);
+}
+
+
+void
+write_RRA_row (rrd_t *rrd, unsigned long rra_idx, unsigned long *rra_current,
+   unsigned short CDP_scratch_idx, FILE *rrd_file)
+{
+   unsigned long ds_idx, cdp_idx;
+
+   for (ds_idx = 0; ds_idx < rrd -> stat_head -> ds_cnt; ds_idx++)
+   {
+      /* compute the cdp index */
+      cdp_idx =rra_idx * (rrd -> stat_head->ds_cnt) + ds_idx;
+#ifdef DEBUG
+	  fprintf(stderr,"  -- RRA WRITE VALUE %e, at %ld CF:%s\n",
+	     rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val,ftell(rrd_file),
+	     rrd -> rra_def[rra_idx].cf_nam);
+#endif
+
+	  if(fwrite(&(rrd -> cdp_prep[cdp_idx].scratch[CDP_scratch_idx].u_val),
+		 sizeof(rrd_value_t),1,rrd_file) != 1)
+	  { 
+	     rrd_set_error("writing rrd");
+		 return;
+	  }
+	  *rra_current += sizeof(rrd_value_t);
+	}
+}