Content deleted Content added
m →History: Added reference to first use of dynamics and filtering over time with Data Reconciliation ~~~~ |
Rescuing 1 sources and tagging 0 as dead.) #IABot (v2.0.9.5 |
||
| (47 intermediate revisions by 33 users not shown) | |||
Line 1:
{{Short description|A technology to correct measurements in industrial processes}}
'''Industrial process data validation and reconciliation''', or more briefly, '''process data ==Models, data and measurement errors==
Line 5 ⟶ 6:
# Models that express the general structure of the processes,
# Data that reflects the state of the processes at a given point in time.
Models can have different levels of detail, for example one can incorporate simple mass or compound conservation balances, or more advanced thermodynamic models including energy conservation laws. Mathematically the model can be expressed by a [[nonlinear system|nonlinear system of equations]] <math
===Error types===
<gallery caption="Random and systematic errors" widths="
File:Normal_no_bias.jpg|Normally distributed measurements without bias.
File:Normal_with_bias.jpg|Normally distributed measurements with bias.
</gallery>
Data originates typically from [[measurements]] taken at different places throughout the industrial site, for example temperature, pressure, volumetric flow rate measurements etc. To understand the basic principles of
# [[random error]]s due to intrinsic [[sensor]] [[accuracy]] and
# [[systematic errors]] (or gross errors) due to sensor [[calibration]] or faulty data transmission.
[[Random error]]s means that the measurement <math
Other sources of errors when calculating plant balances include process faults such as leaks, unmodeled heat losses, incorrect physical properties or other physical parameters used in equations, and incorrect structure such as unmodeled bypass lines. Other errors include unmodeled plant dynamics such as holdup changes, and other instabilities in plant operations that violate steady state (algebraic) models. Additional dynamic errors arise when measurements and samples are not taken at the same time, especially lab analyses.
The normal practice of using time averages for the data input partly reduces the dynamic problems. However, that does not completely resolve timing inconsistencies for infrequently-sampled data like lab analyses.
This use of average values, like a [[moving average]], acts as a [[low-pass filter]], so high frequency noise is mostly eliminated. The result is that, in practice, data reconciliation is mainly making adjustments to correct systematic errors like biases.
===Necessity of removing measurement errors===
Line 27 ⟶ 32:
==History==
,<ref name="Stanley-Mah-1977">G.M. Stanley and R.S.H. Mah, [http://gregstanleyandassociates.com/AIChEJ-1977-EstimationInProcessNetworks.pdf ''Estimation of Flows and Temperatures in Process Networks'', AIChE Journal 23: 642–650, 1977.]</ref><ref>P. Joris, B. Kalitventzeff, ''Process measurements analysis and validation'', Proc. CEF’87: Use Comput. Chem. Eng., Italy, 41–46, 1987.</ref> Quasi steady state dynamics for filtering and simultaneous parameter estimation over time were introduced in 1977 by Stanley and Mah ==Data reconciliation==
Line 43 ⟶ 49:
where
<math
<math
<math
The term <math>\left(\frac{y_i^*-y_i}{\sigma_i}\right)^2\,\!</math> is called the ''penalty'' of measurement ''i''. The objective function is the sum of the penalties, which will be denoted in the following by <math>f(y^*)=\sum_{i=1}^n\left(\frac{y_i^*-y_i}{\sigma_i}\right)^2</math>.
In other words, one wants to minimize the overall correction (measured in the least squares term) that is needed in order to satisfy the [[constraint (mathematics)|system constraints]]. Additionally, each least squares term is weighted by the [[standard deviation]] of the corresponding measurement. The standard deviation is related to the accuracy of the measurement. For example, at a 95% confidence level, the standard deviation is about half the accuracy.
===Redundancy===
<gallery caption="Sensor and topological redundancy"
File:sensor_red.jpg|Sensor redundancy arising from multiple sensors of the same quantity at the same time at the same place.
File:topological_red.jpg|Topological redundancy arising from model information, using the mass conservation constraint <math
</gallery>
Data reconciliation
Topological redundancy is intimately linked with the [[degrees of freedom (physics and chemistry)|degree of freedom]] (<math style="vertical-align:-25%;">dof\,\!</math>) of a mathematical system,<ref name="vdi">VDI-Gesellschaft Energie und Umwelt, "Guidelines - VDI 2048 Blatt 1 - Uncertainties of measurements at acceptance tests for energy conversion and power plants - Fundamentals", ''[http://www.vdi.de/401.0.html Association of German Engineers]'', 2000.</ref> i.e. the minimum number of pieces of information (i.e. measurements) that are required in order to calculate all of the system variables. For instance, in the example above the flow conservation requires that <math style="vertical-align:-10%;">a=b+c\,</math>, and it is clear that one needs to know the value of two of the 3 variables in order to calculate the third one. Therefore the degree of freedom in that case is equal to 2.▼
Redundancy can be due to [[redundancy (engineering)|sensor redundancy]], where sensors are duplicated in order to have more than one measurement of the same quantity. Redundancy also arises when a single variable can be estimated in several independent ways from separate sets of measurements at a given time or time averaging period, using the algebraic constraints.
When speaking about topological redundancy we have to distinguish between measured and unmeasured variables. In the following let us denote by <math style="vertical-align:-0%;">x\,\!</math> the unmeasured variables and <math style="vertical-align:-30%;">y\,\!</math> the measured variables. Then the system of the process constraints becomes <math style="vertical-align:-25%;">F(x,y)=0\,\!</math>, which is a nonlinear system in <math style="vertical-align:-30%;">y\,\!</math> and <math style="vertical-align:-0%;">x\,\!</math>.▼
If the system <math style="vertical-align:-25%;">F(x,y)=0\,\!</math> is calculable with the <math style="vertical-align:-0%;">n\,</math> measurements given, then the level of topological redundancy is defined as <math style="vertical-align:-25%;">red= n - dof\,\!</math>, i.e. the number of additional measurements that are at hand on top of those measurements which are required in order to just calculate the system. Another way of viewing the level of redundancy is to use the definition of <math style="vertical-align:-20%;">dof\,</math>, which is the difference between the number of variables (measured and unmeasured) and the number of equations. Then one gets▼
Redundancy is linked to the concept of [[observability]]. A variable (or system) is observable if the models and sensor measurements can be used to uniquely determine its value (system state). A sensor is redundant if its removal causes no loss of observability. Rigorous definitions of observability, calculability, and redundancy, along with criteria for determining it, were established by Stanley and Mah,<ref name="Stanley-Mah-1981a">
[https://gregstanleyandassociates.com/CES-1981a-ObservabilityRedundancy.pdf Stanley G.M. and Mah, R.S.H., "Observability and Redundancy in Process Data Estimation, Chem. Engng. Sci. 36, 259 (1981)]</ref> for these cases with set constraints such as algebraic equations and inequalities. Next, we illustrate some special cases:
▲Topological redundancy is intimately linked with the [[degrees of freedom (physics and chemistry)|
▲When speaking about topological redundancy we have to distinguish between measured and unmeasured variables. In the following let us denote by <math
▲If the system <math
:<math>\begin{align}
Line 66 ⟶ 78:
\end{align}</math>
i.e. the redundancy is the difference between the number of equations <math
Simple counts of variables, equations, and measurements are inadequate for many systems, breaking down for several reasons: (a) Portions of a system might have redundancy, while others do not, and some portions might not even be possible to calculate, and (b) Nonlinearities can lead to different conclusions at different operating points. As an example, consider the following system with 4 streams and 2 units.
====Example of calculable and non-calculable systems====
<gallery caption="Calculable and non-calculable systems"
File:calculable_system.jpg|Calculable system, from <math
File:uncalculable_system.jpg|non-calculable system, knowing <math
</gallery>
Let us consider a small system with 4 streams and 2 units. We incorporate only flow conservation constraints and obtain <math style="vertical-align:-10%;">a+b=c\,\!</math> and <math style="vertical-align:-0%;">c=d\,\!</math>. If we have measurements for <math style="vertical-align:-0%;">c\,\!</math> and <math style="vertical-align:-0%;">d\,\!</math>, but not for <math style="vertical-align:-0%;">a\,\!</math> and <math style="vertical-align:-0%;">b\,\!</math>, then the system cannot be calculated (knowing <math style="vertical-align:-0%;">c\,\!</math> does not give information about <math style="vertical-align:-0%;">a\,\!</math> and <math style="vertical-align:-0%;">b\,\!</math>). On the other hand, if <math style="vertical-align:-0%;">a\,\!</math> and <math style="vertical-align:-0%;">c\,\!</math> are known, but not <math style="vertical-align:-0%;">b\,\!</math> and <math style="vertical-align:-0%;">d\,\!</math>, then the system can be calculated.▼
We incorporate only flow conservation constraints and obtain <math>a+b=c\,\!</math> and <math>c=d\,\!</math>. It is possible that the system <math>F(x,y)=0\,\!</math> is not calculable, even though <math>p-m\ge 0\,\!</math>.
▲
In 1981, observability and redundancy criteria were proven for these sorts of flow networks involving only mass and energy balance constraints.<ref name="Stanley-Mah-1981b">[https://gregstanleyandassociates.com/CES-1981b-ObservabilityRedundancyProcessNetworks.pdf Stanley G.M., and Mah R.S.H., "Observability and Redundancy Classification in Process Networks", Chem. Engng. Sci. 36, 1941 (1981) ]</ref> After combining all the plant inputs and outputs into an "environment node", loss of observability corresponds to cycles of unmeasured streams. That is seen in the second case above, where streams a and b are in a cycle of unmeasured streams. Redundancy classification follows, by testing for a path of unmeasured streams, since that would lead to an unmeasured cycle if the measurement was removed. Measurements c and d are redundant in the second case above, even though part of the system is unobservable.
===Benefits===
Line 92 ⟶ 111:
* the individual test.
If no gross errors exist in the set of measured values, then each penalty term in the objective function is a [[normal distribution|random variable]] that is normally distributed with mean equal to 0 and variance equal to 1. By consequence, the objective function is a random variable which follows a [[chi-square distribution]], since it is the sum of the square of normally distributed random variables. Comparing the value of the objective function <math
The individual test compares each penalty term in the objective function with the critical values of the normal distribution. If the <math>i</math>-th penalty term is outside the 95% confidence interval of the normal distribution, then there is reason to believe that this measurement has a gross error.
==Advanced process data
Advanced process data
* complex models incorporating besides mass balances also thermodynamics, momentum balances, equilibria constraints, hydrodynamics etc.
* gross error remediation techniques to ensure meaningfulness of the reconciled values,
Line 103 ⟶ 122:
===Thermodynamic models===
Simple models include mass balances only. When adding thermodynamic constraints such as [[
===Gross error remediation===
[[image:scheme reconciliation.jpg|thumb|350px|The workflow of an advanced data validation and reconciliation process.]]
Gross errors are measurement systematic errors that may [[bias]] the reconciliation results. Therefore, it is important to identify and eliminate these gross errors from the reconciliation process. After the reconciliation [[statistical tests]] can be applied that indicate whether or not a gross error does exist somewhere in the set of measurements. These techniques of gross error remediation are based on two concepts:
* gross error elimination
* gross error relaxation.
Line 114 ⟶ 133:
Gross error relaxation targets at relaxing the estimate for the uncertainty of suspicious measurements so that the reconciled value is in the 95% confidence interval. Relaxation typically finds application when it is not possible to determine which measurement around one unit is responsible for the gross error (equivalence of gross errors). Then measurement uncertainties of the measurements involved are increased.
It is important to note that the remediation of gross errors reduces the quality of the reconciliation, either the redundancy decreases (elimination) or the uncertainty of the measured data increases (relaxation). Therefore, it can only be applied when the initial level of redundancy is high enough to ensure that the data reconciliation can still be done (see Section 2,<ref name="vdi" />).
===Workflow===
Advanced
# data acquisition from data historian, data base or manual inputs
# data validation and filtering of raw measurements
Line 125 ⟶ 144:
#* gross error remediation (and go back to step 3)
# result storage (raw measurements together with reconciled values)
The result of an advanced
==Applications==
As
==See also==
Line 137 ⟶ 156:
* [[Industrial processes]]
* [[Chemical engineering]]
==References==
Line 155 ⟶ 161:
{{Reflist}}
* Alexander, Dave, Tannar, Dave & Wasik, Larry "Mill Information System uses Dynamic Data Reconciliation for Accurate Energy Accounting" TAPPI Fall Conference 2007.[http://www.tappi.org/Downloads/Conference-Papers/2007/07EPE/07epe87.aspx]{{Dead link|date=July 2019 |bot=InternetArchiveBot |fix-attempted=yes }}
* Rankin, J. & Wasik, L. "Dynamic Data Reconciliation of Batch Pulping Processes (for On-Line Prediction)" PAPTAC Spring Conference 2009.
* S. Narasimhan, C. Jordache, ''Data reconciliation and gross error detection: an intelligent use of process data'', Golf Publishing Company, Houston, 2000.
* V. Veverka, F. Madron, ''Material and Energy Balancing in the Process Industries'', Elsevier Science BV, Amsterdam, 1997.
* J. Romagnoli, M.C. Sanchez, ''Data processing and reconciliation for chemical process operations'', Academic Press, 2000.
[[Category:Data management]]
| |||