Data validation and reconciliation: Difference between revisions

Content deleted Content added
Changed title of paragraph to better match discussion in the paragraph. Paragraph was titled "Advanced Data Valdiiaton and Reconsiliation", but as stated in the first sentence, the paragraph is about Advanced Process Data Reconciliation (PDR)
Rescuing 1 sources and tagging 0 as dead.) #IABot (v2.0.9.5
 
(8 intermediate revisions by 8 users not shown)
Line 1:
{{Short description|A technology to correct measurements in industrial processes}}
'''Industrial process data validation and reconciliation''', or more briefly, '''process data reconciliation (PDR)''', is a technology that uses process information and mathematical methods in order to automatically ensure [[data validation]] and reconciliation by correcting measurements in industrial processes. The use of PDR allows for extracting accurate and reliable information about the state of industry processes from raw measurement [[data]] and produces a single consistent set of data representing the most likely process operation.
 
Line 12 ⟶ 13:
File:Normal_with_bias.jpg|Normally distributed measurements with bias.
</gallery>
Data originates typically from [[measurements]] taken at different places throughout the industrial site, for example temperature, pressure, volumetric flow rate measurements etc. To understand the basic principles of DVRPDR, it is important to first recognize that plant measurements are never 100% correct, i.e. raw measurement <math>y\,</math> is not a solution of the nonlinear system <math>F(y)=0\,\!</math>. When using measurements without correction to generate plant balances, it is common to have incoherencies. [[Observational error|Measurement errors]] can be categorized into two basic types:
# [[random error]]s due to intrinsic [[sensor]] [[accuracy]] and
# [[systematic errors]] (or gross errors) due to sensor [[calibration]] or faulty data transmission.
Line 68 ⟶ 69:
[https://gregstanleyandassociates.com/CES-1981a-ObservabilityRedundancy.pdf Stanley G.M. and Mah, R.S.H., "Observability and Redundancy in Process Data Estimation, Chem. Engng. Sci. 36, 259 (1981)]</ref> for these cases with set constraints such as algebraic equations and inequalities. Next, we illustrate some special cases:
 
Topological redundancy is intimately linked with the [[degrees of freedom (physics and chemistry)|degrees of freedom]] (<math>dof\,\!</math>) of a mathematical system,<ref name="vdi">VDI-Gesellschaft Energie und Umwelt, "Guidelines - VDI 2048 Blatt 1 - “Control and quality improvement of process data and their uncertainties by means of correction calculation for operation and acceptance tests”; VDI 2048 Part 1; September 2017", ''[http://www.vdi.de/401.0.html Association of German Engineers] {{Webarchive|url=https://web.archive.org/web/20100325223512/http://www.vdi.de/401.0.html |date=2010-03-25 }}'', 2017.</ref> i.e. the minimum number of pieces of information (i.e. measurements) that are required in order to calculate all of the system variables. For instance, in the example above the flow conservation requires that <math>a=b+c\,</math>. One needs to know the value of two of the 3 variables in order to calculate the third one. The degrees of freedom for the model in that case is equal to 2. At least 2 measurements are needed to estimate all the variables, and 3 would be needed for redundancy.
 
When speaking about topological redundancy we have to distinguish between measured and unmeasured variables. In the following let us denote by <math>x\,\!</math> the unmeasured variables and <math>y\,\!</math> the measured variables. Then the system of the process constraints becomes <math>F(x,y)=0\,\!</math>, which is a nonlinear system in <math>y\,\!</math> and <math>x\,\!</math>.
Line 89 ⟶ 90:
We incorporate only flow conservation constraints and obtain <math>a+b=c\,\!</math> and <math>c=d\,\!</math>. It is possible that the system <math>F(x,y)=0\,\!</math> is not calculable, even though <math>p-m\ge 0\,\!</math>.
 
If we have measurements for <math>c\,\!</math> and <math>d\,\!</math>, but not for <math>a\,\!</math> and <math>b\,\!</math>, then the system cannot be calculated (knowing <math>c\,\!</math> does not give information about <math>a\,\!</math> and <math>b\,\!</math>). On the other hand, if <math>a\,\!</math> and <math>cd\,\!</math> are known, but not <math>b\,\!</math> and <math>dc\,\!</math>, then the system can be calculated.
 
In 1981, observability and redundancy criteria were proven for these sorts of flow networks involving only mass and energy balance constraints.<ref name="Stanley-Mah-1981b">[https://gregstanleyandassociates.com/CES-1981b-ObservabilityRedundancyProcessNetworks.pdf Stanley G.M., and Mah R.S.H., "Observability and Redundancy Classification in Process Networks", Chem. Engng. Sci. 36, 1941 (1981) ]</ref> After combining all the plant inputs and outputs into an "environment node", loss of observability corresponds to cycles of unmeasured streams. That is seen in the second case above, where streams a and b are in a cycle of unmeasured streams. Redundancy classification follows, by testing for a path of unmeasured streams, since that would lead to an unmeasured cycle if the measurement was removed. Measurements c and d are redundant in the second case above, even though part of the system is unobservable.
Line 163 ⟶ 164:
* Rankin, J. & Wasik, L. "Dynamic Data Reconciliation of Batch Pulping Processes (for On-Line Prediction)" PAPTAC Spring Conference 2009.
* S. Narasimhan, C. Jordache, ''Data reconciliation and gross error detection: an intelligent use of process data'', Golf Publishing Company, Houston, 2000.
* V. Veverka, F. Madron, ''Material and Energy Balancing in the Process Industries'', Elsevier Science BV, Amsterdam, 1997.
* J. Romagnoli, M.C. Sanchez, ''Data processing and reconciliation for chemical process operations'', Academic Press, 2000.