Smoothing problem (stochastic processes): Difference between revisions

Revision as of 10:12, 23 September 2021

Latest revision as of 02:00, 14 January 2025

(6 intermediate revisions by 5 users not shown)

Line 1:

{{technical|date=November 2017}}

The '''Smoothingsmoothing problem''' (not to be confused with [[smoothing]] in [[statistics]], [[image processing]] and other contexts) refers to [[Recursive Bayesian estimation]] also known as [[Bayes filter]] is the problem of [[density estimation|estimating]] an unknown [[probability density function]] recursively over time using incremental incoming measurements. It is one of the main problems defined by [[Norbert Wiener]].<ref name="wiener-report">1942, ''Extrapolation, Interpolation and Smoothing of Stationary Time Series''. A war-time classified report nicknamed "the yellow peril" because of the color of the cover and the difficulty of the subject. Published postwar 1949 [[MIT Press]]. http://www.isss.org/lumwiener.htm {{Webarchive|url=https://web.archive.org/web/20150816041622/http://www.isss.org/lumwiener.htm |date=2015-08-16 }}</ref><ref name="wiener-book">{{cite book |last=Wiener |first=Norbert |author-link=Norbert Wiener |year=1949 |title=Extrapolation, Interpolation, and Smoothing of Stationary Time Series: With Engineering Applications |url=https://direct.mit.edu/books/oa-monograph/4361/Extrapolation-Interpolation-and-Smoothing-of |publisher=[[MIT Press]] |isbn=9780262257190}}</ref> A '''smoother''' is an algorithm that implements a solution to this problem, typically based on [[recursive Bayesian estimation]]. The smoothing problem is closely related to the [[filtering problem]], both of which are studied in Bayesian smoothing theory.

<ref name="wiener-report">1942, ''Extrapolation, Interpolation and Smoothing of Stationary Time Series''. A war-time classified report nicknamed "the yellow peril" because of the color of the cover and the difficulty of the subject. Published postwar 1949 [[MIT Press]]. http://www.isss.org/lumwiener.htm</ref><ref name="wiener-book">Wiener, Norbert (1949). Extrapolation, Interpolation, and Smoothing of Stationary Time Series. New York: Wiley. {{ISBN|0-262-73005-7}}.</ref>

A smoother is often a two-pass process, composed of forward and backward passes. Consider doing estimation (prediction/retrodiction) about an ongoing process (e.g. tracking a missile) based on incoming observations. When new observations arrive, estimations about past needs to be updated to have a smoother (more accurate) estimation of the whole estimated path until now (taking into account the newer observations). Without a backward pass (for [[retrodiction]]), the sequence of predictions in an online filtering algorithm does not look smooth. In other words, retrospectively, it is as if we are using future observations for improving estimation of a point in past, when those observations about future points become available. Note that time of estimation (which determines which observations are available) can be different to the time of the point that the prediction is about (that is subject to prediction/retrodiction). The observations about later times can be used to update and improved the estimations about earlier times. Doing so leads to smoother-looking estimations (retrodiction) about the whole path.

A '''smoother''' is an algorithm or implementation that implements a solution to such problem. Please refer to the article [[Recursive Bayesian estimation]] for more information.

The [[Smoothing problem]] and [[Filtering problem]] are often considered a closely related pair of problems. They are studied in Bayesian smoothing theory.

Note: Not to be confused with blurring and smoothing using methods such as moving average. See [[smoothing]].

==Example smoothers ==

==ExampleExamples of smoothers ==

Some variants include:<ref name="Sarkka-book">Simo Särkkä. Bayesian Filtering and Smoothing. Publisher: Cambridge University Press (5 Sept. 2013)

Language: English

Line 21 ⟶ 16:

== The confusion in terms and the relation between Filtering and Smoothing problems==

{{Cleanup section|reason=this section needs reorganization and also needs additional citations.|date=December 2021}}

ThereThe terms Smoothing and Filtering are used for four termsconcepts that causemay initially be confusionconfusing: Smoothing (in two senses: estimation and convolution), and Filtering (again in two senses: estimation and convolution).

Smoothing (estimation) and smoothing (convolution) despite being labelled with the same name in English language, can mean totally different, butmathematical soundprocedures. likeThe theyrequirements areof apparentlyproblems similar.they The conceptssolve are different. andThese concepts are useddistinguished inby almostthe differentcontext historical(signal contexts.processing Theversus '''requirements'''estimation areof verystochastic differentprocesses).

NoteThe historical reason for this confusion is that initially, the Wiener's suggested a "smoothing" filter that was just a convolution,. butLater theon his proposed solutions for obtaining a smoother estimation laterseparate developments wereas different:two onedistinct concepts. One was about attaining a smoother estimation by taking into account past observations, and the other one was smoothing using filter design in the sense of (design of a convolution filter. This is a source of confusion).

Both the smoothing problem (in sense of estimation) and the filtering problem (in sense of estimation) are often confused with smoothing and filtering in other contexts (especially non-stochastic signal processing, often a name of various types of convolution). These names are used in the context of World War 2 with problems framed by people like [[Norbert Wiener]].<ref name="wiener-report"/><ref name="wiener-book" /> One source of confusion is the [[Wiener Filter]] is in form of a simple convolution. However, in Wiener's filter, two time-series are given. When the filter is defined, a straightforward convolution is the answer. However, in later developments such as Kalman filtering, the nature of filtering is different to convolution and it deserves a different name.

Line 31 ⟶ 27:

The distinction is described in the following two senses:

1. Convolution: The smoothing in the sense of '''convolution''' is simpler. For example, moving average, low-pass filtering, convolution with a kernel, or blurring using Laplace filters in [[image processing]]. It is often a [[filter design]] problem. Especially non-stochastic and non-Bayesian signal processing, without any hidden variables.

2. Estimation: The '''smoothing problem''' (or Smoothing in the sense of '''estimation''') uses Bayesian and state-space models to estimate the hidden state variables. This is used in the context of World War 2 defined by people like Norbert Wiener, in (stochastic) control theory, radar, signal detection, tracking, etc. The most common use is the Kalman Smoother used with Kalman Filter, which is actually developed by Rauch. The procedure is called Kalman-Rauch recursion.

It is one of the main problems solved by [[Norbert Wiener]].<ref name="wiener-report"/><ref name="wiener-book"/>

Most importantly, in the Filtering problem (sense 2) the information from observation up to the time of the current sample is used. In smoothing (also sense 2) all observation samples (from future) are used. Filtering is causal but smoothing is batch processing of the same problem, namely, estimation of a time-series process based on serial incremental observations.

Line 42 ⟶ 38:

In smoothing all observation samples are used (from future). Filtering is causal, whereas smoothing is batch processing of the given data. Filtering is the estimation of a (hidden) time-series process based on serial incremental observations.

=== RelatedSee conceptsalso ===

* [[Filter (disambiguation)|Filtering]] (disambiguation)

* [[Filtering problem]]

* Not to be confused with [[Filter (signal processing)]]

* [[Kalman filter]], mosta famouswell-known filtering algorithm inrelated theboth senseto ofthe 'filtering problem' and 'the smoothing problem'.

* [[Smoothing]] (not to be confused with the Smoothing problem)

* [[Smoothing (disambiguation)]]

== See also ==

* [[Generalized filtering]]

* [[Smoothing (disambiguation)]]

==References==