Interval predictor model: Difference between revisions

Revision as of 18:14, 19 June 2020

Revision as of 08:47, 19 November 2020

Line 2:

This differs from other techniques in [[machine learning]], where usually one wishes to estimate point values or an entire probability distribution.

Interval Predictor Models are sometimes referred to as a [[nonparametric regression]] technique, because a potentially infinite set of functions are contained by the IPM, and no specific distribution is implied for the regressed variables.

As a consequence of the theory of [[scenario optimization]], in many cases rigorous predictions can be made regarding the performance of the model at test time .<ref name="CampiCalafiore2009">{{cite journal|last1=Campi|first1=M.C.|last2=Calafiore|first2=G.|last3=Garatti|first3=S.|title=Interval predictor models: Identification and reliability|journal=Automatica|volume=45|issue=2|year=2009|pages=382–392|issn=00051098|doi=10.1016/j.automatica.2008.09.004}}</ref>.

Hence an interval predictor model can be seen as a guaranteed bound on [[quantile regression]].

Interval predictor models can also be seen as a way to prescribe the [[Support_(mathematics)#Support_of_a_distribution|support]] of random predictor models, of which a [[Gaussian process]] is a specific case

.<ref name="CrespoKenny2018">{{cite journal|last1=Crespo|first1=Luis G.|last2=Kenny|first2=Sean P.|last3=Giesy|first3=Daniel P.|title=Staircase predictor models for reliability and risk analysis|journal=Structural Safety|volume=75|year=2018|pages=35–44|issn=01674730|doi=10.1016/j.strusafe.2018.05.002}}</ref>.

== Convex interval predictor models ==

Line 11:

Usually the basis is made up of polynomial features or a radial basis is sometimes used.

Then a convex set is assigned to the parameter vector, and the size of the convex set is minimized such that every possible data point can be predicted by one possible value of the parameters.

Ellipsoidal parameters sets were used by Campi (2009), which yield a convex optimization program to train the IPM .<ref name="CampiCalafiore2009" />.

Crespo (2016) proposed the use of a hyperrectangular parameter set, which results in a convenient, linear form for the bounds of the IPM .<ref name="CrespoKenny2016">{{cite journal|last1=Crespo|first1=Luis G.|last2=Kenny|first2=Sean P.|last3=Giesy|first3=Daniel P.|title=Interval Predictor Models With a Linear Parameter Dependency|journal=Journal of Verification, Validation and Uncertainty Quantification|volume=1|issue=2|year=2016|pages=021007|issn=2377-2158|doi=10.1115/1.4032070}}</ref>.

Hence the IPM can be trained with a linear optimization program:

:<math>

Line 20:

The reliability of such an IPM is obtained by noting that for a convex IPM the number of support constraints is less than the dimensionality of the [[trainable parameter]]s, and hence the scenario approach can be applied.

Lacerda (2017) demonstrated that this approach can be extended to situations where the training data is interval valued rather than point valued .<ref name="LacerdaCrespo2017">{{cite book|last1=Lacerda|first1=Marcio J.|title=2017 American Control Conference (ACC)|last2=Crespo|first2=Luis G.|chapter=Interval predictor models for data with measurement uncertainty|year=2017|pages=1487–1492|doi=10.23919/ACC.2017.7963163|isbn=978-1-5090-5992-8|hdl=2060/20170005690}}</ref>.

== Non-convex interval predictor models ==

In Campi (2015) a non-convex theory of scenario optimization was proposed .<ref name="CampiGaratti2015">{{cite book|last1=Campi|first1=Marco C.|title=2015 54th IEEE Conference on Decision and Control (CDC)|last2=Garatti|first2=Simone|last3=Ramponi|first3=Federico A.|chapter=Non-convex scenario optimization with application to system identification|year=2015|pages=4023–4028|doi=10.1109/CDC.2015.7402845|isbn=978-1-4799-7886-1}}</ref>.

This involves measuring the number of support constraints, <math>S</math>, for the Interval Predictor Model after training and hence making predictions about the reliability of the model.

This enables non-convex IPMs to be created, such as a single layer neural network.

Campi (2015) demonstrates that an algorithm where the scenario optimization program is only solved <math>S</math> times which can determine the reliability of the model at test time without a prior evaluation on a validation set .<ref name="CampiGaratti2015" />.

This is achieved by solving the optimisation program

:<math>

Line 34:

where the interval predictor model center line <math> \hat{y}_p(x) = (\overline{y}_p(x) + \underline{y}_p(x)) \times 1/2</math>, and the model width <math> h = (\overline{y}_p(x) - \underline{y}_p(x)) \times 1/2 </math>. This results in an IPM which makes predictions with homoscedastic uncertainty.

Sadeghi (2019) demonstrates that the non-convex scenario approach from Campi (2015) can be extended to train deeper neural networks which predict intervals with hetreoscedastic uncertainty on datasets with imprecision .<ref name="Sadeghi2019">{{cite journal|last1=Sadeghi|first1=Jonathan C.|last2=De Angelis|first2=Marco|last3=Patelli|first3=Edoardo|title=Efficient Training of Interval Neural Networks for Imprecise Training Data|year=2019|journal=Neural Networks|volume=118|pages=338–351|doi=10.1016/j.neunet.2019.07.005|pmid=31369950}}</ref>.

This is achieved by proposing generalizations to the max-error loss function given by

:<math>

Line 52:

== Software implementations ==

* PyIPM provides an [[open source|open-source]] Python implementation of the work of Crespo (2015) .<ref>{{cite journal|doi=10.5281/zenodo.2784750|title=PyIPM|year=2019|last1=Sadeghi|first1=Jonathan}}</ref>.

* [http://cossan.co.uk/software/open-cossan-engine.php OpenCOSSAN] provides a Matlab implementation of the work of Crespo (2015) .<ref name="PatelliBroggi2017" />.

== References ==