Generalized p-value: Difference between revisions

Revision as of 01:33, 4 December 2009

Latest revision as of 04:31, 9 June 2024

(38 intermediate revisions by 21 users not shown)

Line 1:

{{Short description|Statistics concept}}

{{New unreviewed article|source=ArticleWizard|date=November 2009}}

{{Expert-subject|StatisticsMore footnotes|date=DecemberJanuary 20092017}}

{{DISPLAYTITLE:Generalized ''p''-value}}

In [[statistics]], a '''generalized ''p''-value''' is an extended version of the classical [[p-value|''p''-value]], which except in a limited number of applications, provides only approximate solutions.

Conventional statistical methods do not provide exact solutions to many statistical problems, such as those arising in [[mixed model]]s and [[MANOVA]], especially when the problem involves manya number of [[nuisance parametersparameter]]s. As a result, practitioners often resort to approximate statistical methods or [[Asymptotic theory (statistics)|asymptotic statistical methods]] that are primarilyvalid basedonly onwhen largethe samplessample size is large. With small samples, in most cases, these approximatesuch methods andoften asymptotichave methods perform verypoor poorlyperformance.<ref Duename=WE/> to the utilizationUse of these approximate and asymptotic methods, experimentersmay often faillead to detectmisleading theconclusions significanceor of their experiments. Furthermore, there are well-documented cases where these methods not onlymay fail to detect thetruly [[statisticalStatistical significance|significant]], butresults mayfrom also lead to misleading conclusions[[experiment]]s.

'''Generalized p-values'''

Tests based on generalized ''p''-values are exact statistical methods in that they are based on exact probability statements. While conventional statistical methods do not provide exact solutions to such problems as testing [[variance components]] or [[ANOVA]] under unequal variances, exact tests for such problems can be obtained based on generalized ''p''-values.<ref name=WE>Weerahandi (1995)</ref><ref name=TW>Tsui & Weerahandi (1989)</ref>

Conventional statistical methods do not provide exact solutions to many statistical problems, especially when the problem involves many nuisance parameters. As a result, practitioners often resort to approximate statistical methods or asymptotic statistical methods that are primarily based on large samples. With small samples, in most cases, these approximate methods and asymptotic methods perform very poorly. Due to the utilization of these approximate and asymptotic methods, experimenters often fail to detect the significance of their experiments. Furthermore, there are well-documented cases where these methods not only fail to detect the [[statistical significance]], but may also lead to misleading conclusions.

In order to overcome the shortcomings of the classical ''p''-values, Tsui and Weerahandi<ref name=TW/> extended the classical definition so that one can obtain exact solutions for such problems as the [[Behrens&ndash;Fisher problem]] and testing variance components. This is accomplished by allowing test variables to depend on observable random vectors as well as their observed values, as in the Bayesian treatment of the problem, but without having to treat constant parameters as random variables.

Generalized p-values method is an exact statistical method based on exact probability statements rather than asymptotic statistical methods to tackle difficult statistical problems where conventional statistical methods do not provide exact solutions. Moreover, the generalized p-value approach is an extension of the classical p-value approach.

==Example==

In order to over come the shortcomings of the classical p-values, Tsui and Weerahandi (1989) extended the definition of the classical p-values so that one can obtain the exact solutions for problems such as the [[Behrens-Fisher problem]].

To describe the idea of generalized ''p''-values in a simple example, consider a situation of sampling from a normal population with the mean <math>\mu</math>, and the variance <math>\sigma ^2</math>. Let <math>\overline{X}</math> and <math>S ^2</math> be the sample mean and the sample variance. Inferences on all unknown parameters can be based on the distributional results

Later, Weerahandi (1993) showed how one can utilize the generalized p-values to construct generalized [[confidence interval]]s.

:<math> Z = \sqrt{n}(\overline{X} - \mu)/ \sigma \sim N(0,1)</math>

A complete coverage of the definitions and applications of generalized p-values can be found in Weerahandi (1995) and Weerahandi (2004).

and

:<math>U = n S^2 / \sigma^2 \sim \chi^2 _ {n-1} .</math>

Many of the generalize p-values procedures are incorporated into software packages that are freely available in the Internet (eg. XPro).

Now suppose we need to test the coefficient of variation, <math>\rho = \mu /\sigma </math>. While the problem is not trivial with conventional ''p''-values, the task can be easily accomplished based on the generalized test variable

:<math>R = \frac {\overline{x} S} {s \sigma} - \frac{\overline{X}- \mu} {\sigma}

= \frac {\overline{x}} {s} \frac {\sqrt{U}} {\sqrt{n}} ~-~ \frac {Z} {\sqrt{n}} ,</math>

where <math>\overline{x}</math> is the observed value of <math>\overline{X}</math> and <math>s</math> is the observed value of <math>S</math>. Note that the distribution of <math>R</math> and its observed value are both free of nuisance parameters. Therefore, a test of a hypothesis with a one-sided alternative such as <math> H_A : \rho < \rho_0 </math> can be based on the generalized ''p''-value <math> p = Pr( R \ge \rho_0 )</math>, a quantity that can be easily evaluated via Monte Carlo simulation or using the non-central t-distribution.

==ReferencesNotes==

{{Reflist}}

[1] Tsui, K. and Weerahandi, S. (1989): Generalized p-values in significance testing of hypotheses in the presence of nuisance parameters. Journal of the American Statistical Association, 84, 602-607 (1989). [[http://www.jstor.org/stable/2289949]]

[2] Weerahandi, S. (1993): Generalized confidence intervals. Journal of the American Statistical Association, 88, 899-905 (1993). [[http://www.jstor.org/stable/2290779]]

[3] [http://www.springer.com/statistics/statistical+theory+and+methods/book/978-0-387-40621-3 Weerahandi, S. 1995. Exact Statistical Method for Data Analysis. Springer-Verlag, New York. ]

==References==

[4] [http://www.wiley-vch.de/publish/en/books/bySubjectST00/bySubSubjectST12/0-471-47017-1/?sID=d05b Weerahandi, S. 2004. Generalized Inference in Repeated Measures: Exact Methods in MANOVA and Mixed Models. Wiley, New York]

*Gamage J, Mathew T, and Weerahandi S. (2013). Generalized prediction intervals for BLUPs in mixed models, Journal of Multivariate Analysis}, 220, 226-233.

[5] [http://www.x-techniques.com/ XPro, Windows software package for exact parametric statistics]

*Hamada, M., and Weerahandi, S. (2000). Measurement System Assessment via Generalized Inference. Journal of Quality Technology, 32, 241-253.

*Krishnamoorthy, K. and Tian, L. (2007), “Inferences on the ratio of means of two inverse Gaussian distributions: the generalized variable approach”, Journal of Statistical Planning and Inferences, Volume 138, Issue 7, 1, Pages 2082-2089.

*Li, X., Wang J., Liang H. (2011). Comparison of several means: a fiducial based approach. Computational Statistics and Data Analysis, 55, 1993-2002.

* Mathew, T. and Webb, D. W. (2005). Generalized p-values and confidence intervals for variance components: Applications to Army test and evaluation, Technometrics, 47, 312-322.

*Wu, J. and Hamada, M. S. (2009) Experiments: Planning, Analysis, and Optimization. Wiley, Hoboken, New Jersey.

*Zhou, L., and Mathew, T. (1994). Some Tests for Variance Components Using Generalized p-Values, Technometrics, 36, 394-421.

*Tian, L. and Wu, Jianrong (2006) “Inferences on the Common Mean of Several Log-normal Populations: The Generalized Variable Approach”, Biometrical Journal.

[1] *Tsui, K. and Weerahandi, S. (1989): [https://www.jstor.org/stable/2289949 "Generalized ''p''-values in significance testing of hypotheses in the presence of nuisance parameters"]. ''[[Journal of the American Statistical Association]]'', 84, 602-&ndash;607 (1989). [[http://www.jstor.org/stable/2289949]]

[3]*Weerahandi, S. (1995) [httphttps://www.springer.com/statistics/statistical+theory+and+methods/book/978-0-387-40621-3 Weerahandi, S. 1995. ''Exact Statistical MethodMethods for Data Analysis.'' ] Springer-Verlag, New York. ]{{ISBN|978-0-387-40621-3}}

==External links==

[5] *[http://www.x-techniques.com/ XPro, WindowsFree software package for exact parametric statistics]

{{DEFAULTSORT:Generalized P-Value}}

[[Category:HypothesisStatistical hypothesis testing]]