Generalized p-value: Difference between revisions

Revision as of 09:55, 10 December 2009

Revision as of 19:49, 22 August 2010

Line 1:

'''generalized p-values''' is an extended version the classical [[p-value]]s, which except in a limited number of applications, provide only approximate solutions.

{{New unreviewed article|source=ArticleWizard|date=November 2009}}

{{Expert-subject|Statistics|date=December 2009}}

{{Cleanup|date=December 2009}}

Conventional statistical methods do not provide exact solutions to many statistical problems, especially when the problem involves many [[nuisance parameter]]s. As a result, practitioners often resort to approximate statistical methods or asymptotic statistical methods that are primarilyvalid basedonly onwith large samples. With small samples, in most cases, these approximatesuch methods and asymptotic methodsoften performhave verypoor poorlyperformance. Due to the utilizationUse of these approximate and asymptotic methods, experimentersmay often faillead to detectmisleading theconclusions significanceor of their experiments. Furthermore, there are well-documented cases where these methods not onlymay fail to detect thetruly [[statisticalsignificant significance]],results butfrom mayexperiments. also lead to misleading conclusions.

In [[statistics]], '''generalized p-values''' are used to give exact solutions of problems for which the classical method of [[p-value]]s gives only approximate solutions, the approximations being good only when the [[sample size]]s are big enough.

Tests based on generalized p-values are exact statistical methods in that they are based on exact probability statements. While conventional statistical methods do not provide exact solutions to such problems as testing variance components or ANOVA under unequal variances, the references below provide exact tests based on generalized p-values..

Conventional statistical methods do not provide exact solutions to many statistical problems, especially when the problem involves many [[nuisance parameter]]s. As a result, practitioners often resort to approximate statistical methods or asymptotic statistical methods that are primarily based on large samples. With small samples, in most cases, these approximate methods and asymptotic methods perform very poorly. Due to the utilization of these approximate and asymptotic methods, experimenters often fail to detect the significance of their experiments. Furthermore, there are well-documented cases where these methods not only fail to detect the [[statistical significance]], but may also lead to misleading conclusions.

In order to over comeovercome the shortcomings of the classical p-values, Tsui and Weerahandi (1989) extended the definition of the classical p-values so that one can obtain the exact solutions for problems such as the [[Behrens–Fisher problem]].

Generalized p-values method is an exact statistical method based on exact probability statements rather than asymptotic statistical methods to tackle difficult statistical problems where conventional statistical methods do not provide exact solutions. Moreover, the generalized p-value approach is an extension of the classical p-value approach.

To describe the idea in simple example consider a situation of sampling from a normal population with mean <math>\mu</math>, and variance <math>\sigma ^2</math>, suppose <math>\overline{X}</math> and <math>S ^2</math> are the sample mean and the sample variance. Inferences on all unknwon paramters can be based on the distributional results

In order to over come the shortcomings of the classical p-values, Tsui and Weerahandi (1989) extended the definition of the classical p-values so that one can obtain the exact solutions for problems such as the [[Behrens&ndash;Fisher problem]].

<math> Z = \sqrt{n}(\overline{X} - \mu)/ \sigma \sim N(0,1)</math>

Later, Weerahandi (1993) showed how one can utilize the generalized p-values to construct generalized [[confidence interval]]s.

and <math>U = n S^2 / \sigma^2 \sim \chi^2 _ {n-1}</math>.

Now suppose we need to test the coefficient of variation, <math>\rho = \mu /\sigma </math>. This can be easily accomplished based on the generalized test variable

A complete coverage of the definitions and applications of generalized p-values can be found in Weerahandi (1995) and Weerahandi (2004).

<math>R = \frac {\overline{x} S} {s \sigma} - \frac{\overline{X}- \mu} {\sigma}

= \frac {\overline{x}} {s} \frac {\sqrt{U}} {\sqrt{n}} ~-~ \frac {Z} {\sqrt{n}} </math>,

Many of the generalize p-values procedures are incorporated into software packages that are freely available in the Internet (eg. XPro).

where <math>\overline{x}</math> is the observed value of <math>\overline{X}</math> and <math>S</math> is the observed value of <math>s</math>. Note that the distribution of <math>R</math> and its observed value are both free of nuisance parameters. Therefore, one-sided hypotheses such as <math> H_0 : \rho < \rho_0 </math> can be tested based on the generalized p-value <math> p = Pr( R \ge \rho_0 )</math>, a quantity that can be evaluated via Monte Carlo simulation or using the non-central t-distribution.

==References==

[1] Tsui, K. and Weerahandi, S. (1989): Generalized p-values in significance testing of hypotheses in the presence of nuisance parameters. Journal of the American Statistical Association, 84, 602-607 (1989). [http://www.jstor.org/stable/2289949]

[32] [http://www.springer.com/statistics/statistical+theory+and+methods/book/978-0-387-40621-3 Weerahandi, S. 1995. Exact Statistical Method for Data Analysis. Springer-Verlag, New York. ]

[2] Weerahandi, S. (1993): Generalized confidence intervals. Journal of the American Statistical Association, 88, 899-905 (1993). [http://www.jstor.org/stable/2290779]

[3] [http://www.springer.com/statistics/statistical+theory+and+methods/book/978-0-387-40621-3 Weerahandi, S. 1995. Exact Statistical Method for Data Analysis. Springer-Verlag, New York. ]

[4] [http://www.wiley-vch.de/publish/en/books/bySubjectST00/bySubSubjectST12/0-471-47017-1/?sID=d05b Weerahandi, S. 2004. Generalized Inference in Repeated Measures: Exact Methods in MANOVA and Mixed Models. Wiley, New York]

[53] [http://www.x-techniques.com/ XPro, WindowsFree software package for exact parametric statistics]

[[Category:Hypothesis testing]]