Data set: Difference between revisions

Revision as of 10:28, 12 July 2023

Revision as of 00:49, 20 August 2025

(24 intermediate revisions by 16 users not shown)

Line 1:

{{aboutfor-multi|Thethe generaltype concept|files onof IBM mainframesfile|Data set (IBM mainframe)|datathe communications device|ModemDataset (device)}}

{{short description|Collection of data}}

[[File:Iris dataset scatterplot.svg|thumb|right|Various plots of the multivariate datasetdata set [[Iris flower data set|''Iris'' flower data set]] introduced by [[Ronald Fisher]] (1936).<ref name="fisher36"/>]]

A '''data set''' (or '''dataset''') is a collection of [[data]]. In the case of [[tabular data]], a data set corresponds to one or more [[table (database)|database tables]], where every [[column (database)|column]] of a table represents a particular [[Variable (computer science)|variable]], and each [[row (database)|row]] corresponds to a given [[Record (computer science)|record]] of the data set in question. The data set lists values for each of the variables, such as for example height and weight of an object, for each member of the data set. Data sets can also consist of a collection of documents or files.<ref name="Editorial">{{cite journal | last1 = Snijders | first1 = C. | last2 = Matzat | first2 = U. | last3 = Reips | first3 = U.-D. | year = 2012 | title = 'Big Data': Big gaps of knowledge in the field of Internet | url = http://www.ijis.net/ijis7_1/ijis7_1_editorial.html | journal = International Journal of Internet Science | volume = 7 | pages = 1–5 | access-date = 2017-02-10 | archive-date = 2019-11-23 | archive-url = https://web.archive.org/web/20191123051001/http://www.ijis.net/ijis7_1/ijis7_1_editorial.html | url-status = dead }}</ref>

In the [[open data]] discipline, dataa setdataset is thea unit used to measure the amount of information released in a public open data repository. The European [[data.europa.eu]] portal aggregates more than a million data sets.<ref>{{Cite web|url=http://www.europeandataportal.eu/data/en/dataset|title=European open data portal|website=European open data portal|publisher=European Commission|access-date=2016-09-23}}</ref>

==Properties==

Several characteristics define a data set's structure and properties. These include the number and types of the attributes or variables, and various [[statistical measure]]s applicable to them, such as [[standard deviation]] and [[kurtosis]].<ref>{{Cite book |url=https://books.google.com/books?id=uTzeRZFmaBgC&pg=PA100 |title=Principles of data mining and knowledge discovery |author=Jan M. Żytkow, Jan Rauch |isbn=978-3-540-66490-1 |year=2000|publisher=Springer }}</ref>

The values may be numbers, such as [[real number]]s or [[integer]]s, for example representing a person's height in centimeters, but may also be [[nominal data]] (i.e., not consisting of [[Number|numerical]] values), for example representing a person's ethnicity. More generally, values may be of any of the kinds described as a [[level of measurement]]. For each variable, the values are normally all of the same kind. However, there may also be ''[[missingMissing values]]'' may exist, which must be indicated in some waysomehow.

In [[statistics]], data sets usually come from actual observations obtained by [[sampling (statistics)|sampling]] a [[statistical population]], and each row corresponds to the observations on one element of that population. Data sets may further be generated by [[algorithms]] for the purpose of testing certain kinds of [[software]]. Some modern statistical analysis software such as [[SPSS]] still present their data in the classical data set fashion. If data is missing or suspicious an [[imputation (statistics)|imputation]] method may be used to complete a data set.<ref name="sde">{{cite book |title=Statistical Data Editing: Impact on Data Quality: Volume 3 of Statistical Data Editing, Conference of European Statisticians Statistical standards and studies |author=United Nations Statistical Commission |author2=United Nations Economic Commission for Europe |year=2007 |publisher=United Nations Publications |isbn=978-9211169522 |page=20 |url=https://booksunece.google.comorg/books?id=X0wtLo2XY9gC |access-date=19 JulyDAM/stats/publications/editing/SDE3.pdf 2015}}</ref>

== Classic data setsClassics ==

Several classic data sets have been used extensively in the [[statistical]] literature:

Line 19:

* [[MNIST database]] – Images of handwritten digits commonly used to test classification, clustering, and [[Digital image processing|image processing]] algorithms

* ''[[Categorical data analysis]]'' – Data sets used in the book, ''An Introduction to Categorical Data Analysis'', [https://stats.oarc.ucla.edu/other/examples/icda/ provided online] by UCLA Advanced Research Computing.<ref>{{cite web |url=https://stats.oarc.ucla.edu/other/examples/icda/ |title=Textbook Examples An Introduction to Categorical Data Analysis by Alan Agresti |access-date=2023-05-02 |url-status=live |archive-url=https://web.archive.org/web/20230131013107/https://stats.oarc.ucla.edu/other/examples/icda/ |archive-date=2023-01-31}}</ref>

*''[[Robust statistics]]'' – Data sets used in ''[[Robust Regression and Outlier Detection]]'' ([[Peter Rousseeuw|Rousseeuw]] and Leroy, 1968). [https://web.archive.org/web/20050207032959/http://www.uni-koeln.de/themen/statistik/data/rousseeuw/ Provided online] at the University of Cologne.<ref>{{cite web |url=http://www.uni-koeln.de/themen/statistik/data/rousseeuw/ |title=The ROUSSEEUW datasets |url-status=dead |archive-url=https://web.archive.org/web/20050207032959/http://www.uni-koeln.de/themen/statistik/data/rousseeuw/ |archive-date=2005-02-07}}</ref>

*''[[Time series]]'' – Data used in Chatfield's book, ''The Analysis of Time Series'', are [https://web.archive.org/web/20110102201323/http://lib.stat.cmu.edu/modules.php?op=modload&name=PostWrap&file=index&page=datasets/ provided on-line] by StatLib.<ref>{{cite web |url=http://lib.stat.cmu.edu/modules.php?op=modload&name=PostWrap&file=index&page=datasets/ |title=StatLib :: Data, Software and News from the Statistics Community |url-status=dead |archive-url=https://web.archive.org/web/20110102201323/http://lib.stat.cmu.edu/modules.php?op=modload&name=PostWrap&file=index&page=datasets/ |archive-date=2011-01-02}}</ref>

*''Extreme values'' – Data used in the book, ''An Introduction to the Statistical Modeling of Extreme Values'' are [https://web.archive.org/web/20060910161517/http://homes.stat.unipd.it/coles/public_html/ismev/ismev.dat a snapshot of the data as it was provided on-line by Stuart Coles], the book's author.

*''Bayesian Data Analysis'' – Data used in the book are [http://www.stat.columbia.edu/~gelman/book/data/ provided on-line] ([https://web.archive.org/web/20230122121643/http://www.stat.columbia.edu/~gelman/book/data/ archive link]) by [[Andrew Gelman]], one of the book's authors.

* The [https://web.archive.org/web/20171023174701/http://ftp.ics.uci.edu:80/pub/machine-learning-databases/liver-disorders/ Bupa liver data] – Used in several papers in the [[machine learning]] (data mining) literature.

* [[Anscombe's quartet]] – Small data set illustrating the importance of graphing the data to avoid statistical fallacies.

== See also Example==

Loading datasets using [[Python (programming language)|Python]]:

<syntaxhighlight lang="console">

$ pip install datasets

</syntaxhighlight>

<syntaxhighlight lang="python">

from datasets import load_dataset

dataset = load_dataset(NAME OF DATASET)

* [[Data]]

</syntaxhighlight>

== See also ==

* [[List of datasets for machine-learning research]]

* [[List of datasets in computer vision and image processing]]

* [[Data blending]]

* [[Data (computingcomputer science)]]

* [[Sampling (statistics)|Sampling]]

* [[Data samples]]

* [[Data store]]

* [[Interoperability]]

* [[Data collection system]]

* [[List of datasets for machine-learning research]]

{{Clear}}

Line 45 ⟶ 55:

* [https://www.data.gov/ Data.gov] – the U.S. Government's open data

* [https://earthdata.nasa.gov/gcmd GCMD] – the Global Change Master Directory containing over 34,000 descriptions of Earth science and environmental science data sets and services

* [https://data.humdata.org/ Humanitarian Data Exchange(HDX)] – The Humanitarian Data Exchange (HDX) is an open humanitarian [[data sharing]] platform managed by the [[United Nations Office for the Coordination of Humanitarian Affairs]].

* [https://opendata.cityofnewyork.us/ NYC Open Data] – free public data published by New York City agencies and other partners.

* [https://relational.fitfel.cvut.cz/ Relational data set repository] {{Webarchive|url=https://web.archive.org/web/20180307150058/https://relational.fit.cvut.cz/ |date=2018-03-07 }}

* [https://web.archive.org/web/20190214051201/http://www.researchpipeline.com/mediawiki/index.php?title=Main_Page Research Pipeline] – a wiki/website with links to data sets on many different topics

* [http://lib.stat.cmu.edu/jasadata/ StatLib–JASA Data Archive]