Data-driven astronomy: Difference between revisions

Revision as of 23:06, 24 May 2024

Latest revision as of 15:48, 13 October 2024

(7 intermediate revisions by 7 users not shown)

Line 1:

{{Afd-merge#REDIRECT to|[[Astroinformatics|Data-driven astronomy|24 May 2024}}]]

{{Merge to|Astroinformatics|discuss=Talk:Astroinformatics#Proposed merge of Data-driven astronomy into Astroinformatics|date=May 2024}}

{{Multiple issues |

{{notability|date=May 2024}}

{{coi|date=May 2024}}

{{Orphan|date=May 2024|att=May 2024|incat=May 2024}}

}}

{{R from merge}}

'''Data-driven astronomy (DDA)''' refers to the use of [[data science]] in [[astronomy]]. Several outputs of [[Telescopic observational astronomy|telescopic observations]] and [[Astronomical survey|sky surveys]] are taken into consideration and approaches related to [[data mining]] and big data management are used to analyze, filter, and [[Normalization (statistics)|normalize]] the [[data set]] that are further used for making Classifications, Predictions, and Anomaly detections by [[Advances in Statistics|advanced Statistical approaches]], [[digital image processing]] and [[machine learning]]. The output of these processes is used by [[astronomer]]s and space scientists to study and identify patterns, anomalies, and movements in outer space and conclude theories and discoveries in the [[cosmos]].

== History ==

In 2007, the [[Galaxy Zoo|Galaxy Zoo project]]<ref>{{Cite web |title=Zooniverse |url=https://www.zooniverse.org/projects/zookeeper/galaxy-zoo |access-date=2024-05-10 |website=www.zooniverse.org}}</ref> was launched for [[Galaxy morphological classification|morphological classification]]<ref>{{Cite journal |last1=Cavanagh |first1=Mitchell K. |last2=Bekki |first2=Kenji |last3=Groves |first3=Brent A. |date=2021-07-08 |title=Morphological classification of galaxies with deep learning: comparing 3-way and 4-way CNNs |journal=Monthly Notices of the Royal Astronomical Society |volume=506 |issue=1 |pages=659–676 |doi=10.1093/mnras/stab1552 |arxiv=2106.01571 |issn=0035-8711}}</ref><ref>{{Cite journal |last1=Goyal |first1=Lalit Mohan |last2=Arora |first2=Maanak |last3=Pandey |first3=Tushar |last4=Mittal |first4=Mamta |date=2020-12-01 |title=Morphological classification of galaxies using Conv-nets |url=https://doi.org/10.1007/s12145-020-00526-w |journal=Earth Science Informatics |language=en |volume=13 |issue=4 |pages=1427–1436 |doi=10.1007/s12145-020-00526-w |issn=1865-0481}}</ref> of a large number of [[Galaxy|galaxies]]. In this project, 900,000 images were considered for classification that were taken from the [[Sloan Digital Sky Survey]] (SDSS)<ref name=":0">{{Cite web |title=Sloan Digital Sky Survey-V: Pioneering Panoptic Spectroscopy - SDSS-V |url=https://www.sdss.org/ |access-date=2024-05-10 |language=en-US}}</ref> for the past 7 years. The task was to study each picture of a galaxy, classify it as [[Elliptical galaxy|elliptical]] or [[Spiral galaxy|spiral]], and determine whether it was spinning or not. The team of Astrophysicists led by [[Kevin Schawinski]] in [[University of Oxford|Oxford University]] were in charge of this project and Kevin and his colleague [[Chris Lintott|Chris Linlott]] figured out that it would take a period of 3–5 years for such a team to complete the work.<ref>{{Cite web |last=Pati |first=Satavisa |date=2021-06-18 |title=How Data Science is Used in Astronomy? |url=https://www.analyticsinsight.net/data-science/how-data-science-is-used-in-astronomy |access-date=2024-05-10 |website=Analytics Insight |language=en}}</ref> There they came up with the idea of using Machine Learning and Data Science techniques for analyzing the images and classifying them.<ref>{{Citation |last=Baron |first=Dalya |title=Machine Learning in Astronomy: a practical overview |date=2019-04-15 |arxiv=1904.07248}}</ref>

== Methodology ==

The data retrieved from the sky surveys are first brought for [[data preprocessing]]. In this, [[Data redundancy|redundancies]] are removed and filtrated. Further, [[Feature engineering|feature extraction]] is performed on this filtered data set, which is further taken for processes.<ref name=":1">{{Cite journal |last1=Zhang |first1=Yanxia |last2=Zhao |first2=Yongheng |date=2015-05-22 |title=Astronomy in the Big Data Era |journal=Data Science Journal |volume=14 |pages=11 |doi=10.5334/dsj-2015-011 |doi-access=free |bibcode=2015DatSJ..14...11Z |issn=1683-1470}}</ref> Some of the renowned sky surveys are listed below:

* The Palomar Digital Sky Survey (DPOSS)<ref>{{Cite web |title=The Palomar Digital Sky Survey (DPOSS) |url=https://sites.astro.caltech.edu/~george/dposs/dposs_pop.html |access-date=2024-05-10 |website=sites.astro.caltech.edu}}</ref>

* The Two-Micron All Sky Survey (2MASS)<ref>{{Cite web |title=IRSA - Two Micron All Sky Survey (2MASS) |url=https://irsa.ipac.caltech.edu/Missions/2mass.html |access-date=2024-05-10 |website=irsa.ipac.caltech.edu}}</ref>

* [[Green Bank Telescope|Green Bank Telescope (GBT)]]<ref>{{Cite web |date=2023-06-26 |title=GBT |url=https://greenbankobservatory.org/portal/gbt/ |access-date=2024-05-10 |website=Green Bank Observatory |language=en-US}}</ref>

* The Galaxy Evolution Explorer (GALEX)<ref>{{Cite web |title=GALEX - Galaxy Evolution Explorer |url=http://www.galex.caltech.edu/ |access-date=2024-05-10 |website=www.galex.caltech.edu}}</ref>

* The Sloan Digital Sky Survey (SDSS)<ref name=":0" />

* [[SkyMapper|SkyMapper Southern Sky Survey (SMSS)]]<ref>{{Cite web |title=SkyMapper Southern Sky Survey |url=https://skymapper.anu.edu.au/ |access-date=2024-05-10 |website=skymapper.anu.edu.au}}</ref>

* [[Pan-STARRS|The Panoramic Survey Telescope and Rapid Response System (PanSTARRS)]]<ref>{{Cite web |title=Pan-STARRS1 data archive home page - PS1 Public Archive - STScI Outerspace |url=https://outerspace.stsci.edu/display/PANSTARRS/ |access-date=2024-05-10 |website=outerspace.stsci.edu}}</ref>

* [[Vera C. Rubin Observatory|The Large Synoptic Survey Telescope (LSST)]]<ref>{{Cite web |last=Telescope |first=Large Synoptic Survey |title=Rubin Observatory |url=https://www.lsst.org/ |access-date=2024-05-10 |website=Rubin Observatory |language=en}}</ref>

* [[Square Kilometre Array|The Square Kilometer Array (SKA)]]<ref>{{Cite web |title=Explore {{!}} SKAO |url=https://www.skao.int/en |access-date=2024-05-10 |website=www.skao.int}}</ref>

The size of data from the above-mentioned sky surveys ranges from 3&nbsp;[[Terabyte|TB]] to almost 4.6&nbsp;[[Exabyte|EB]].<ref name=":1" /> Further, [[data mining]] tasks that are involved in the management and manipulation of the data involve methods like [[Statistical classification|classification]], [[Regression analysis|regression]], [[Cluster analysis|clustering]], [[anomaly detection]], and [[Time series|time-series analysis]]. Several approaches and applications for each of these methods are involved in the task accomplishments.

=== Classification ===

'''''Classification'''''<ref>{{Cite book |last1=Chowdhury |first1=Shovan |last2=Schoen |first2=Marco P. |chapter=Research Paper Classification using Supervised Machine Learning Techniques |date=2020-10-02 |title=2020 Intermountain Engineering, Technology and Computing (IETC) |chapter-url=https://ieeexplore.ieee.org/document/9249211 |publisher=IEEE |pages=1–6 |doi=10.1109/IETC47856.2020.9249211 |isbn=978-1-7281-4291-3}}</ref> is used for specific identifications and categorizations of astronomical data such as [[Stellar classification|Spectral classification]], Photometric classification, Morphological classification, and classification of [[Solar phenomena|solar activity]]. The approaches of classification techniques are listed below:

* [[Artificial neural network]] (ANN)

* [[Support vector machine]] (SVM)

* [[Learning vector quantization]] (LVQ)

* [[Decision tree]]

* [[Random forest]]

* [[K-nearest neighbors algorithm|''k''-nearest neighbors]]

* [[Naive Bayes classifier|Naïve Bayesian networks]]

* [[Radial basis function|Radial basis function network]]

* [[Gaussian process]]

* [[Decision table]]

* [[Alternating decision tree]] (ADTree)

=== Regression ===

'''''Regression'''''<ref>{{Citation |last1=Sarstedt |first1=Marko |title=Regression Analysis |date=2014 |work=A Concise Guide to Market Research: The Process, Data, and Methods Using IBM SPSS Statistics |pages=193–233 |editor-last=Sarstedt |editor-first=Marko |url=https://doi.org/10.1007/978-3-642-53965-7_7 |access-date=2024-05-10 |place=Berlin, Heidelberg |publisher=Springer |language=en |doi=10.1007/978-3-642-53965-7_7 |isbn=978-3-642-53965-7 |last2=Mooi |first2=Erik |editor2-last=Mooi |editor2-first=Erik}}</ref> is used to make predictions based on the retrieved data through statistical trends and statistical modeling. Different uses of this technique are used for fetching [[Photometric redshift]]s and measurements of physical parameters of stars.<ref>{{Cite journal |title=Bulletin de la Société Royale des Sciences de Liège {{!}} PoPuPS |url=https://popups.uliege.be/0037-9565/index.php |journal=Bulletin de la Société Royale des Sciences de Liège |language=fr |issn=0037-9565}}</ref> The approaches are listed below:

* [[Artificial neural network]] (ANN)

* [[Support vector regression]] (SVR)

* [[Decision tree]]

* [[Random forest]]

* [[K-nearest neighbors algorithm|''k''-nearest neighbors regression]]

* [[Kernel regression]]

* [[Principal component regression]] (PCR)

* [[Gaussian process]]

* [[Linear least squares|Least squared regression (LSR)]]

* [[Partial least squares regression|Partial least squares regression]]

=== Clustering ===

'''''Clustering'''''<ref>{{Cite book |last1=Bindra |first1=Kamalpreet |last2=Mishra |first2=Anuranjan |chapter=A detailed study of clustering algorithms |date=September 2017 |title=2017 6th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO) |chapter-url=https://ieeexplore.ieee.org/document/8342454 |publisher=IEEE |pages=371–376 |doi=10.1109/ICRITO.2017.8342454 |isbn=978-1-5090-3012-5}}</ref> is classifying objects based on a [[similarity measure]] metric. It is used in Astronomy for Classification as well as [[Object detection|Special/rare object detection]]. The approaches are listed below:

* [[Principal component analysis]] (PCA)

* [[DBSCAN]]

* [[K-means clustering|''k''-means clustering]]

* [[OPTICS algorithm|OPTICS]]

* [[Cobweb model]]

* [[Self-organizing map]] (SOM)

* [[Expectation–maximization algorithm|Expectation Maximization]]

* [[Hierarchical clustering|Hierarchical Clustering]]

* AutoClass<ref>{{Cite journal |last1=Pizzuti |first1=C. |last2=Talia |first2=D. |date=May 2003 |title=P-autoclass: scalable parallel clustering for mining large data sets |url=https://ieeexplore.ieee.org/document/1198395 |journal=IEEE Transactions on Knowledge and Data Engineering |language=en |volume=15 |issue=3 |pages=629–641 |doi=10.1109/TKDE.2003.1198395 |issn=1041-4347}}</ref>

* [[Gaussian process|Gaussian Mixture Modeling (GMM)]]

=== Anomaly detection ===

'''''Anomaly detection'''''<ref>{{Cite journal |last1=Thudumu |first1=Srikanth |last2=Branch |first2=Philip |last3=Jin |first3=Jiong |last4=Singh |first4=Jugdutt (Jack) |date=2020-07-02 |title=A comprehensive survey of anomaly detection techniques for high dimensional big data |journal=Journal of Big Data |volume=7 |issue=1 |pages=42 |doi=10.1186/s40537-020-00320-x |doi-access=free |issn=2196-1115|hdl=10536/DRO/DU:30158643 |hdl-access=free }}</ref> is used for detecting irregularities in the dataset. However, this technique is used here to detect [[Object detection|rare/special objects]]. The following approaches are used:

* [[Principal component analysis|Principal Component Analysis (PCA)]]

* [[K-means clustering|''k''-means clustering]]

* [[Expectation–maximization algorithm|Expectation Maximization]]

* [[Hierarchical clustering]]

* [[Support vector machine|One-class SVM]]

=== Time-series analysis ===

'''''Time-Series analysis'''''<ref>{{Cite book |url=https://onlinelibrary.wiley.com/doi/book/10.1002/0471264385 |title=Handbook of Psychology |date=2003-04-15 |publisher=Wiley |isbn=978-0-471-17669-5 |editor-last=Weiner |editor-first=Irving B. |edition=1 |language=en |doi=10.1002/0471264385.wei0223}}</ref> helps in analyzing trends and predicting outputs over time. It is used for trend prediction and novel detection (detection of unknown data). The approaches used here are:

* [[Artificial neural network]] (ANN)

* [[Support vector regression]] (SVR)

* [[Decision tree]]

==References ==

{{Reflist}}

[[Category:Astronomy]]

[[Category:Astrophysics]]

[[Category:Data science]]

[[Category:Statistics]]

[[Category:Machine learning]]

[[Category:MathematicsApplied statistics]]