Astrophysics Data System: Difference between revisions

Revision as of 13:48, 25 July 2024

Revision as of 20:04, 26 July 2024

Line 34:

[[Johann Friedrich Weidler]] published the first comprehensive history of astronomy in 1741 and the first astronomical bibliography in 1755. This was an effort to archive and classify earlier astronomical knowledge and works.

This effort was continued by [[Jérôme Lalande|Jérôme de La Lande]], who published his ''Bibliographie astronomique'' in 1803, a work that covered the period from 480 B.C.BCE to the year of publication.

The ''Bibliographie générale de l’astronomie, Volume I and Volume II'', published by J.C. Houzeau and A. Lancaster, followed in 1882 until 1889.<ref>{{cite book |last1=Houzeau |first1=J. C. |title=Bibliographie générale de l'astronomie |date=1887 |publisher=F. Hayez, Imprimeur de L'Académie Royale de Belgique |url=https://books.google.com/books?id=xJhA9noqjT0C&q=Bibliographie+g%C3%A9n%C3%A9rale+de+l%27astronomie+volume+I |language=fr}}</ref><ref>{{cite book |last1=Houzeau |first1=Jean-Charles |title=Bibliographie générale de l'astronomie ou catalogue méthodique des ouvrages, des mémoires et des observations astronomiques publiés depuis l'origine de l'imprimerie jusqu'en 1880: Mémoires et notices insérés dans les Collections académiques et les Revues |date=1882 |url=https://books.google.com/books?id=GzxeAAAAcAAJ |language=fr}}</ref>

Line 41:

== History ==

The first suggestion of a digital database of journal paper abstracts was made at a conference on ''Astronomy from Large Data-Bases,'' held in [[Garching bei München]] in 1987.<ref>{{cite journal |last1=Squibb |first1=G.F. |last2=Cheung |first2=C.Y. |year=1988 |title=NASA astrophysics data system (ADS) study |journal=European Southern Observatory Conference and Workshop Proceedings |volume=28 |page=489 |bibcode=1988ESOC...28..489S}}</ref><ref>{{cite conference |last1=Adorf |first1=H.-M. |last2=Busch |first2=E.K. |year=1988 |title=Intelligent access to a bibliographical full text data base |volume=28 |page=143 |bibcode=1988ESOC...28..143A |journal=European Southern Observatory Conference and Workshop Proceedings}}</ref><ref>{{cite conference |last=Rey-Watson |first=J.M. |year=1988 |title=Access to astronomical literature through commercial databases |volume=28 |page=453 |bibcode=1988ESOC...28..453R |journal=European Southern Observatory Conference and Workshop Proceedings}}</ref><ref>{{cite conference |last1=Rhodes |first1=C. |last2=Kurtz |first2=M.J. |last3=Rey-Watson |first3=J.M. |year=1988 |title=A library collection of software documentation specific to astronomical data reduction |volume=28 |page=459 |bibcode=1988ESOC...28..459R |journal=European Southern Observatory Conference and Workshop Proceedings}}</ref>

An initial version of ADS, with a database consisting of 40 papers, was created as a [[proof of concept]] in 1988. The ADS Abstract Service became available for general use via proprietary network software in April 1993, with it becoming connected to [[SIMBAD]] a few months later. In early 1994, the ADS web-based service was launched, which effectively quadrupled the number of active users in the five weeks following its introduction.<ref name="overview">{{cite journal |last=Kurtz |first=M.J. |author2=Eichhorn G. |author3=Accomazzi A. |author4=Grant C.S. |author5=Murray S.S. |author6=Watson J.M. |year=2000 |title=The NASA Astrophysics Data System: Overview |journal=Astronomy and Astrophysics Supplement Series |volume=143 |issue=1 |pages=41–59 |arxiv=astro-ph/0002104 |bibcode=2000A&AS..143...41K |doi=10.1051/aas:2000170 |s2cid=17583122}}</ref>

In 2011 the ADS launched ADS Labs Streamlined Search which introduced [https://www.researchgate.net/publication/228582891_Retrieval_from_facet_spaces facets] for query refinement and selection. In 2013 ADS Labs 2.0 featuring a new search engine, full-text search functionality, scalable facets and an API was introduced. In 2015 the new ADS, code-named Bumblebee, was released as ADS-beta. The ADS-beta system features a microservices API and client-side dynamic page loading served on a cloud platform. In May 2018 the beta label was dropped and Bumblebee became the default ADS interface—with some legacy features (ADS Classic) remaining available.<ref>{{Cite conference |last1=Accomazzi |first1=Alberto |last2=Kurtz |first2=Michael J. |last3=Henneken |first3=Edwin |last4=Grant |first4=Carolyn S. |last5=Thompson |first5=Donna M. |last6=Chyla |first6=Roman |last7=McDonald |first7=Steven |last8=Shaulis |first8=Taylor J. |last9=Blanco-Cuaresma |first9=Sergi |last10=Shapurian |first10=Golnaz |last11=Hostetler |first11=Timothy W. |last12=Templeton |first12=Matthew R. |last13=Lockhart |first13=Kelly E. |date=January 2018 |title=ADS Bumblebee comes of age |conference=231st Meeting of the American Astronomical Society |at=362.17 |bibcode=2018AAS...23136217A}}</ref> Development continues to the present day, with an extensible [[API]] available: enabling users to build their own utilities on top of the ADS bibliographic record.

The ADS service is distributed worldwide, with twelve [[Web mirror|mirror sites]] in twelve countries, and with the database synchronized by weekly updates using [[rsync]], a mirroring utility which allows updates to only the portions of the database which have changed. All updates are triggered centrally, but they initiate scripts at the mirror sites which "pull" updated data from the main ADS servers.<ref name="architecture">{{cite journal |last1=Accomazzi |first1=A. |last2=Eichhorn |first2=G. |last3=Kurtz |first3=M.J. |last4=Grant |first4=C.S. |last5=Murray |first5=S.S. |year=2000 |title=The NASA Astrophysics Data System: Architecture |journal=Astronomy and Astrophysics Supplement Series |volume=143 |issue=1 |pages=85–109 |doi=10.1051/aas:2000172 |bibcode=2000A&AS..143...85A |arxiv=astro-ph/0002105 |s2cid=7182316}}</ref>

==Data in the system==

At first, the journal articles available via ADS were exclusively [[Image scanner|scan]]ned [[bitmap]]s created from the paper journals and the abstracts created using [[optical character recognition]] software. Some of these scanned articles up to around 1995 are available for free by agreement with the journal publishers,<ref name="data">{{cite web |date=23 June 2005 |title=NASA ADS Abstract Service Mirroring Information |url=http://ads.harvard.edu/mirror/ |access-date=2 November 2008 |publisher=Harvard-Smithsonian Center for Astrophysics}}</ref> with some dating from as far back as the early 19th century. Eventually, because of a wider spread of online editions of journal publications, abstracts would start to instead be loaded into ADS directly.

Papers are indexed within the database by their bibliographic record which contains the details of the journal they were published in, and various associated [[metadata]], such as author lists, [[Image resolution|reference]]s and [[citation]]s. Originally this data was stored in [[ASCII]] format, but, eventually, the limitations of this encouraged the database maintainers to migrate all records to an [[XML]] (Extensible Markup Language) format in 2000. Bibliographic records are now stored as an XML element, with sub-elements for the various metadata.<ref name="architecture" />

Scanned articles are stored in [[TIFF]] format, at both medium and high [[Image resolution|resolution]]. The TIFF files are converted on demand into GIF files, for on-screen viewing, and [[PDF]], or [[PostScript]] files for printing. The generated files are then [[cache (computing)|cached]] to eliminate needlessly frequent regenerations for popular articles. As of 2000, ADS contained 250 [[gigabyte|GB]] of scans, which consisted of 1,128,955 article pages comprising 138,789 articles. By 2005 this had grown to 650 GB, and was expected to grow further, to about 900 GB by 2007.<ref name="data" /> No further information has been published (2005).

The database initially contained only astronomical references, but has now grown to incorporate three databases, covering [[astronomy]]

references (including planetary sciences and solar physics), [[physics]] references (including instrumentation and geosciences), as well as preprints of scientific papers from [[arXiv]]. The astronomy database is by far the most advanced and its use accounts for about 85% of the total ADS usage. Articles are assigned to the different databases according to the subject rather than the journal they are published in, so that articles from any one journal might appear in all three subject databases. The separation of the databases allows searching in each discipline to be tailored, so that words can automatically be given different [[weight function]]s in different database searches, depending on how common they are in the relevant field.<ref name="architecture"/>

Data in the preprint archive is updated daily from the [[arXiv]], which is the maindominant repository of physics and astronomy preprints. The advent of preprint servers has, like ADS, had a significant impact on the rate of astronomical research, as papers are often made available from preprint servers weeks or months before they are published in the journals. The incorporation of preprints from the arXiv into ADS means that the search engine can return the most current research available, with the caveat that preprints may not have been peer-reviewed or [[proofread]] to the required standard for publication in the main journals. ADS'sThe database of ADS links preprints with subsequently published articles wherever possible, so that citation and reference searches will return links to the journal article where the preprint was cited.<ref>{{cite conference |url=http://meetings.aps.org/Meeting/MAR07/Event/61837 |title=myADS-arXiv: A fully customized, open access virtual journal |conference=March Meeting 2007, American Physical Society |volume=52 |issue=1 |at=U20.9 |access-date=30 October 2008}}</ref>

==Software and hardware==

The software runs on a system that was written specifically for it, allowing for extensive customization for astronomical needs that would not have been possible with general purpose [[database]] software. The scripts are designed to be as [[platform independent]] as possible, given the need to facilitate mirroring on different systems around the world, although the growing use of [[Linux]] as the [[operating system]] of choice within astronomy has led to increasing optimization of the scripts for installation on that platform.<ref name="architecture"/>

The main ADS server is located at the [[Harvard-Smithsonian Center for Astrophysics|Center for Astrophysics {{!}} Harvard & Smithsonian]] in [[Cambridge, Massachusetts]], and is a dual 64-bit X86 [[Intel]] server with two quad-core 3.0 [[GHz]] [[Central processing unit|CPU]]s and 32 GB of [[random access memory|RAM]], running the [[CentOS]] 5.4 [[Linux]] distribution.<ref name="data" /> MirrorsAs of 2022, there are mirrors located in Brazil, China, Chile, France, Germany, India, Indonesia, Japan, Russia, South Korea,the United Kingdom, and Ukraine.<ref>{{cite web |url=http://doc.adsabs.harvard.edu/mirrors.html |title=SAO/NASA ADS at SAO: Mirror Sites |publisherurl=Harvard-Smithsonian Center for Astrophysics |access-date=30 October 2008http://doc.adsabs.harvard.edu/mirrors.html |archiveurl=https://web.archive.org/web/2008022708381920220128092136/httphttps://doc.adsabs.harvard.edu/mirrors.html |archive-date=2728 FebruaryJanuary 20082022 |access-date=26 July 2024 |publisher=Harvard-Smithsonian Center for Astrophysics}}</ref>

==Indexing==

Line 79:

==Search engine==

[[Image:A complex ADS search.png|thumb|right|An example of a complex search combining object, title and abstract queries with a date filter.]]

Since its inception, the ADS has developed a highly complex [[search engine]] to query the abstract and [[Object database|object databases]]. The search engine is tailor-made for searching astronomical abstracts, and the engine and its [[user interface]] assume that the user is well-versed in astronomy and able to interpret search results which are designed to return more than just the most relevant papers. The database can be queried for author names, [[astronomical object]] names, title words, and words in the abstract text, and results can be filtered according to a number of criteria. It works by first gathering synonyms and simplifying search terms as described above, and then generating an "inverted file", which is a list of all the documents matching each search term. The user-selected logic and filters are then applied to this inverted list to generate the final search results.<ref name="search">{{cite journal |last1=Eichhorn |first1=G. |last2=Kurtz |first2=M.J. |last3=Accomazzi |first3=A. |last4=Grant |first4=C.S. |last5=Murray |first5=S.S. |year=2000 |title=The NASA Astrophysics Data System: The search engine and its user interface |journal=Astronomy and Astrophysics Supplement Series |volume=143 |issue=1 |pages=61–83 |doi=10.1051/aas:2000171 |bibcode=2000A&AS..143...61E |arxiv=astro-ph/0002102 |s2cid=2787647}}</ref>

Line 110:

===Result filtering===

Search results can be filtered according to a number of criteria, including specifying a range of years such as '"1945 to 1975'", '"2000 to the present day'" or '"before 1900'", and what type of journal the article appears in [–] non-peer-reviewed articles such as [[academic conference|conference]] proceedings. These can be excluded or specifically searched for, or specific journals can be included in or excluded from the search.

==Search results==

[[Image:ADS search results.png|thumb|right|Search results page from ADS – A, F, G, C, R etc. are links to associated data for each abstract such as full-text article, citations, also-read papers and so on.]]

Although it was conceived as a means of accessing abstracts and papers, ADS provides a substantial amount of ancillary information along with search results. For each abstract returned, links are provided to other papers in the database which are referenced, and which cite the paper, and a link is provided to a preprint, where one exists. The system also generates a link to '"also-read'" articles – that is, those which have been most commonly accessed by those reading the article. In this way, an ADS user can determine which papers are of most interest to astronomers who are interested in the subject of a given paper.<ref name="search" />

Also returned are links to the [[SIMBAD]] and/or [[NASA Extragalactic Database]] object name databases, via which a user can quickly find out basic observational data about the objects analyzed in a paper, and find further papers on those objects.