Content deleted Content added
Jasper Deng (talk | contribs) m Reverted edits by 27.251.171.83 (talk) to last version by ClueBot NG |
Rescuing 1 sources and tagging 0 as dead.) #IABot (v2.0.9.5 |
||
Line 1:
{{Short description|Organized collection of data in computing}}
{{about|the computing concept|instances of the general concept|Lists of databases}}
{{Merge from|Data bank|discuss=Talk:Database#Proposed merge of Data bank into Database|date=May 2025}}
{{protection padlock|1=vandalism|small=y}}
[[Image:DVD Rental Query.png|thumb|An [[SQL]] select statement and its result|upright=1.35]]
In [[computing]], a '''database''' is an organized collection of [[Data (computing)|data]] or a type of [[data store]] based on the use of a '''database management system''' ('''DBMS'''), the [[software]] that interacts with [[end user]]s, [[Application software|applications]], and the database itself to capture and analyze the data. The DBMS additionally encompasses the core facilities provided to administer the database. The sum total of the database, the DBMS and the associated applications can be referred to as a '''database system'''. Often the term "database" is also used loosely to refer to any of the DBMS, the database system or an application associated with the database.
Before digital storage and retrieval of data have become widespread, [[index cards]] were used for [[data storage]] in a wide range of applications and environments: in the home to record and store recipes, shopping lists, contact information and other organizational data; in business to record presentation notes, project research and notes, and contact information; in schools as flash cards or other visual aids; and in academic research to hold data such as bibliographical citations or notes in a [[card file]]. Professional book indexers used index cards in the creation of book indexes until they were replaced by [[indexing software]] in the 1980s and 1990s.
Small databases can be stored on a [[file system]], while large databases are hosted on [[computer clusters]] or [[cloud storage]]. The [[Database design|design of databases]] spans formal techniques and practical considerations, including [[data modeling]], efficient data representation and storage, [[query language]]s, [[Database security|security]] and [[Information privacy|privacy]] of sensitive data, and [[distributed computing]] issues, including supporting [[concurrent computing|concurrent]] access and [[fault tolerance]].
[[Computer scientists]] may classify database management systems according to the [[database model]]s that they support. [[Relational database]]s became dominant in the 1980s. These model data as [[Row (database)|rows]] and [[Column (database)|columns]] in a series of [[Table (database)|tables]], and the vast majority use [[SQL]] for writing and querying data. In the 2000s, non-relational databases became popular, collectively referred to as [[NoSQL]], because they use different [[query language]]s.
==Terminology and overview==
Formally, a "database" refers to a set of related data accessed through the use of a "database management system" (DBMS), which is an integrated set of [[computer software]] that allows [[user (computing)|users]] to interact with one or more databases and provides access to all of the data contained in the database (although restrictions may exist that limit access to particular data). The DBMS provides various functions that allow entry, storage and retrieval of large quantities of information and provides ways to manage how that information is organized.
Because of the close relationship between them, the term "database" is often used casually to refer to both a database and the DBMS used to manipulate it.
Outside the world of professional [[information technology]], the term ''database'' is often used to refer to any collection of related data (such as a [[spreadsheet]] or a card index) as size and usage requirements typically necessitate use of a database management system.{{sfn|Ullman|Widom|1997|p=1}}
Existing DBMSs provide various functions that allow management of a database and its data which can be classified into four main functional groups:
* '''Data definition''' – Creation, modification and removal of definitions that detail how the data is to be organized.
* '''Update''' – Insertion, modification, and deletion of the data itself.<ref>{{cite web|url=http://www.merriam-webster.com/dictionary/update|title=Update Definition & Meaning |work=Merriam-Webster |url-status=live |archive-url=https://web.archive.org/web/20240225065959/https://www.merriam-webster.com/dictionary/update |archive-date= Feb 25, 2024 }}</ref>
* '''Retrieval''' – Selecting data according to specified criteria (e.g., a query, a position in a hierarchy, or a position in relation to other data) and providing that data either directly to the user, or making it available for further processing by the database itself or by other applications. The retrieved data may be made available in a more or less direct form without modification, as it is stored in the database, or in a new form obtained by altering it or combining it with existing data from the database.<ref>{{cite web|url=http://www.merriam-webster.com/dictionary/retrieval|title=Retrieval Definition & Meaning |work=Merriam-Webster |url-status=live |archive-url= https://web.archive.org/web/20230627174611/https://www.merriam-webster.com/dictionary/retrieval |archive-date= Jun 27, 2023 }}</ref>
* '''Administration''' – Registering and monitoring users, enforcing data security, monitoring performance, maintaining data integrity, dealing with concurrency control, and recovering information that has been corrupted by some event such as an unexpected system failure.<ref>{{cite web|url=http://www.merriam-webster.com/dictionary/administration|title=Administration Definition & Meaning |work=Merriam-Webster |url-status=live |archive-url=https://web.archive.org/web/20231206055116/https://www.merriam-webster.com/dictionary/administration |archive-date= Dec 6, 2023 }}</ref>
Both a database and its DBMS conform to the principles of a particular [[database model]].{{sfn|Tsitchizris| Lochovsky| 1982}} "Database system" refers collectively to the database model, database management system, and database.{{sfn|Beynon-Davies|2003}}
Physically, database [[Server (computing)|servers]] are dedicated computers that hold the actual databases and run only the DBMS and related software. Database servers are usually [[multiprocessor]] computers, with generous memory and [[RAID]] disk arrays used for stable storage. Hardware database accelerators, connected to one or more servers via a high-speed channel, are also used in large-volume [[Transaction processing system|transaction processing environments]]. DBMSs are found at the heart of most [[database application]]s. DBMSs may be built around a custom [[computer multitasking|multitasking]] [[Kernel (operating system)|kernel]] with built-in [[Computer network|networking]] support, but modern DBMSs typically rely on a standard [[operating system]] to provide these functions.{{citation needed|reason=This has become a mish-mash people's opinion and while boardly accurate may be in places undue and ignored e.g. SAN storage and needs cited rewrite from scrach|date=January 2020}}
Since DBMSs comprise a significant [[Market (economics)|market]], computer and storage vendors often take into account DBMS requirements in their own development plans.{{sfn|Nelson|Nelson|2001}}
Databases and DBMSs can be categorized according to the database model(s) that they support (such as [[Relational database|relational]] or [[XML]]), the type(s) of computer they run on (from a [[server cluster]] to a [[mobile phone]]), the [[query language]](s) used to access the database (such as SQL or [[XQuery]]), and their internal engineering, which affects performance, [[scalability]], resilience, and security.
==History==
The sizes, capabilities, and performance of databases and their respective DBMSs have grown in orders of magnitude. These performance increases were enabled by the technology progress in the areas of [[Central processing unit|processors]], [[computer memory]], [[computer storage]], and [[computer network]]s. The concept of a database was made possible by the emergence of direct access [[storage media]] such as [[Hard disk drive|magnetic disks]], which became widely available in the mid-1960s; earlier systems relied on sequential storage of data on [[magnetic tape]]. The subsequent development of database technology can be divided into three eras based on data model or structure: [[navigational database|navigational]],{{sfn|Bachman|1973}} SQL/[[relational database|relational]], and post-relational.
The two main early navigational [[data model]]s were the [[hierarchical model]] and the [[CODASYL]] model ([[network model]]). These were characterized by the use of [[Pointer (computer programming)|pointers]] (often physical disk addresses) to follow relationships from one record to another.
The [[relational model]], first proposed in 1970 by [[Edgar F. Codd]], departed from this tradition by insisting that [[Application software|applications]] should search for data by content, rather than by following links. The relational model employs sets of ledger-style tables, each used for a different type of [[entity]]. Only in the mid-1980s did computing hardware become powerful enough to allow the wide deployment of relational systems (DBMSs plus applications). By the early 1990s, however, relational systems dominated in all large-scale [[data processing]] applications, and {{as of | 2018 | lc = on}} they remain dominant: [[IBM Db2]], [[Oracle database|Oracle]], [[MySQL]], and [[Microsoft SQL Server]] are the most searched [[DBMS]].<ref>{{cite web|url=https://pypl.github.io/DB.html|title=TOPDB Top Database index|work=pypl.github.io}}</ref> The dominant database language, standardized SQL for the relational model, has influenced database languages for other data models.{{Citation needed|date=March 2013}}
[[Object database]]s were developed in the 1980s to overcome the inconvenience of [[object–relational impedance mismatch]], which led to the coining of the term "post-relational" and also the development of hybrid [[object–relational database]]s.
The next generation of post-relational databases in the late 2000s became known as [[NoSQL]] databases, introducing fast [[key–value store]]s and [[document-oriented database]]s. A competing "next generation" known as [[NewSQL]] databases attempted new implementations that retained the relational/SQL model while aiming to match the high performance of NoSQL compared to commercially available relational DBMSs.
===
{{Further|Navigational database}}
[[File:CodasylB.png|thumb|upright=1.15|Basic structure of navigational [[CODASYL]] database model]]
The introduction of the term ''database'' coincided with the availability of direct-access storage (disks and drums) from the mid-1960s onwards. The term represented a contrast with the tape-based systems of the past, allowing shared interactive use rather than daily [[batch processing]]. The [[Oxford English Dictionary]] cites a 1962 report by the [[System Development Corporation]] of California as the first to use the term "data-base" in a specific technical sense.<ref>{{cite web|title=database, n|url=http://www.oed.com/view/Entry/47411|work=OED Online|publisher=Oxford University Press|access-date=July 12, 2013|date=June 2013}} {{subscription required|s}}</ref>
As computers grew in speed and capability, a number of general-purpose database systems emerged; by the mid-1960s a number of such systems had come into commercial use. Interest in a standard began to grow, and [[Charles Bachman]], author of one such product, the [[Integrated Data Store]] (IDS), founded the [[Data Base Task Group|Database Task Group]] within [[CODASYL]], the group responsible for the creation and standardization of [[COBOL]]. In 1971, the Database Task Group delivered their standard, which generally became known as the ''CODASYL approach'', and soon a number of commercial products based on this approach entered the market.
The CODASYL approach offered applications the ability to navigate around a linked data set which was formed into a large network. Applications could find records by one of three methods:
#Use of a primary key (known as a CALC key, typically implemented by [[Hash function|hashing]])
#Navigating relationships (called ''sets'') from one record to another
#Scanning all the records in a sequential order
Later systems added [[B-tree]]s to provide alternate access paths. Many CODASYL databases also added a declarative query language for end users (as distinct from the navigational [[API]]). However, CODASYL databases were complex and required significant training and effort to produce useful applications.
[[IBM]] also had its own DBMS in 1966, known as [[Information Management System]] (IMS). IMS was a development of software written for the [[Apollo program]] on the [[System/360]]. IMS was generally similar in concept to CODASYL, but used a strict hierarchy for its model of data navigation instead of CODASYL's network model. Both concepts later became known as navigational databases due to the way data was accessed: the term was popularized by Bachman's 1973 [[Turing Award]] presentation ''The Programmer as Navigator''. IMS is classified by IBM as a [[hierarchical database]]. IDMS and [[Cincom Systems]]' [[Cincom Systems#1970s|TOTAL]] databases are classified as network databases. IMS remains in use {{as of | 2014 | lc = on}}.<ref>{{cite web|last=IBM Corporation|title=IBM Information Management System (IMS) 13 Transaction and Database Servers delivers high performance and low total cost of ownership|url=http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?subtype=ca&infotype=an&appname=iSource&supplier=897&letternum=ENUS213-381|access-date=Feb 20, 2014|date=October 2013}}</ref>
===1970s, relational DBMS===
[[Edgar F. Codd]] worked at IBM in [[San Jose, California]], in an office primarily involved in the development of [[hard disk]] systems.{{r|rdbmsearlyyearsoh20070612}} He was unhappy with the navigational model of the CODASYL approach, notably the lack of a "search" facility. In 1970, he wrote a number of papers that outlined a new approach to database construction that eventually culminated in the groundbreaking ''A Relational Model of Data for Large Shared Data Banks''.{{sfn|Codd|1970}}
The paper described a new system for storing and working with large databases. Instead of records being stored in some sort of [[linked list]] of free-form records as in CODASYL, Codd's idea was to organize the data as a number of "[[Table (database)|tables]]", each table being used for a different type of entity. Each table would contain a fixed number of columns containing the attributes of the entity. One or more columns of each table were designated as a [[primary key]] by which the rows of the table could be uniquely identified; cross-references between tables always used these primary keys, rather than disk addresses, and queries would join tables based on these key relationships, using a set of operations based on the mathematical system of [[relational calculus]] (from which the model takes its name). Splitting the data into a set of normalized tables (or ''relations'') aimed to ensure that each "fact" was only stored once, thus simplifying update operations. Virtual tables called ''views'' could present the data in different ways for different users, but views could not be directly updated.
Codd used mathematical terms to define the model: relations, tuples, and domains rather than tables, rows, and columns. The terminology that is now familiar came from early implementations. Codd would later criticize the tendency for practical implementations to depart from the mathematical foundations on which the model was based.
[[File:Relational key SVG.svg|thumb|In the [[relational model]], records are "linked" using virtual keys not stored in the database but defined as needed between the data contained in the records.]]
The use of primary keys (user-oriented identifiers) to represent cross-table relationships, rather than disk addresses, had two primary motivations. From an engineering perspective, it enabled tables to be relocated and resized without expensive database reorganization. But Codd was more interested in the difference in semantics: the use of explicit identifiers made it easier to define update operations with clean mathematical definitions, and it also enabled query operations to be defined in terms of the established discipline of [[first-order predicate calculus]]; because these operations have clean mathematical properties, it becomes possible to rewrite queries in provably correct ways, which is the basis of query optimization. There is no loss of expressiveness compared with the hierarchic or network models, though the connections between tables are no longer so explicit.
In the hierarchic and network models, records were allowed to have a complex internal structure. For example, the salary history of an employee might be represented as a "repeating group" within the employee record. In the relational model, the process of normalization led to such internal structures being replaced by data held in multiple tables, connected only by logical keys.
For instance, a common use of a database system is to track information about users, their name, login information, various addresses and phone numbers. In the navigational approach, all of this data would be placed in a single variable-length record. In the relational approach, the data would be ''normalized'' into a user table, an address table and a phone number table (for instance). Records would be created in these optional tables only if the address or phone numbers were actually provided.
As well as identifying rows/records using logical identifiers rather than disk addresses, Codd changed the way in which applications assembled data from multiple records. Rather than requiring applications to gather data one record at a time by navigating the links, they would use a declarative query language that expressed what data was required, rather than the access path by which it should be found. Finding an efficient access path to the data became the responsibility of the database management system, rather than the application programmer. This process, called query optimization, depended on the fact that queries were expressed in terms of mathematical logic.
Codd's paper inspired teams at various universities to research the subject, including one at [[University of California, Berkeley]]{{r|rdbmsearlyyearsoh20070612}} led by [[Eugene Wong]] and [[Michael Stonebraker]], who started [[INGRES]] using funding that had already been allocated for a geographical database project and student programmers to produce code. Beginning in 1973, INGRES delivered its first test products which were generally ready for widespread use in 1979. INGRES was similar to [[IBM System R|System R]] in a number of ways, including the use of a "language" for [[data access]], known as [[QUEL query languages|QUEL]]. Over time, INGRES moved to the emerging SQL standard.
IBM itself did one test implementation of the relational model, [[PRTV]], and a production one, [[Business System 12]], both now discontinued. [[Honeywell]] wrote [[Multics Relational Data Store|MRDS]] for [[Multics]], and now there are two new implementations: [[Dataphor|Alphora Dataphor]] and Rel. Most other DBMS implementations usually called ''relational'' are actually SQL DBMSs.
In 1970, the University of Michigan began development of the [[MICRO Information Management System]]{{sfn|Hershey|Easthope|1972}} based on [[David L. Childs|D.L. Childs]]' Set-Theoretic Data model.{{sfn|North|2010}}{{sfn|Childs|1968a}}{{sfn|Childs|1968b}} The university in 1974 hosted a debate between Codd and Bachman which Bruce Lindsay of IBM later described as "throwing lightning bolts at each other!".<ref name="rdbmsearlyyearsoh20070612">{{Cite interview |interviewer=Burton Grad |title=RDBMS Plenary 1: Early Years |url=https://archive.computerhistory.org/resources/access/text/2013/05/102702562-05-01-acc.pdf |access-date=2025-05-30 |publisher=Computer History Museum |date=2007-06-12}}</ref> MICRO was used to manage very large data sets by the [[US Department of Labor]], the [[U.S. Environmental Protection Agency]], and researchers from the [[University of Alberta]], the [[University of Michigan]], and [[Wayne State University]]. It ran on IBM mainframe computers using the [[Michigan Terminal System]].<ref name=MICROManual1977>{{cite book |author1=M.A. Kahn |author2=D.L. Rumelhart |author3=B.L. Bronson |date=October 1977 |url=https://docs.google.com/viewer?a=v&pid=explorer&chrome=true&srcid=0B4t_NX-QeWDYZGMwOTRmOTItZTg2Zi00YmJkLTg4MTktN2E4MWU0YmZlMjE3 |title=MICRO Information Management System (Version 5.0) Reference Manual |publisher=Institute of Labor and Industrial Relations (ILIR), University of Michigan and Wayne State University}}</ref> The system remained in production until 1998.
===Integrated approach===
{{Main|Database machine}}
In the 1970s and 1980s, attempts were made to build database systems with integrated hardware and software. The underlying philosophy was that such integration would provide higher performance at a lower cost. Examples were [[IBM System/38]], the early offering of [[Teradata]], and the [[Britton Lee, Inc.]] database machine.
Another approach to hardware support for database management was [[International Computers Limited|ICL]]'s [[Content Addressable File Store|CAFS]] accelerator, a hardware disk controller with programmable search capabilities. In the long term, these efforts were generally unsuccessful because specialized database machines could not keep pace with the rapid development and progress of general-purpose computers. Thus most database systems nowadays are software systems running on general-purpose hardware, using general-purpose computer data storage. However, this idea is still pursued in certain applications by some companies like [[Netezza]] and Oracle ([[Exadata]]).
===Late 1970s, SQL DBMS===
IBM formed a team led by Codd that started working on a prototype system, ''[[IBM System R|System R]]'' despite opposition from others at the company.{{r|rdbmsearlyyearsoh20070612}} The first version was ready in 1974/5, and work then started on multi-table systems in which the data could be split so that all of the data for a record (some of which is optional) did not have to be stored in a single large "chunk". Subsequent multi-user versions were tested by customers in 1978 and 1979, by which time a standardized [[query language]] – SQL{{Citation needed|reason=First version of SQL standard was SQL-86 adopted in 1986|date=May 2012}} – had been added. Codd's ideas were establishing themselves as both workable and superior to CODASYL, pushing IBM to develop a true production version of System R, known as ''SQL/DS'', and, later, ''Database 2'' ([[IBM Db2]]).
[[Larry Ellison]]'s Oracle Database (or more simply, [[Oracle Database|Oracle]]) started from a different chain, based on IBM's papers on System R. Though Oracle V1 implementations were completed in 1978, it was not until Oracle Version 2 when Ellison beat IBM to market in 1979.<ref>{{cite web|url=https://www.oracle.com/us/corporate/profit/p27anniv-timeline-151918.pdf |archive-url=https://web.archive.org/web/20110320220813/http://www.oracle.com/us/corporate/profit/p27anniv-timeline-151918.pdf |archive-date=2011-03-20 |url-status=live |title=Oracle 30th Anniversary Timeline |access-date=23 August 2017}}</ref>
Stonebraker went on to apply the lessons from INGRES to develop a new database, Postgres, which is now known as [[PostgreSQL]]. PostgreSQL is often used for global mission-critical applications (the .org and .info ___domain name registries use it as their primary [[data store]], as do many large companies and financial institutions).
In Sweden, Codd's paper was also read and [[Mimer SQL]] was developed in the mid-1970s at [[Uppsala University]]. In 1984, this project was consolidated into an independent enterprise.
Another data model, the [[entity–relationship model]], emerged in 1976 and gained popularity for [[database design]] as it emphasized a more familiar description than the earlier relational model. Later on, entity–relationship constructs were retrofitted as a [[data modeling]] construct for the relational model, and the difference between the two has become irrelevant.{{Citation needed|date=March 2013}}
===1980s, on the desktop===
Besides IBM and various software companies such as [[Sybase]] and [[Informix Corporation]], most large computer hardware vendors by the 1980s had their own database systems such as [[DEC (company)|DEC]]'s [[VAX Rdb/VMS]].<ref name="rdbmslateryears20070612">{{Cite interview |interviewer=Burton Grad |title=RDBMS Plenary Session: The Later Years |url=https://archive.computerhistory.org/resources/access/text/2013/05/102701921-05-01-acc.pdf |access-date=2025-05-30 |publisher=Computer History Museum |date=2007-06-12}}</ref> The decade ushered in the age of [[desktop computing]]. The new computers empowered their users with spreadsheets like [[Lotus 1-2-3]] and database software like [[dBASE]]. The dBASE product was lightweight and easy for any computer user to understand out of the box. [[C. Wayne Ratliff]], the creator of dBASE, stated: "dBASE was different from programs like BASIC, C, FORTRAN, and COBOL in that a lot of the dirty work had already been done. The data manipulation is done by dBASE instead of by the user, so the user can concentrate on what he is doing, rather than having to mess with the dirty details of opening, reading, and closing files, and managing space allocation."<ref>[http://www.foxprohistory.org/interview_wayne_ratliff.htm Interview with Wayne Ratliff]. The FoxPro History. Retrieved on 2013-07-12.</ref> dBASE was one of the top selling software titles in the 1980s and early 1990s.
===1990s, object-oriented===
By the start of the decade databases had become a billion-dollar industry in about ten years.{{r|rdbmslateryears20070612}} The 1990s, along with a rise in [[object-oriented programming]], saw a growth in how data in various databases were handled. Programmers and designers began to treat the data in their databases as [[object (computer science)|objects]]. That is to say that if a person's data were in a database, that person's attributes, such as their address, phone number, and age, were now considered to belong to that person instead of being extraneous data. This allows for relations between data to be related to objects and their [[property (programming)|attributes]] and not to individual fields.<ref>Development of an object-oriented DBMS; Portland, Oregon, United States; Pages: 472–482; 1986; {{ISBN|0-89791-204-7}}</ref> The term "[[object–relational impedance mismatch]]" described the inconvenience of translating between programmed objects and database tables. [[Object database]]s and [[object–relational database]]s attempt to solve this problem by providing an object-oriented language (sometimes as extensions to SQL) that programmers can use as alternative to purely relational SQL. On the programming side, libraries known as [[object–relational mapping]]s (ORMs) attempt to solve the same problem.
===2000s, NoSQL and NewSQL===
{{Main|NoSQL|NewSQL}}
Database sales grew rapidly during the [[dotcom bubble]] and, after its end, the rise of [[ecommerce]]. The popularity of [[open source]] databases such as [[MySQL]] has grown since 2000, to the extent that Ken Jacobs of Oracle said in 2005 that perhaps "these guys are doing to us what we did to IBM".{{r|rdbmslateryears20070612}}
[[XML database]]s are a type of structured document-oriented database that allows querying based on [[XML]] document attributes. XML databases are mostly used in applications where the data is conveniently viewed as a collection of documents, with a structure that can vary from the very flexible to the highly rigid: examples include scientific articles, patents, tax filings, and personnel records.
[[NoSQL]] databases are often very fast,<ref>{{Cite web |last=Jordan |first=Meghan |title=NoSQL Latency |url=https://www.scylladb.com/glossary/nosql-latency/ |access-date=2025-06-09 |website=ScyllaDB |language=en-US}}</ref><ref>{{Cite web |title=SQL vs. NoSQL: Full comparison of features, differences, and more |url=https://www.testgorilla.com/blog/sql-vs-nosql/ |access-date=2025-06-09 |website=www.testgorilla.com |language=en}}</ref> do not require fixed table schemas, avoid join operations by storing [[denormalization|denormalized]] data, and are designed to [[horizontal scaling|scale horizontally]].
In recent years, there has been a strong demand for massively distributed databases with high partition tolerance, but according to the [[CAP theorem]], it is impossible for a [[distributed system]] to simultaneously provide [[consistency model|consistency]], availability, and partition tolerance guarantees. A distributed system can satisfy any two of these guarantees at the same time, but not all three. For that reason, many NoSQL databases are using what is called [[eventual consistency]] to provide both availability and partition tolerance guarantees with a reduced level of data consistency.
[[NewSQL]] is a class of modern relational databases that aims to provide the same scalable performance of NoSQL systems for online transaction processing (read-write) workloads while still using SQL and maintaining the [[ACID]] guarantees of a traditional database system.
==Use cases==
{{unreferenced section|date=March 2013}}
Databases are used to support internal operations of organizations and to underpin online interactions with customers and suppliers (see [[Enterprise software]]).
Databases are used to hold administrative information and more specialized data, such as engineering data or economic models. Examples include computerized [[library]] systems, [[flight reservation system]]s, computerized [[parts inventory system]]s, and many [[content management system]]s that store [[website]]s as collections of webpages in a database.
==Classification==
One way to classify databases involves the type of their contents, for example: [[bibliographic database|bibliographic]], document-text, statistical, or multimedia objects. Another way is by their application area, for example: accounting, music compositions, movies, banking, manufacturing, or insurance. A third way is by some technical aspect, such as the database structure or interface type. This section lists a few of the adjectives used to characterize different kinds of databases.
* An [[in-memory database]] is a database that primarily resides in [[main memory]], but is typically backed-up by non-volatile computer data storage. Main memory databases are faster than disk databases, and so are often used where response time is critical, such as in telecommunications network equipment.
* An [[active database]] includes an event-driven architecture which can respond to conditions both inside and outside the database. Possible uses include security monitoring, alerting, statistics gathering and authorization. Many databases provide active database features in the form of [[database trigger]]s.
* A [[cloud database]] relies on [[cloud technology]]. Both the database and most of its DBMS reside remotely, "in the cloud", while its applications are both developed by programmers and later maintained and used by end-users through a [[web browser]] and [[Open API]]s.
* [[Data warehouse]]s{{citation needed|date=December 2022|reason=Data warehouses usually aren't classified as a type of database.}} archive data from operational databases and often from external sources such as market research firms. The warehouse becomes the central source of data for use by managers and other end-users who may not have access to operational data. For example, sales data might be aggregated to weekly totals and converted from internal product codes to use [[Universal Product Code|UPCs]] so that they can be compared with [[ACNielsen]] data. Some basic and essential components of data warehousing include extracting, analyzing, and [[Data mining|mining]] data, transforming, loading, and managing data so as to make them available for further use.
* A [[deductive database]] combines [[logic programming]] with a relational database.
* A [[distributed database]] is one in which both the data and the DBMS span multiple computers.
* A [[document-oriented database]] is designed for storing, retrieving, and managing document-oriented, or semi structured, information. Document-oriented databases are one of the main categories of NoSQL databases.
* An [[embedded database]] system is a DBMS which is tightly integrated with an application software that requires access to stored data in such a way that the DBMS is hidden from the application's end-users and requires little or no ongoing maintenance.<ref>Graves, Steve. [http://www.embedded-computing.com/articles/id/?2020 "COTS Databases For Embedded Systems"] {{Webarchive|url=https://web.archive.org/web/20071114050734/http://www.embedded-computing.com/articles/id/?2020 |date=2007-11-14 }}, ''Embedded Computing Design'' magazine, January 2007. Retrieved on August 13, 2008.</ref>
*End-user databases consist of data developed by individual end-users. Examples of these are collections of documents, spreadsheets, presentations, multimedia, and other files. Several products{{which|date=December 2022}} exist to support such databases.
* A [[federated database system]] comprises several distinct databases, each with its own DBMS. It is handled as a single database by a federated database management system (FDBMS), which transparently integrates multiple autonomous DBMSs, possibly of different types (in which case it would also be a [[heterogeneous database system]]), and provides them with an integrated conceptual view.
* Sometimes the term ''multi-database'' is used as a synonym for federated database, though it may refer to a less integrated (e.g., without an FDBMS and a managed integrated schema) group of databases that cooperate in a single application. In this case, typically [[Middleware (distributed applications)|middleware]] is used for distribution, which typically includes an atomic commit protocol (ACP), e.g., the [[two-phase commit protocol]], to allow [[Distributed transaction|distributed (global) transactions]] across the participating databases.
* A [[graph database]] is a kind of NoSQL database that uses [[Graph (data structure)|graph structures]] with nodes, edges, and properties to represent and store information. General graph databases that can store any graph are distinct from specialized graph databases such as [[triplestore]]s and [[network database model|network databases]].
* An [[array DBMS]] is a kind of NoSQL DBMS that allows modeling, storage, and retrieval of (usually large) multi-dimensional [[Array data structure|arrays]] such as satellite images and climate simulation output.
* In a [[hypertext]] or [[hypermedia]] database, any word or a piece of text representing an object, e.g., another piece of text, an article, a picture, or a film, can be [[hyperlink]]ed to that object. Hypertext databases are particularly useful for organizing large amounts of disparate information. For example, they are useful for organizing [[online encyclopedia]]s, where users can conveniently jump around the text. The [[World Wide Web]] is thus a large distributed hypertext database.
* A [[knowledge base]] (abbreviated '''KB''', '''kb''' or Δ<ref>Argumentation in Artificial Intelligence by Iyad Rahwan, Guillermo R. Simari</ref><ref>{{cite web
| title = OWL DL Semantics
| url = http://www.obitko.com/tutorials/ontologies-semantic-web/owl-dl-semantics.html
|
* A [[mobile database]] can be carried on or synchronized from a mobile computing device.
* [[Operational database]]s store detailed data about the operations of an organization. They typically process relatively high volumes of updates using [[transaction (database)|transactions]]. Examples include [[Customer relationship management|customer databases]] that record contact, credit, and demographic information about a business's customers, personnel databases that hold information such as salary, benefits, skills data about employees, [[enterprise resource planning]] systems that record details about product components, parts inventory, and financial databases that keep track of the organization's money, accounting and financial dealings.
* A [[parallel database]] seeks to improve performance through [[parallelization]] for tasks such as loading data, building indexes and evaluating queries.
::The major parallel DBMS architectures
::* '''[[
::* '''Shared disk architecture''', where each processing unit (typically consisting of multiple processors) has its own main memory, but all units share the other storage.
::* '''[[Shared
* [[Probabilistic database]]s employ [[fuzzy logic]] to draw inferences from imprecise data.
* [[Real-time database]]s process transactions fast enough for the result to come back and be acted on right away.
* A [[spatial database]] can store the data with multidimensional features. The queries on such data include ___location-based queries, like "Where is the closest hotel in my area?".
* A [[temporal database]] has built-in time aspects, for example a temporal data model and a temporal version of [[SQL]]. More specifically the temporal aspects usually include valid-time and transaction-time.
* A [[terminology-oriented database]] builds upon an [[object-oriented database]], often customized for a specific field.
* An [[unstructured data]] database is intended to store in a manageable and protected way diverse objects that do not fit naturally and conveniently in common databases. It may include email messages, documents, journals, multimedia objects, etc. The name may be misleading since some objects can be highly structured. However, the entire possible object collection does not fit into a predefined structured framework. Most established DBMSs now support unstructured data in various ways, and new dedicated DBMSs are emerging.<!-- Isn't this a document-oriented database? If not, clearly distinguish. -->
==Database management system==
Connolly and Begg define database management system (DBMS) as a "software system that enables users to define, create, maintain and control access to the database."{{sfn|Connolly|Begg|2014|p=64}} Examples of DBMS's include [[MySQL]], [[MariaDB]], [[PostgreSQL]], [[Microsoft SQL Server]], [[Oracle Database]], and [[Microsoft Access]].
The DBMS acronym is sometimes extended to indicate the underlying [[database model]], with RDBMS for the [[Relational model|relational]], OODBMS for the [[Object model|object (oriented)]] and ORDBMS for the [[object–relational model]]. Other extensions can indicate some other characteristics, such as DDBMS for a distributed database management systems.
The functionality provided by a DBMS can vary enormously. The core functionality is the storage, retrieval and update of data. [[Edgar F. Codd|Codd]] proposed the following functions and services a fully-fledged general purpose DBMS should provide:{{sfn|Connolly|Begg|2014|pp=97–102}}
* Data storage, retrieval and update
* User accessible catalog or [[data dictionary]] describing the metadata
* Support for transactions and concurrency
* Facilities for recovering the database should it become damaged
* Support for authorization of access and update of data
* Access support from remote locations
* Enforcing constraints to ensure data in the database abides by certain rules
It is also generally to be expected the DBMS will provide a set of utilities for such purposes as may be necessary to administer the database effectively, including import, export, monitoring, defragmentation and analysis utilities.{{sfn|Connolly|Begg|2014|p=102}} The core part of the DBMS interacting between the database and the application interface sometimes referred to as the [[database engine]].
Often DBMSs will have configuration parameters that can be statically and dynamically tuned, for example the maximum amount of main memory on a server the database can use. The trend is to minimize the amount of manual configuration, and for cases such as [[embedded database]]s the need to target zero-administration is paramount.
The large major enterprise DBMSs have tended to increase in size and functionality and have involved up to thousands of human years of development effort throughout their lifetime.{{efn|This article quotes a development time of five years involving 750 people for DB2 release 9 alone.{{sfn|Chong|Wang|Dang|Snow|2007}}}}
Early multi-user DBMS typically only allowed for the application to reside on the same computer with access via [[Computer terminal|terminals]] or terminal emulation software. The [[client–server architecture]] was a development where the application resided on a client desktop and the database on a server allowing the processing to be distributed. This evolved into a [[multitier architecture]] incorporating [[application server]]s and [[web server]]s with the end user interface via a [[web browser]] with the database only directly connected to the adjacent tier.{{sfn|Connolly|Begg|2014|pp=106–113}}
A general-purpose DBMS will provide public [[application programming interface]]s (API) and optionally a processor for [[database language]]s such as [[SQL]] to allow applications to be written to interact with and manipulate the database. A special purpose DBMS may use a private API and be specifically customized and linked to a single application. For example, an [[email]] system performs many of the functions of a general-purpose DBMS such as message insertion, message deletion, attachment handling, blocklist lookup, associating messages an email address and so forth however these functions are limited to what is required to handle email.
==Application==
{{main|Database application}}
External interaction with the database will be via an application program that interfaces with the DBMS.{{sfn|Connolly|Begg|2014|p=65}} This can range from a [[Comparison of database tools|database tool]] that allows users to execute SQL queries textually or graphically, to a website that happens to use a database to store and search information.
===Application program interface===
A [[programmer]] will [[Computer programming|code]] interactions to the database (sometimes referred to as a [[datasource]]) via an [[application program interface]] (API) or via a [[#database language|database language]]. The particular API or language chosen will need to be supported by DBMS, possibly indirectly via a [[preprocessor]] or a bridging API. Some API's aim to be database independent, [[ODBC]] being a commonly known example. Other common API's include [[JDBC]] and [[ADO.NET]].
==Database languages==
Database languages are special-purpose languages, which allow one or more of the following tasks, sometimes distinguished as [[sublanguage]]s:
* [[Data control language]] (DCL) – controls access to data;
* [[Data definition language]] (DDL) – defines data types such as creating, altering, or dropping tables and the relationships among them;
* [[Data manipulation language]] (DML) – performs tasks such as inserting, updating, or deleting data occurrences;
* [[Data query language]] (DQL) – allows searching for information and computing derived information.
Database languages are specific to a particular data model. Notable examples include:
* SQL combines the roles of data definition, data manipulation, and query in a single language. It was one of the first commercial languages for the relational model, although it departs in some respects from [[Codd's 12 rules|the relational model as described by Codd]] (for example, the rows and columns of a table can be ordered). SQL became a standard of the [[American National Standards Institute]] (ANSI) in 1986, and of the [[International Organization for Standardization]] (ISO) in 1987. The standards have been regularly enhanced since and are supported (with varying degrees of conformance) by all mainstream commercial relational DBMSs.{{sfn|Chapple|2005}}<ref name="IBM-sql">{{cite web | title = Structured Query Language (SQL) | publisher = International Business Machines | url = http://publib.boulder.ibm.com/infocenter/db2luw/v9/index.jsp?topic=com.ibm.db2.udb.admin.doc/doc/c0004100.htm | date = October 27, 2006 | access-date = 2007-06-10 }}</ref>
* [[OQL]] is an object model language standard (from the [[Object Data Management Group]]). It has influenced the design of some of the newer query languages like [[JDOQL]] and [[EJB QL]].
* [[XQuery]] is a standard XML query language implemented by XML database systems such as [[MarkLogic]] and [[eXist]], by relational databases with XML capability such as Oracle and Db2, and also by in-memory XML processors such as [[Saxon XSLT|Saxon]].
* [[SQL/XML]] combines [[XQuery]] with SQL.{{sfn|Wagner|2010}}
A database language may also incorporate features like:
* DBMS-specific configuration and storage engine management
* Computations to modify query results, like counting, summing, averaging, sorting, grouping, and cross-referencing
* Constraint enforcement (e.g. in an automotive database, only allowing one engine type per car)
* Application programming interface version of the query language, for programmer convenience
==Storage==
{{Main|Computer data storage|Database engine}}
Database storage is the container of the physical materialization of a database. It comprises the ''internal'' (physical) ''level'' in the database architecture. It also contains all the information needed (e.g., [[metadata]], "data about the data", and internal [[data structure]]s) to reconstruct the ''conceptual level'' and ''external level'' from the internal level when needed. Databases as digital objects contain three layers of information which must be stored: the data, the structure, and the semantics. Proper storage of all three layers is needed for future [[Database preservation|preservation]] and longevity of the database.<ref>{{cite journal |author1=Ramalho, J.C. |author2=Faria, L. |author3=Helder, S. |author4=Coutada, M. |date=31 December 2013 |title=Database Preservation Toolkit: A flexible tool to normalize and give access to databases |journal=Biblioteca Nacional de Portugal |publisher=University of Minho |url=https://core.ac.uk/display/55635702?algorithmId=15&similarToDoc=55614406&similarToDocKey=CORE&recSetID=f3ffea4d-1504-45e9-bfd6-a0495f5c8f9c&position=2&recommendation_type=same_repo&otherRecs=55614407,55635702,55607961,55613627,2255664 }}</ref> Putting data into permanent storage is generally the responsibility of the [[database engine]] a.k.a. "storage engine". Though typically accessed by a DBMS through the underlying operating system (and often using the operating systems' [[file system]]s as intermediates for storage layout), storage properties and configuration settings are extremely important for the efficient operation of the DBMS, and thus are closely maintained by database administrators. A DBMS, while in operation, always has its database residing in several types of storage (e.g., memory and external storage). The database data and the additional needed information, possibly in very large amounts, are coded into bits. Data typically reside in the storage in structures that look completely different from the way the data look at the conceptual and external levels, but in ways that attempt to optimize (the best possible) these levels' reconstruction when needed by users and programs, as well as for computing additional types of needed information from the data (e.g., when querying the database).
Some DBMSs support specifying which [[character encoding]] was used to store data, so multiple encodings can be used in the same database.
Various low-level database storage structures are used by the storage engine to serialize the data model so it can be written to the medium of choice. Techniques such as indexing may be used to improve performance. Conventional storage is row-oriented, but there are also [[column-oriented DBMS|column-oriented]] and [[correlation database]]s.
===
{{Main|
Often storage redundancy is employed to increase performance. A common example is storing ''materialized views'', which consist of frequently needed ''external views'' or query results. Storing such views saves the expensive computing them each time they are needed. The downsides of materialized views are the overhead incurred when updating them to keep them synchronized with their original updated database data, and the cost of storage redundancy.
===Replication===
{{See also|Replication (computing)#Database replication}}
Occasionally a database employs storage redundancy by [[database object]]s replication (with one or more copies) to increase data availability (both to improve performance of simultaneous multiple end-user accesses to the same database object, and to provide resiliency in a case of partial failure of a distributed database). Updates of a replicated object need to be synchronized across the object copies. In many cases, the entire database is replicated.
===Virtualization===
With [[data virtualization]], the data used remains in its original locations and real-time access is established to allow analytics across multiple sources. This can aid in resolving some technical difficulties such as compatibility problems when combining data from various platforms, lowering the risk of error caused by faulty data, and guaranteeing that the newest data is used. Furthermore, avoiding the creation of a new database containing personal information can make it easier to comply with privacy regulations. However, with data virtualization, the connection to all necessary data sources must be operational as there is no local copy of the data, which is one of the main drawbacks of the approach.<ref name="Paiho">{{cite journal | doi=10.1049/smc2.12044 | title=Opportunities of collected city data for smart cities | year=2022 | last1=Paiho | first1=Satu | last2=Tuominen | first2=Pekka | last3=Rökman | first3=Jyri | last4=Ylikerälä | first4=Markus | last5=Pajula | first5=Juha | last6=Siikavirta | first6=Hanne | journal=IET Smart Cities | volume=4 | issue=4 | pages=275–291 | s2cid=253467923 | doi-access=free | issn=2631-7680}}</ref>
==
{{Main|Database security}}
{{Contradicts other|date=March 2013|1=Database security}}
[[Database security]] deals with all various aspects of protecting the database content, its owners, and its users. It ranges from protection from intentional unauthorized database uses to unintentional database accesses by unauthorized entities (e.g., a person or a computer program).
Database access control deals with controlling who (a person or a certain computer program) are allowed to access what information in the database. The information may comprise specific database objects (e.g., record types, specific records, data structures), certain computations over certain objects (e.g., query types, or specific queries), or using specific access paths to the former (e.g., using specific indexes or other data structures to access information). Database access controls are set by special authorized (by the database owner) personnel that uses dedicated protected security DBMS interfaces.
This may be managed directly on an individual basis, or by the assignment of individuals and [[Privilege (Computing)|privileges]] to groups, or (in the most elaborate models) through the assignment of individuals and groups to roles which are then granted entitlements. Data security prevents unauthorized users from viewing or updating the database. Using passwords, users are allowed access to the entire database or subsets of it called "subschemas". For example, an employee database can contain all the data about an individual employee, but one group of users may be authorized to view only payroll data, while others are allowed access to only work history and medical data. If the DBMS provides a way to interactively enter and update the database, as well as interrogate it, this capability allows for managing personal databases.
[[Data security]] in general deals with protecting specific chunks of data, both physically (i.e., from corruption, or destruction, or removal; e.g., see [[physical security]]), or the interpretation of them, or parts of them to meaningful information (e.g., by looking at the strings of bits that they comprise, concluding specific valid credit-card numbers; e.g., see [[data encryption]]).
Change and access logging records who accessed which attributes, what was changed, and when it was changed. Logging services allow for a forensic [[database audit]] later by keeping a record of access occurrences and changes. Sometimes application-level code is used to record changes rather than leaving this in the database. Monitoring can be set up to attempt to detect security breaches. Therefore, organizations must take database security seriously because of the many benefits it provides. Organizations will be safeguarded from security breaches and hacking activities like firewall intrusion, virus spread, and ransom ware. This helps in protecting the company's essential information, which cannot be shared with outsiders at any cause.<ref>{{Cite book |title=Continuous auditing : theory and application |date=2018 |author1=David Y. Chan |author2=Victoria Chiu |author3=Miklos A. Vasarhelyi |isbn=978-1-78743-413-4 |edition=1st |___location=Bingley, UK |oclc=1029759767 |publisher=Emerald Publishing }}</ref>
==Transactions and concurrency==
{{Further|Concurrency control}}
[[Database transactions]] can be used to introduce some level of [[fault tolerance]] and [[data integrity]] after recovery from a [[Crash (computing)|crash]]. A database transaction is a unit of work, typically encapsulating a number of operations over a database (e.g., reading a database object, writing, acquiring or releasing a [[Lock (database)|lock]], etc.), an abstraction supported in database and also other systems. Each transaction has well defined boundaries in terms of which program/code executions are included in that transaction (determined by the transaction's programmer via special transaction commands).
The acronym [[ACID]] describes some ideal properties of a database transaction: [[Atomicity (database systems)|atomicity]], [[Consistency (database systems)|consistency]], [[Isolation (database systems)|isolation]], and [[Durability (database systems)|durability]].
==Migration==
{{
A database built with one DBMS is not portable to another DBMS (i.e., the other DBMS cannot run it). However, in some situations, it is desirable to migrate a database from one DBMS to another. The reasons are primarily economical (different DBMSs may have different [[Total cost of ownership|total costs of ownership]] or TCOs), functional, and operational (different DBMSs may have different capabilities). The migration involves the database's transformation from one DBMS type to another. The transformation should maintain (if possible) the database related application (i.e., all related application programs) intact. Thus, the database's conceptual and external architectural levels should be maintained in the transformation. It may be desired that also some aspects of the architecture internal level are maintained. A complex or large database migration may be a complicated and costly (one-time) project by itself, which should be factored into the decision to migrate. This is in spite of the fact that tools may exist to help migration between specific DBMSs. Typically, a DBMS vendor provides tools to help import databases from other popular DBMSs.
==
{{Main|Database tuning}}
After designing a database for an application
When the database is ready (all its data structures and other needed components are defined), it is typically populated with initial application's data (database initialization, which is typically a distinct project; in many cases using specialized DBMS interfaces that support bulk insertion) before making it operational. In some cases, the database becomes operational while empty
After
==Backup and restore==
{{Main|Backup}}
Sometimes it is desired to bring a database back to a previous state (for many reasons, e.g., cases when the database is found corrupted due to a software error, or if it has been updated with erroneous data). To achieve this, a backup operation is done occasionally or continuously, where each desired database state (i.e., the values of its data and their embedding in database's data structures) is kept within dedicated backup files (many techniques exist to do this effectively). When it is decided by a database administrator to bring the database back to this state (e.g., by specifying this state by a desired point in time when the database was in this state), these files are used to restore that state.
==Static analysis==
Static analysis techniques for software verification can be applied also in the scenario of query languages. In particular, the *[[Abstract interpretation]] framework has been extended to the field of query languages for relational databases as a way to support sound approximation techniques.{{sfn|Halder|Cortesi|2011}} The semantics of query languages can be tuned according to suitable abstractions of the concrete ___domain of data. The abstraction of relational database systems has many interesting applications, in particular, for security purposes, such as fine-grained access control, watermarking, etc.
==Miscellaneous features==
Other DBMS features might include:
* [[Database log]]s – This helps in keeping a history of the executed functions.
* Graphics component for producing graphs and charts, especially in a data warehouse system.
* [[Query optimizer]] – Performs query optimization on every query to choose an efficient ''[[query plan]]'' (a partial order (tree) of operations) to be executed to compute the query result. May be specific to a particular storage engine.
* Tools or hooks for database design, application programming, application program maintenance, database performance analysis and monitoring, database configuration monitoring, DBMS hardware configuration (a DBMS and related database may span computers, networks, and storage units) and related database mapping (especially for a distributed DBMS), storage allocation and database layout monitoring, storage migration, etc.
Increasingly, there are calls for a single system that incorporates all of these core functionalities into the same build, test, and deployment framework for database management and source control. Borrowing from other developments in the software industry, some market such offerings as "[[DevOps]] for database".<ref>{{cite web
| title = How Database Administration Fits into DevOps |author= Ben Linders |date= January 28, 2016
| url = https://www.infoq.com/news/2016/01/database-administration-devops
| access-date = April 15, 2017 }}</ref>
==Design and modeling==
{{Main|Database design}}
[[File:Process of database design v2.png|upright=2|thumb]]
The first task of a database designer is to produce a [[conceptual data model]] that reflects the structure of the information to be held in the database. A common approach to this is to develop an [[entity–relationship model]], often with the aid of drawing tools. Another popular approach is the [[Unified Modeling Language]]. A successful data model will accurately reflect the possible state of the external world being modeled: for example, if people can have more than one phone number, it will allow this information to be captured. Designing a good conceptual data model requires a good understanding of the application ___domain; it typically involves asking deep questions about the things of interest to an organization, like "can a customer also be a supplier?", or "if a product is sold with two different forms of packaging, are those the same product or different products?", or "if a plane flies from New York to Dubai via Frankfurt, is that one flight or two (or maybe even three)?". The answers to these questions establish definitions of the terminology used for entities (customers, products, flights, flight segments) and their relationships and attributes.
Producing the conceptual data model sometimes involves input from [[Business process modeling|business processes]], or the analysis of [[workflow]] in the organization. This can help to establish what information is needed in the database, and what can be left out. For example, it can help when deciding whether the database needs to hold historic data as well as current data.
Having produced a conceptual data model that users are happy with, the next stage is to translate this into a [[Database schema|schema]] that implements the relevant data structures within the database. This process is often called logical database design, and the output is a [[logical data model]] expressed in the form of a schema. Whereas the conceptual data model is (in theory at least) independent of the choice of database technology, the logical data model will be expressed in terms of a particular database model supported by the chosen DBMS. (The terms ''data model'' and ''database model'' are often used interchangeably, but in this article we use ''data model'' for the design of a specific database, and ''database model'' for the modeling notation used to express that design).
The most popular database model for general-purpose databases is the relational model, or more precisely, the relational model as represented by the SQL language. The process of creating a logical database design using this model uses a methodical approach known as [[Database normalization|normalization]]. The goal of normalization is to ensure that each elementary "fact" is only recorded in one place, so that insertions, updates, and deletions automatically maintain consistency.
The final stage of database design is to make the decisions that affect performance, scalability, recovery, security, and the like, which depend on the particular DBMS. This is often called ''physical database design'', and the output is the [[physical data model]]. A key goal during this stage is [[data independence]], meaning that the decisions made for performance optimization purposes should be invisible to end-users and applications. There are two types of data independence: Physical data independence and logical data independence. Physical design is driven mainly by performance requirements, and requires a good knowledge of the expected workload and access patterns, and a deep understanding of the features offered by the chosen DBMS.
Another aspect of physical database design is security. It involves both defining [[access control]] to database objects as well as defining security levels and methods for the data itself.
===
{{Main|
[[File:Database models.jpg|thumb|upright=2|Collage of five types of database models]]
A database model is a type of data model that determines the logical structure of a database and fundamentally determines in which manner [[data]] can be stored, organized, and manipulated. The most popular example of a database model is the relational model (or the SQL approximation of relational), which uses a table-based format.
Common logical data models for databases include:
*[[Navigational database]]s
**[[Hierarchical database model]]
**[[Network model]]
**[[Graph database]]
*[[Relational model]]
*[[Entity–relationship model]]
**[[Enhanced entity–relationship model]]
*[[Object database|Object model]]
*[[Document-oriented database|Document model]]
*[[Entity–attribute–value model]]
*[[Star schema]]
An object–relational database combines the two related structures.
[[Physical data model]]s include:
*[[Inverted index]]
*[[Flat file]]
Other models include:
*[[Multidimensional database|Multidimensional model]]
*[[Array DBMS|Array model]]
*[[Multivalue model]]
Specialized models are optimized for particular types of data:
*[[XML database]]
*[[Semantic data model|Semantic model]]
*[[Content store]]
*[[Event store]]
*[[Time series database|Time series model]]
===External, conceptual, and internal views===
[[Image:Traditional View of Data SVG.svg|thumb|upright=1.15|Traditional view of data<ref name="ITL93">itl.nist.gov (1993) [http://www.itl.nist.gov/fipspubs/idef1x.doc ''Integration Definition for Information Modeling (IDEFIX)''] {{Webarchive|url=https://web.archive.org/web/20131203223034/http://www.itl.nist.gov/fipspubs/idef1x.doc |date=2013-12-03 }}. 21 December 1993.</ref>]]
A database management system provides three views of the database data:
* The '''external level''' defines how each group of end-users sees the organization of data in the database. A single database can have any number of views at the external level.
* The '''conceptual level''' (or ''logical level'') unifies the various external views into a compatible global view.{{sfn|Date|2003|pages=31–32}} It provides the synthesis of all the external views. It is out of the scope of the various database end-users, and is rather of interest to database application developers and database administrators.
* The '''internal level''' (or ''physical level'') is the internal organization of data inside a DBMS. It is concerned with cost, performance, scalability and other operational matters. It deals with storage layout of the data, using storage structures such as [[Index (database)|indexes]] to enhance performance. Occasionally it stores data of individual views ([[materialized view]]s), computed from generic data, if performance justification exists for such redundancy. It balances all the external views' performance requirements, possibly conflicting, in an attempt to optimize overall performance across all activities.
While there is typically only one conceptual and internal view of the data, there can be any number of different external views. This allows users to see database information in a more business-related way rather than from a technical, processing viewpoint. For example, a financial department of a company needs the payment details of all employees as part of the company's expenses, but does not need details about employees that are in the interest of the [[human resources]] department. Thus different departments need different ''views'' of the company's database.
The three-level database architecture relates to the concept of ''data independence'' which was one of the major initial driving forces of the relational model.{{sfn|Date|2003|pages=31–32}} The idea is that changes made at a certain level do not affect the view at a higher level. For example, changes in the internal level do not affect application programs written using conceptual level interfaces, which reduces the impact of making physical changes to improve performance.
The conceptual view provides a level of indirection between internal and external. On the one hand it provides a common view of the database, independent of different external view structures, and on the other hand it abstracts away details of how the data are stored or managed (internal level). In principle every level, and even every external view, can be presented by a different data model. In practice usually a given DBMS uses the same data model for both the external and the conceptual levels (e.g., relational model). The internal level, which is hidden inside the DBMS and depends on its implementation, requires a different level of detail and uses its own types of data structure types.
==Research==
Database technology has been an active research topic since the 1960s, both in [[academia]] and in the research and development groups of companies (for example [[IBM Research]]). Research activity includes [[Database theory|theory]] and development of [[prototype]]s. Notable research topics have included [[Data model|models]], the atomic transaction concept, related [[concurrency control]] techniques, query languages and [[query optimization]] methods, [[RAID]], and more.
The database research area has several dedicated [[academic journal]]s (for example, ''[[ACM Transactions on Database Systems]]''-TODS, ''[[Data and Knowledge Engineering]]''-DKE) and annual [[Academic conference|conferences]] (e.g., [[Association for Computing Machinery|ACM]] [[SIGMOD]], ACM [[Symposium on Principles of Database Systems|PODS]], [[VLDB conference|VLDB]], [[IEEE]] ICDE).
==See also==
{{for outline|Outline of databases}}
{{Div col|colwidth=18em}}
* [[Comparison of database tools]]
* [[Comparison of object database management systems]]
* [[Comparison of object–relational database management systems]]
* [[Comparison of relational database management systems]]
* [[Data hierarchy]]
* [[Data bank]]
* [[Data store]]
* [[Database theory]]
* [[Database testing]]
* [[Database-centric architecture]]
* [[
* [[Database-as-IPC]]
* [[DBOS]]
* [[Flat-file database]]
* [[INP (database)]]
* [[Journal of Database Management]]
* Question-focused dataset
{{Div col end}}
==Notes==
{{notelist}}
==References==
{{Reflist|
==Sources==
{{refbegin|30em}}
* {{cite journal
| first1 = Charles W.
| last1 = Bachman
| author1-link = Charles Bachman
| title = The Programmer as Navigator
| date = 1973
| volume = 16
| issue = 11
| pages = 653–658
| journal = Communications of the ACM
| doi = 10.1145/355611.362534
| doi-access= free
}}
* {{cite book
| last1 = Beynon-Davies
| first1 = Paul
| date = 2003
| title = Database Systems
| edition = 3rd
| publisher = Palgrave Macmillan
| isbn = 978-1403916013
}}
* {{cite web
|last1 = Chapple
|first1 = Mike
|title = SQL Fundamentals
|date = 2005
|work = Databases
|publisher = About.com
|url = http://databases.about.com/od/sql/a/sqlfundamentals.htm
|access-date = 28 January 2009
|archive-url = https://web.archive.org/web/20090222225300/http://databases.about.com/od/sql/a/sqlfundamentals.htm
|archive-date = 22 February 2009
|url-status = live
}}
* {{cite tech report
| url = https://deepblue.lib.umich.edu/bitstream/handle/2027.42/4163/bac0294.0001.001.pdf?sequence=5&isAllowed=y
| title = Description of a set-theoretic data structure
| first1 = David L.
| last1 = Childs
| date = 1968a
| id = Technical Report 3
| series = CONCOMP (Research in Conversational Use of Computers) Project
| publisher = University of Michigan
| author-link = David L. Childs
}}
* {{cite tech report
| url = https://deepblue.lib.umich.edu/bitstream/handle/2027.42/4164/bac0293.0001.001.pdf?sequence=5&isAllowed=y
| title = Feasibility of a set-theoretic data structure: a general structure based on a reconstituted definition
| first1 = David L.
| last1 = Childs
| date = 1968b
| id = Technical Report 6
| series = CONCOMP (Research in Conversational Use of Computers) Project
| publisher = University of Michigan
| author-link = David L. Childs
}}
* {{cite book
|first1 = Raul F.
|last1 = Chong
|first2 = Xiaomei
|last2 = Wang
|first3 = Michael
|last3 = Dang
|first4 = Dwaine R.
|last4 = Snow
|chapter = Introduction to DB2
|chapter-url = http://www.ibmpressbooks.com/articles/article.asp?p=1163083
|title = Understanding DB2: Learning Visually with Examples
|edition = 2nd
|year = 2007
|publisher = IBM Press Pearson plc
|isbn = 978-0131580183
|access-date = 17 March 2013
}}
* {{cite journal
|last1 = Codd
|first1 = Edgar F.
|author1-link = Edgar F. Codd
|date = 1970
|url = http://www.seas.upenn.edu/~zives/03f/cis550/codd.pdf
|title = A Relational Model of Data for Large Shared Data Banks
|journal = Communications of the ACM
|volume = 13
|issue = 6
|pages = 377–387
|doi = 10.1145/362384.362685
|s2cid = 207549016
}}
* {{cite book
| last1 = Connolly
| first1 = Thomas M.
| last2 = Begg
| first2 = Carolyn E.
| date = 2014
| title = Database Systems – A Practical Approach to Design Implementation and Management
| edition = 6th
| isbn = 978-1292061184
| publisher = Pearson
}}
* {{cite book
|last1 = Date
|first1 = C. J.
|author1-link = Christopher J. Date
|title = An Introduction to Database Systems
|edition = 8th
|publisher = Pearson
|year = 2003
|isbn = 978-0321197849
|url-access = registration
|url = https://archive.org/details/introductiontoda0000date
}}
* {{cite journal
|first1 = Raju
|last1 = Halder
|first2 = Agostino
|last2 = Cortesi
|url = http://www.dsi.unive.it/~cortesi/paperi/CL2012.pdf
|doi = 10.1016/j.cl.2011.10.004
|title = Abstract Interpretation of Database Query Languages
|journal = Computer Languages, Systems & Structures
|volume = 38
|issue = 2
|year = 2011
|pages = 123–157
|issn = 1477-8424
|archive-date = 2024-11-23
|access-date = 2015-06-18
|archive-url = https://web.archive.org/web/20241123005344/https://www.dsi.unive.it/~cortesi/paperi/CL2012.pdf
|url-status = dead
}}
* {{cite conference
|first1 = William
|last1 = Hershey
|first2 = Carol
|last2 = Easthope
|url = https://docs.google.com/open?id=0B4t_NX-QeWDYNmVhYjAwMWMtYzc3ZS00YjI0LWJhMjgtZTYyODZmNmFkNThh
|title = A set theoretic data structure and retrieval language
|conference = Spring Joint Computer Conference, May 1972
|journal = ACM SIGIR Forum
|volume = 7
|issue = 4
|year = 1972
|pages = 45–55
|doi = 10.1145/1095495.1095500
}}
* {{cite book
| last1 = Nelson
| first1 = Anne Fulcher
| last2 = Nelson
| first2 = William Harris Morehead
| date = 2001
| title = Building Electronic Commerce: With Web Database Constructions
| publisher = Prentice Hall
| isbn = 978-0201741308
}}
* {{cite journal
|first1 = Ken
|last1 = North
|url = http://drdobbs.com/blogs/database/228700616
|title = Sets, Data Models and Data Independence
|journal = Dr. Dobb's
|date = 10 March 2010
|archive-url = https://web.archive.org/web/20121024064523/http://www.drdobbs.com/database/sets-data-models-and-data-independence/228700616
|archive-date = 24 October 2012
|url-status = live
}}
* {{cite book
|last1 = Tsitchizris
|first1 = Dionysios C.
|last2 = Lochovsky
|first2 = Fred H.
|date = 1982
|title = Data Models
|isbn = 978-0131964280
|publisher = Prentice–Hall
|url = https://archive.org/details/datamodels00tsic
}}
* {{cite book
|first1 = Jeffrey
|last1 = Ullman
|first2 = Jennifer
|last2 = Widom
|date = 1997
|title = A First Course in Database Systems
|publisher = Prentice–Hall
|isbn = 978-0138613372
|url = https://archive.org/details/firstcourseindat00ullm
}}
* {{Citation
| title = SQL/XML:2006 – Evaluierung der Standardkonformität ausgewählter Datenbanksysteme
| last1 = Wagner
| first1 = Michael
| year = 2010
| publisher = Diplomica Verlag
| isbn = 978-3836696098
}}
{{refend}}
==Further reading==
* [[Ling Liu (computer scientist)|Ling Liu]] and Tamer M. Özsu (Eds.) (2009). "[https://www.springer.com/computer/database+management+&+information+retrieval/book/978-0-387-49616-0 Encyclopedia of Database Systems], 4100 p. 60 illus. {{ISBN|978-0-387-49616-0}}.
* Gray, J. and Reuter, A. ''Transaction Processing: Concepts and Techniques'', 1st edition, Morgan Kaufmann Publishers, 1992.
* Kroenke, David M. and David J. Auer. ''Database Concepts.'' 3rd ed. New York: Prentice, 2007.
* [[Raghu Ramakrishnan]] and [[Johannes Gehrke]], ''[http://pages.cs.wisc.edu/~dbbook/ Database Management Systems]''.
* [[Abraham Silberschatz]], [[Henry F. Korth]], S. Sudarshan, ''[http://www.db-book.com/ Database System Concepts]''.
* {{cite book|last1=Lightstone |first1=S. |first2=T. |last2=Teorey |first3=T. |last3=Nadeau |title=Physical Database Design: the database professional's guide to exploiting indexes, views, storage, and more |publisher=Morgan Kaufmann Press |year=2007 |isbn=978-0-12-369389-1 |ref=none}}
* Teorey, T.; Lightstone, S. and Nadeau, T. ''Database Modeling & Design: Logical Design'', 4th edition, Morgan Kaufmann Press, 2005. {{ISBN|0-12-685352-5}}.
* ''[https://www.youtube.com/@CMUDatabaseGroup/playlists CMU Database courses playlist]''
* ''[https://ocw.mit.edu/courses/6-830-database-systems-fall-2010/ MIT OCW 6.830 | Fall 2010 | Database Systems]''
* ''[https://cs186berkeley.net Berkeley CS W186]''
==External links==
*[http://www.fileextension.org/DB DB File extension] – information about files with the DB extension
{{Sister bar|wikt=database|commons=Category:Database|v=Topic:Databases}}
{{Navboxes
|list=
{{Computer science}}
{{Database}}
{{Databases}}
{{Data warehouse}}
{{Semantic Web}}
}}
{{Authority control}}
[[Category:Databases| ]]
[[Category:Database management systems| ]]
| |||