Overlay (programming): Difference between revisions

Content deleted Content added
better links
improved refs
Line 1:
{{Distinguish|Overload (programming)|Exec (system call)|Overlapping code|Shared code}}
{{Use dmy dates|date=July 2019|cs1-dates=y}}
{{Use list-defined references|date=July 2022}}
In a general computing sense, '''overlaying''' means "the process of transferring a [[Block (data storage)|block]] of program code or other data into [[main memory]], replacing what is already stored".<ref name="OEDOED_2015"/> Overlaying is a [[computer programming|programming]] method that allows programs to be larger than the computer's [[main memory]].<ref name="CommodoreButterfield_1986"/> An [[embedded system]] would normally use overlays because of the limitation of [[physical memory]], which is [[internal memory]] for a [[system-on-chip]], and the lack of [[virtual memory]] facilities.
 
==Usage==
Constructing an overlay program involves manually dividing a program into self-contained [[object code]] blocks called '''overlays''' or '''links''', generally laid out in a [[tree structure]].{{efn|In [[OS/360 and successors]], there may be multiple ''regions''{{efn|This has nothing to do with the term ''region'' in [[OS/360 and successors#MVT|MVT]] storage management.}} each containing a complete overlay tree.}} ''Sibling'' segments, those at the same depth level, share the same memory, called ''overlay region''{{efn|The nomenclature varies depending on the system, e.g., in [[OS/360]] ''region'' refers to an entire overlay tree.}} or ''destination region''. An overlay manager, either part of the [[operating system]] or part of the overlay program, loads the required overlay from [[Auxiliary memory|external memory]] into its destination region when it is needed; this may be automatic or via explicit code. Often [[Linker (computing)|linker]]s provide support for overlays.<ref name="GNULinkGNULink_2008"/>
 
==Example==
Line 21 ⟶ 22:
INCLUDE SYSLIB(MOD7)
</pre>
 
<pre>
+--------------+
Line 42:
+-------------+ +-------------+
</pre>
These statements define a tree consisting of the permanently resident segment, called the ''root'', and two overlays A and B which will be loaded following the end of MOD2. Overlay A itself consists of two overlay segments, AA, and AB. At execution time overlays A and B will both utilize the same memory locations; AA and AB will both utilize the same locations following the end of MOD3.
 
These statements define a tree consisting of the permanently resident segment, called the ''root'', and two overlays A and B which will be loaded following the end of MOD2. Overlay A itself consists of two overlay segments, AA, and AB. At execution time overlays A and B will both utilize the same memory locations; AA and AB will both utilize the same locations following the end of MOD3.
 
All the segments between the root and a given overlay segment are called a ''path''.
 
==Applications==
{{As of|2015}}, most business applications are intended to run on platforms with [[virtual memory]]. A developer on such a platform can design a program as if the memory constraint does not exist unless the program's [[working set]] exceeds the available physical memory. Most importantly, the architect can focus on the problem being solved without the added design difficulty of forcing the processing into steps constrained by the overlay size. Thus, the designer can use higher-level programming languages that do not allow the programmer much control over size (e.g. [[Java (programming language)|Java]], [[C++]], [[Smalltalk]]).
 
Still, overlays remain useful in embedded systems.<ref name="LevineLevine_2000"/> Some low-cost processors used in [[embedded systems]] do not provide a [[memory management unit]] (MMU). In addition many embedded systems are [[Real-time computing|real-time]] systems and overlays provide more determinate response-time than [[paging]]. For example, the [[Space Shuttle]] ''Primary Avionics System Software (PASS)'' uses programmed overlays.<ref name="ShuttleNRC_1993"/>
 
Even on platforms with virtual memory, [[Software componentry|software components]] such as [[codec]]s may be [[Coupling (computer science)#Low coupling|decoupled]] to the point where they can be [[Library (computing)#Dynamic loading|loaded in and out as needed]].
 
==Historical use==
IBM introduced the concept of a ''chain job'' <ref name="IBM_1963_FORTRAN"/>{{cite manual
| title = IBM 7090/7094 Programming Systems – FORTRAN II Programming
| id = C28-6054-4
| date = August 1963
| section = Chapter 12 The Chain Job
| section-url = http://bitsavers.org/pdf/ibm/7090/C28-6054-4_7090_FORTRANII.pdf#page=34
| pages = 34–35
| url = http://bitsavers.org/pdf/ibm/7090/C28-6054-4_7090_FORTRANII.pdf
| work = Systems Reference Library
| publisher = IBM
}}
</ref>
in FORTRAN II,
The program had to explicitly call the CHAIN subroutine to load a new link,
and the new link replaced all of the old link's storage except for the Fortran COMMON area.
 
IBM introduced more general overlay handling<ref name="IBM_1963_IBJOB"/>{{cite manualin [[IBSYS]]/IBJOB, including a tree structure and automatic loading of links as part of CALL processing.
| title = IBM 7090/7094 Programming Systems – IBJOB Processor – Overlay feature of IBLDR
| id = C28-6331
| year = 1963
| url = http://bitsavers.org/pdf/ibm/7090/C28-6331_IBLDR_overlay_1963.pdf
| work = Systems Reference Library
| publisher = IBM
}}
</ref> in [[IBSYS]]/IBJOB, including a tree structure and automatic loading of links as part of CALL processing.
 
In OS/360, IBM extended the overlay facility of IBLDR by allowing an overlay program to have independent overlay regions, each with its own overlay tree. OS/360 also had a simpler overlay system for transient [[Supervisor call|SVC]] routines, using 1024-byte SVC transient areas.
 
<!-- Need information on other vendors here. -->
 
In the [[home computer]] era overlays were popular because the operating system and many of the computer systems it ran on lacked virtual memory and had very little RAM by current standards: the original [[IBM PC]] had between 16K and 64K, depending on configuration. Overlays were a popular technique in [[Commodore BASIC]] to load graphics screens.<ref name="CommodoreButterfield_1986"/>
 
"Several DOS linkers in the 1980s supported [overlays] in a form nearly identical to that used 25 years earlier on mainframe computers."<ref name="LevineLevine_2000"/> [[Binary file]]s containing memory overlays had a de facto standard extension, '''.OVL'''. This file type was used among others by [[WordStar]], [[dBase]], and the ''Enable'' DOS office automation software package from [[Enable Software, Inc.]] The [[GFA BASIC]] compiler was able to produce .OVL files.
 
== See also ==
Line 101 ⟶ 80:
==References==
{{reflist|refs=
<ref name="OEDOED_2015">{{cite web |title=Oxford Dictionaries |date=2015-11-26 | url=https://www.oxforddictionaries.com/definition/american_english/overlay |titleaccess-date=Oxford2022-07-10 Dictionaries|url-status=live |archive-url=https://web.archive.org/web/20220710211317/https://www.lexico.com/en/definition/overlay |archive-date=2022-07-10}}</ref>
<ref name="CommodoreButterfield_1986">{{cite book |authortitle=Loading And Linking Commodore Programs |chapter=Part 4: Overlaying |issue=73 |date=June 1986 |page=74 |magazine=[[Compute!]] |editor-first=Jim |authoreditor-last=Butterfield |url=http://www.atarimagazines.com/compute/issue73/loading_and_linking.php |titleaccess-date=Loading And Linking Commodore Programs2022-07-10 |chapterurl-status=Partlive 4|archive-url=https://web.archive.org/web/20220710211439/https://www.atarimagazines.com/compute/issue73/loading_and_linking.php Overlaying|archive-date=2022-07-10 |issuequote=73This lets you run programs which are, in effect, much larger than the amount of memory in your computer.}}</ref>
<ref name="GNULink_2008">{{cite web |title=The GNU Linker documentation: Overlay Description |date=2008-06-03 |url=http://sourceware.org/binutils/docs/ld/Overlay-Description.html |access-date=2022-07-10 |url-status=live |archive-url=https://web.archive.org/web/20220623060507/https://sourceware.org/binutils/docs/ld/Overlay-Description.html |archive-date=2022-06-23}}</ref>
|date=June 1986 |page=74 |magazine=[[Compute!]] |quote=This lets you run programs which are, in effect, much larger than the amount of memory in your computer.}}</ref>
<ref name="Levine_2000">{{cite book |title=Linkers & Loaders |author-last=Levine |author-first=John R. |author-link=John R. Levine |date=2000 |publisher=[[Morgan Kaufmann Publishers]] |isbn=1-55860-496-0 |page=177 |url=http://linker.iecc.com/ |access-date=2022-07-10 |url-status=live |archive-url=https://web.archive.org/web/20220406090302/https://linker.iecc.com/ |archive-date=2022-04-06}} [http://www.iecc.com/linker/<!-- https://web.archive.org/web/20220705125042/https://www.iecc.com/linker/ -->]</ref>
<ref name="GNULink">{{cite web |date=2008-06-03 |url=http://sourceware.org/binutils/docs/ld/Overlay-Description.html |title=The GNU Linker documentation: Overlay Description}}</ref>
<ref name="LevineNRC_1993">{{cite bookweb |author-lasttitle=LevineAn |author-first=JohnAssessment R.of |author-link=JohnSpace R.Shuttle LevineFlight |title=LinkersSoftware &Development LoadersProcesses |date=20001993 |publisher=[[MorganThe KaufmannNational Publishers]]Academies Press |author=National Research Council |publication-place=Washington, DC, USA |isbn=1978-558600-496309-004880-4 |doi=10.17226/2222 |url=http://linkerwww.ieccnap.comedu/openbook.php?record_id=2222&page=39 |pageaccess-date=1772012-10-29}} [http://www.iecc.com/linker/](208 pages)</ref>
<ref name="IBM_1963_FORTRAN">{{cite manual |title=IBM 7090/7094 Programming Systems – FORTRAN II Programming |id=C28-6054-4 |date=August 1963 |work=Systems Reference Library |publisher=[[IBM Corporation]] |pages=34–35 |section=Chapter 12 The Chain Job |section-url=http://bitsavers.org/pdf/ibm/7090/C28-6054-4_7090_FORTRANII.pdf#page=34 |url=http://bitsavers.org/pdf/ibm/7090/C28-6054-4_7090_FORTRANII.pdf |access-date=2022-07-10 |url-status=live |archive-url=https://web.archive.org/web/20220315144942/http://bitsavers.org/pdf/ibm/7090/C28-6054-4_7090_FORTRANII.pdf |archive-date=2022-03-15}}</ref>
<ref name="Shuttle">{{cite web |title=An Assessment of Space Shuttle Flight Software Development Processes |url=http://www.nap.edu/openbook.php?record_id=2222&page=39 |access-date=2012-10-29}}</ref>
<ref name="IBM_1963_IBJOB">{{cite manual |title=IBM 7090/7094 Programming Systems – IBJOB Processor – Overlay feature of IBLDR |id=Form C28-6331 File No. 7090-27 |edition=1 |date=May 1963 |work=Systems Reference Library |publisher=[[IBM Corporation]] |url=http://bitsavers.org/pdf/ibm/7090/C28-6331_IBLDR_overlay_1963.pdf |access-date=2021-12-26 |url-status=live |archive-url=https://web.archive.org/web/20220315161341/http://bitsavers.org/pdf/ibm/7090/C28-6331_IBLDR_overlay_1963.pdf |archive-date=2022-03-15}} (8 pages)</ref>
}}
 
==ExternalFurther linksreading==
* {{cite book |publisher=[[IBM Corporation]] |title=IBM OS Linkage Editor and Loader |id=Order No. GC28-6538-9 File No. S360-31 |version=Release 21 |date=March 1972 |orig-yeardate=January 1972 |edition=10 |publisher=[[IBM Corporation]] |pages=244 |url=http://www.bitsavers.org/pdf/ibm/360/os/R21.0_Mar72/GC28-6538-9_OS_Linkage_Editor_and_Loader_Release_21_Jan72.pdf |url-status=live |archive-url=https://web.archive.org/web/20220710205449/http://www.bitsavers.org/pdf/ibm/360/os/R21.0_Mar72/GC28-6538-9_OS_Linkage_Editor_and_Loader_Release_21_Jan72.pdf |archive-date=2022-07-10}} (250 pages)
* {{cite manual
| title = IBM 7090/7094 Programming Systems IBJOB Processor Overlay Feature of IBLDR
| date = May 1963
| edition = First
| id = C28-6331
| url = http://bitsavers.org/pdf/ibm/7090/C28-6331_IBLDR_overlay_1963.pdf
| access-date = December 26, 2021
| work = Systems Reference Library
| publisher = [[IBM Corporation]]
}}
* {{cite book |publisher=[[IBM Corporation]] |title=IBM OS Linkage Editor and Loader |id=GC28-6538 |version=Release 21 |date=March 1972 |orig-year=January 1972 |pages=244 |url=http://www.bitsavers.org/pdf/ibm/360/os/R21.0_Mar72/GC28-6538-9_OS_Linkage_Editor_and_Loader_Release_21_Jan72.pdf}}
 
[[Category:Virtual memory]]