XML pipeline: Difference between revisions

Revision as of 17:39, 11 July 2013

Latest revision as of 04:23, 11 August 2025

(22 intermediate revisions by 16 users not shown)

Line 1:

{{Short description|Part of Extensible Markup Language}}
In [[software]], an '''XML Pipelinepipeline''' is formed when [[XML]] (Extensible [[Markup language|Markup Language]]) processes, especially [[XML transformation language|XML transformations]] and [[XML validation]]s, are connected.

For instance, given two transformations T₁ and T₂, the two can be connected so that an input XML document is transformed by T₁ and then the output of T₁ is fed as input document to T₂. Simple pipelines like the one described above are called ''linear''; a single input document always goes through the same sequence of transformations to produce a single output document.

== Linear Operationsoperations ==

Linear operations can be divided in at least two parts

=== Micro-operations ===

They operate at the inner document level

* '''Rename''' - renames elements or attributes without modifying the content

* '''Replace''' - replaces elements or attributes

Line 29 ⟶ 30:

* '''wrap-sequence''' - takes a sequence of documents as input and wraps them into one or more documents

== Non-linear ==

Non-linear operations on pipelines may include:

Line 37 ⟶ 38:

* '''Tees''' — where a document is fed to multiple transformations potentially happening in parallel

* '''Aggregations''' — where multiple documents are aggregated into a single document

* '''Exception Handling''' — where failures in processing can result in an alternate pipeline being processed

Some standards also categorize transformation as macro (changes impacting an entire file) or micro (impacting only an element or attribute)

== XML Pipelinepipeline languages ==

XML pipeline languages are used to define pipelines. A program written with an XML pipeline language is implemented by software known as an XML pipeline engine, which creates processes, connects them together and finally executes the pipeline. Existing XML pipeline languages include:

=== Standards ===

* '''[[XProc|XProc: An XML Pipeline Language]]''' is a W3C Recommendation <ref>{{cite web|url=http://www.w3.org/TR/xproc |title=XProc: An XML Pipeline Language |publisher=W3.org |access-date= |accessdate=2013-06-14}}</ref> for defining linear and non-linear XML pipelines.

=== Product-specific ===

* '''W3C XML Pipeline Definition Language''' is specified in a W3C Note.<ref>{{cite web|title=W3C XML Pipeline Definition Language|url=http://www.w3.org/TR/xml-pipeline/}}</ref>

* '''W3C XML Pipeline Language (XPL) Version 1.0 (Draft) <ref>{{cite web|url=http://www.w3.org/Submission/xpl/ |title=XML Pipeline Language (XPL) Version 1.0 (Draft) |publisher=W3.org |access-date= |accessdate=2013-06-14}}</ref><ref>{{cite web|url=http://www.w3.org/TR/xml-pipeline/ |title=XML Pipeline Definition Language Version 1.0 |publisher=W3.org |date=2002-02-28 |accessdateaccess-date=2013-06-14}}</ref>''' is specified in a W3C Submission and a component of Orbeon Presentation Server OPS (now called Orbeon Forms). This specification provides an implementation of an earlier version of the language. XPL allows the declaration of complex pipelines with conditionals, loops, tees, aggregations, and sub-pipelines. XProc is roughly a superset of XPL.<ref name=xpl_and_xproc>{{cite web|title=XML pipelines: XPL and XProc|url=http://blog.orbeon.com/2007/05/xml-pipelines-xpl-and-xproc_22.html|publisher=Orbeon|accessdateaccess-date=14 March 2012|date=22|month= May|year= 2007}}</ref>

* '''[[Apache Cocoon|Cocoon]] sitemaps''' allow, among other functionality, the declaration of XML pipelines. Cocoon sitemaps are one of the earliest implementations of the concept of XML pipeline.

* '''smallx XML Pipelines''' are used by the smallx project.

* '''ServingXML''' defines a vocabulary for expressing flat-XML, XML-flat, flat-flat, and XML-XML transformations in pipelines.

* '''[http://www.polarlake.com/reference-data-distribution/xml-data-pipelining-platform PolarLake Circuit Markup Language] {{Webarchive|url=https://web.archive.org/web/20100830081429/http://www.polarlake.com/reference-data-distribution/xml-data-pipelining-platform |date=2010-08-30 }}''' used by PolarLake's runtime to define [http://www.polarlake.com/reference-data-distribution/xml-data-pipelining-platform XML pipelines] {{Webarchive|url=https://web.archive.org/web/20100830081429/http://www.polarlake.com/reference-data-distribution/xml-data-pipelining-platform |date=2010-08-30 }}. Circuits are collections of paths through which fragments of XML stream (usually as SAX or DOM events). Components are placed on paths to interact with the stream (and/or the outside world) in a low latency process.

* '''xmlsh''' is a scripting language based on the unix shells which natively supports xml and text pipelines [http://www.xmlsh.org]

* '''[[Stylus Studio]] XML Pipeline''' is a visual grammar which defines the following operations: Input, Output, [[XQuery]], XSLT, Validate, XSL-FO to PDF, Convert To XML, Convert From XML, Choose, Warning, Stop.

== Pipe Granularitygranularity ==

Different XML Pipeline implementations support different granularity of flow.

Line 65 ⟶ 67:

== Standardization ==

Until May 2010, there was no widely used standard for XML pipeline languages. However, with the introduction of the [[W3C]] XProc standard as a [[W3C recommendation#Recommendation (REC)|W3C Recommendation]] as of May 2010,<ref>{{cite web|url=http://www.w3.org/TR/xproc/ |title=XProc: An XML Pipeline Language |publisher=W3.org |access-date= |accessdate=2013-06-14}}</ref> widespread adoption can be expected.

==XML Pipeline History ==

* 1972 [[Douglas McIlroy]] of [[Bell Laboratories]] adds the pipe operator to the [[UNIX]] command shell. This allows the output from one shell program to go directly into input of another shell program without going to disk. This allowed programs such as the UNIX [[awk]] and [[sed]] to be specialized yet work together [http://www.cs.dartmouth.edu/~doug/].<ref>{{cite web|url=httphttps://cmwww.bell-labs.com/cmusr/csdmr/who/dmrwww/hist.html |title=Early Unix history and evolution |publisher=Cmwww.bell-labs.com |access-date= |accessdate=2013-06-14 |url-status=live |archive-url=https://web.archive.org/web/20150408054606/http://cm.bell-labs.com/cm/cs/who/dmr/hist.html |archive-date=April 8, 2015 }}</ref> For more details see [[Pipeline (Unix)]].

* 1993 [http://seanmcgrath.blogspot.com Sean McGrath] developed a C++ toolkit for [[SGML]] processing.<ref>{{cite web|url=httphttps://xpipe.sourceforge.net/Articles/Miscellaneous/fog0000000020.html |title=FAQ |publisher=Xpipe.sourceforge.net |date=2001-12-09 |accessdateaccess-date=2013-06-14}}</ref>

* 1998 [[Stefano Mazzocchi]] releases the first version of [[Apache Cocoon]], one of the first software programs to use XML pipelines.

* 1998 [http://www.polarlake.com/ PolarLake] {{Webarchive|url=https://web.archive.org/web/20080418064200/http://www.polarlake.com/ |date=2008-04-18 }} build [http://www.polarlake.com/reference-data-distribution/xml-data-pipelining-platform XML Operating System]{{Dead link|date=July 2025 |bot=InternetArchiveBot |fix-attempted=yes }}, which includes [http://www.polarlake.com/reference-data-distribution/xml-data-pipelining-platform XML Pipelining]{{Dead link|date=July 2025 |bot=InternetArchiveBot |fix-attempted=yes }}.

* 2002 Notes submitted by Norman Walsh and [[Eve Maler]] from [[Sun Microsystems]], as well as a W3C Submission submitted in 2005 by [[Erik Bruchez]] and [[Alessandro Vernet]] from [[Orbeon]], were important steps toward spawning an actual standardization effort. While neither submission directly became a W3C recommendation, they were considered key sources of inspiration for the W3C XML Processing [[Working group|Working Group]].

* September 2005 W3C XML Processing [[Working group|Working Group]] started. The task of this working group was to create a specification for an XML pipelining language.

Line 84 ⟶ 86:

* [[W3C recommendation]]

* [[XSLT]]

* [[SYSQ]]

== References ==

* [[kyachahiye]]

<references/>

== External links ==

Line 100 ⟶ 104:

=== Product specific ===

* [http://www.data2type.de/en/xml-xslt-xslfo/xproc/ XProc tutorial and reference]

* [http://www.oracle.com/technology/tech/xml/xdkhome.html Oracle's XML Pipeline Definition Language Controller Implementation] ''Part of XML Developer's kit, no individual download''

* [http://cocoon.apache.org/2.1/userdocs/concepts/sitemap.html Cocoon sitemap]

* [http://www.1060research-server-1.co.uk/docs/3.2.0/book/tutorial/doc_guide_xml_pipelines_main.html NetKernel XML Pipelines]

* [http://idealliance.org/proceedings/xtech05/papers/04-03-01/ Managing Complex Document Generation through Pipelining] {{Webarchive|url=https://web.archive.org/web/20051028015614/http://idealliance.org/proceedings/xtech05/papers/04-03-01/ |date=2005-10-28 }}

* [http://www.orbeon.com/ois/doc/reference-xpl-pipelines XML Pipeline Language (XPL) Documentation] {{Webarchive|url=https://web.archive.org/web/20041212090900/http://www.orbeon.com/ois/doc/reference-xpl-pipelines |date=2004-12-12 }}

* [http://norman.walsh.name/2004/06/20/sxpipe SXPipe]

* [http://www.polarlake.com/ PolarLake Reference data management] {{Webarchive|url=https://web.archive.org/web/20080418064200/http://www.polarlake.com/ |date=2008-04-18 }} PolarLake XML circuits and reference data management

* [https://smallx.dev.java.net/ smallx] {{Webarchive|url=https://web.archive.org/web/20070708133342/https://smallx.dev.java.net/ |date=2007-07-08 }}

* [httphttps://servingxml.sourceforge.net/ ServingXML]

* [http://www.stylusstudio.com/xml/pipeline.html XML Pipeline Implementation from Stylus Studio] - This program allows XML transforms to be chained together along with other operations on XML files such as validation and [[HTML tidyTidy]].

* [http://xmlpipelineserver.com IVI XML Pipeline Server] XML Pipeline Server is an implementation for the Stylus Studio XML Pipeline language

* [http://xproc.org/ Norman Walsh's XProc web site] - Norman Walsh is the chair of the W3C XProc standards committee.

* [httphttps://yax.sourceforge.net/ yax - an XProc Implementation] currently with commandline and [[Apache Ant|Apache ant]] interface

* [https://web.archive.org/web/20070209060027/http://pipes.yahoo.com/ Yahoo! Pipes] let's users create multi-source data mashups in a web-based visual environment

* [http://www.xmlsh.org xmlsh] A shell for manipulating xml based on the unix shells. Supports in-process multithreaded xml and text processing pipelines.

* [http://www.gerixsoft.com/blog/xslt/xml-pipeline-xslt]{{Dead link|date=July 2025 |bot=InternetArchiveBot |fix-attempted=yes }} How to implement XML Pipeline in XSLT

* [http://xmlcalabash.com/ Calabash is an implementation of XProc]

* [https://community.emc.com/community/edn/xmltech/ Calumet] is an XProc implementation from EMC

* [http://code.google.com/p/quixproc QuiXProc] is an XProc implementation of Innovimax

<references/>

{{DEFAULTSORT:Xml Pipeline}}