Data mapping: Difference between revisions

Revision as of 18:01, 1 February 2019

Revision as of 15:44, 2 May 2019

Line 23:

==Semantic mapping==

[[Semantic mapper|Semantic mapping]] is similar to the auto-connect feature of data mappers with the exception that a [[metadata registry]] can be consulted to look up data element synonyms. For example, if the source system lists ''FirstName'' but the destination lists ''PersonGivenName'', the mappings will still be made if these data elements are listed as [[synonyms]] in the metadata registry. Semantic mapping is only able to discover exact matches between columns of data and will not discover any transformation logic or exceptions between columns.

Data lineage is a track of the life cycle of each piece of data as it is ingested, processed, and output by the analytics system. This provides visibility into the analytics pipeline and simplifies tracing errors back to their sources. It also enables replaying specific portions or inputs of the data flow for step-wise debugging or regenerating lost output. In fact, database systems have used such information, called data provenance, to address similar validation and debugging challenges already.<ref>De, Soumyarupa. (2012). Newt : an architecture for lineage based replay and debugging in DISC systems. UC San Diego: b7355202. Retrieved from: https://escholarship.org/uc/item/3170p7zn</ref>