Big data maturity model: Difference between revisions

Revision as of 14:10, 21 May 2017

Latest revision as of 08:05, 5 January 2025

(48 intermediate revisions by 22 users not shown)

Line 1:

{{short description|Aspect of computer science}}

Big Data maturity can be defined as “the evolution of an organization to integrate, manage, and leverage all relevant internal and external data sources” and involves building an ecosystem that includes technologies, data management, analytics, governance and organizational components <ref name=":0">{{Cite journal|last=Halper|first=Fern|date=2016|title=A Guide to Achieving Big Data Analytics Maturity|url=|journal=TDWI Benchmark guide|volume=|pages=|via=}}</ref>.

{{cleanup reorganize|date=May 2017}}

'''Big data maturity models (BDMM)''' are the artifacts used to measure big data maturity.<ref name=":1">{{Cite journal|last=Braun|first=Henrik|date=2015|title=Evaluation of Big Data Maturity Models: A benchmarking study to support big data assessment in organizations|journal=Masters Thesis – Tampere University of Technology}}</ref> These models help organizations to create structure around their big data capabilities and to identify where to start.<ref>Halper, F., & Krishnan, K. (2014). ''TDWI Big Data Maturity Model Guide.'' TDWI Research.</ref> They provide tools that assist organizations to define goals around their big data program and to communicate their big data vision to the entire organization. BDMMs also provide a methodology to measure and monitor the state of a company's big data capability, the effort required to complete their current stage or phase of maturity and to progress to the next stage. Additionally, BDMMs measure and manage the speed of both the progress and adoption of big data programs in the organization.<ref name=":1" />

== Overview ==

'''Big Data Maturity Models''' (BDMM) are the artefacts used to measure Big Data maturity <ref name=":1">{{Cite journal|last=Braun|first=Henrik|date=2015|title=Evaluation of Big Data Maturity Models: A benchmarking study to support big data assessment in organizations|url=|journal=Masters Thesis - Tampere University of Technology|volume=|pages=|via=}}</ref>. These models help organizations to create structure around their Big Data capabilities and to identify where to start<ref name=":0" />. They provide tools that assist organizations to define goals around their big data program and to communicate their big data vision to the entire organization. BDMMs also provide a methodology to measure and monitor the state of a company’s big data capability, the effort required to complete their current stage or phase of maturity and to progress to the next stage. Additionally, BDMMs measure and manage the speed of both the progress and adoption of big data programs in the organization <ref name=":1" />.

The goals of BDMMs are:

* To provide a capability assessment tool that generates specific focus on big data in key organizational areas

* To help guide development milestones

* To avoid pitfalls in establishing and building big data capabilities

Key organizational areas refer to "people, process and technology" and the subcomponents include<ref>{{Cite web|url=http://ibmdatamag.com/2014/09/measuring-maturity-of-big-data-initiatives/|title=Measuring maturity of big data initiatives|last=Krishnan|date=2014|access-date=2017-05-21|archive-url=https://web.archive.org/web/20150316120424/http://ibmdatamag.com/2014/09/measuring-maturity-of-big-data-initiatives/|archive-date=2015-03-16|url-status=dead}}</ref> alignment, architecture, data, [[data governance]], delivery, development, measurement, program governance, scope, skills, sponsorship, [[statistical model]]ling, technology, value and visualization.

The stages or phases in BDMMs depict the various ways in which data can be used in an organization and is one of the key tools to set direction and monitor the health of an organization's big data programs.<ref name=":2">{{Cite journal|last=El-Darwiche |display-authors=etal |date=2014|title=Big Data Maturity: An action plan for policymakers and executives|journal=World Economic Forum}}</ref><ref name=":3">{{Cite web|url=http://www.radcliffeadvisory.com/research/download.php?file=RAS_BD_MatMod.pdf|title=Leverage a Big Data Maturity model to build your big data roadmap|date=2014|access-date=2017-05-21|archive-url=https://web.archive.org/web/20170802005853/http://www.radcliffeadvisory.com/research/download.php?file=RAS_BD_MatMod.pdf|archive-date=2017-08-02|url-status=dead}}</ref>

1.      To provide a capability assessment tool that generates specific focus on big data in key organizational areas,

An underlying assumption is that a high level of big data maturity correlates with an increase in revenue and reduction in operational expense. However, reaching the highest level of maturity involves major investments over many years.<ref name=":0">{{Cite journal|last=Halper|first=Fern|date=2016|title=A Guide to Achieving Big Data Analytics Maturity|journal=TDWI Benchmark Guide}}</ref> Only a few companies are considered to be at a "mature" stage of big data and analytics.<ref>{{Cite web |date=December 15, 2023 |title=How to summit "data maturity mountain" and make data your superpower |url=https://www.fastcompany.com/90998039/how-to-summit-data-maturity-mountain-and-make-data-your-superpower |access-date=October 8, 2024 |website=Fast Company |author1=Altair }}</ref> These include internet-based companies (such as [[LinkedIn]], [[Facebook]], and [[Amazon.com|Amazon]]) and other non-Internet-based companies, including financial institutions (fraud analysis, real-time customer messaging and behavioral modeling) and retail organizations ([[Clickstream|click-stream]] analytics together with self-service analytics for teams).<ref name=":0" />

2.      To help guide development milestones, and

== Categories ==

3.      To avoid pitfalls in establishing and building a big data capabilities.

Big data maturity models can be broken down into three broad categories namely:<ref name=":1" />

* Descriptive

* Comparative

* Prescriptive

== Descriptive ==

Key organizational areas refer to “People, Process and Technology” and the subcomponents include (Krishnan, 2014):

Descriptive models assess the current firm maturity through qualitative positioning of the firm in various stages or phases. The model does not provide any recommendations as to how a firm would improve their big data maturity.

=== Big data and analytics maturity model (IBM model) ===

1.      Alignment,

This descriptive model aims to assess the value generated from big data investments towards supporting strategic business initiatives.

'''Maturity levels'''

2.      Architecture,

The model consists of the following maturity levels:

3.      Data,

* Ad-hoc

* Foundational

* Competitive differentiating

* Break away

'''Assessment areas'''

Maturity levels also cover areas in matrix format focusing on: business strategy, information, analytics, culture and execution, architecture and governance.

4.      Data governance,

<ref>{{Cite news|url=http://www.ibmbigdatahub.com/blog/big-data-analytics-maturity-model|title=Big Data & Analytics Maturity Model|work=IBM Big Data & Analytics Hub|access-date=2017-05-21|language=en}}</ref>

5.      Delivery,

=== Knowledgent big data maturity assessment ===

6.      Development,

Consisting of an assessment survey, this big data maturity model assesses an organization's readiness to execute big data initiatives. Furthermore, the model aims to identify the steps and appropriate technologies that will lead an organization towards big data maturity.<ref>{{Cite web|url=https://bigdatamaturity.knowledgent.com|title=Home {{!}} Big Data Maturity Assessment|website=bigdatamaturity.knowledgent.com|access-date=2017-05-21|archive-url=https://web.archive.org/web/20150214173709/https://bigdatamaturity.knowledgent.com/|archive-date=2015-02-14|url-status=dead}}</ref>

== Comparative ==

7.      Measurement,

8.      Program governance,

9.      Scope,

10.   Skills,

11.   Sponsorship,

12.   Statistical modelling,

13.   Technology,

14.   Value, and

15.   Visualization.

The stages or phases in BDMMs depict the various ways in which data can be used in an organization and is one of the key tools to set direction and monitor the health of organization’s big data programs<ref name=":2">{{Cite journal|last=El-Darwiche et. al.|first=|date=2014|title=Big Data Maturity: An action plan for policymakers and executives|url=|journal=World Economic Forum|volume=|pages=|via=}}</ref> <ref name=":3">{{Cite web|url=http://www.radcliffeadvisory.com/research/download.php?file=RAS_BD_MatMod.pdf|title=Leverage a Big Data Maturity model to build your big data roadmap|last=|first=|date=2014|website=|archive-url=|archive-date=|dead-url=|access-date=}}</ref>.   

An underlying assumption is that a high level of big data maturity correlates with an increase in revenue and reduction in operational expense. However, reaching the highest level of maturity involves major investments over many years <ref name=":0" />. Only a few companies are considered to be at a “Mature” stage of big data and analytics. These include internet-based companies (such as [[LinkedIn]], [[Facebook]], and [[Amazon.com|Amazon]]) and other non-internet-based companies, including financial institutions (fraud analysis, real-time customer messaging and behavioral modeling) and retail organizations (click-stream analytics together with self-service analytics for teams). These companies are dealing with vast amounts of big data and employ many data scientists to create new and exciting ways to analyze and act on their data<ref name=":0" />.    

== Categories of Big Data Maturity Models ==

Big data maturity models can be broken down into three broad categories namely<ref name=":1" />:

1.     Descriptive,

2.     Comparative and

3. Prescriptive models.    

=== Descriptive Models ===

==== '''Big Data & Analytics Maturity Model (IBM model)'''<ref>{{Cite news|url=http://www.ibmbigdatahub.com/blog/big-data-analytics-maturity-model|title=Big Data &amp; Analytics Maturity Model|work=IBM Big Data & Analytics Hub|access-date=2017-05-21|language=en}}</ref> ====

This descriptive model aims to assess the value generated from big data investments towards supporting strategic business initiatives. The model consists of the following maturity levels:  Ad-hoc, Foundational, Competitive Differentiating and Break Away.

Maturity levels also cover areas in matrix format focusing on: Business Strategy, Information, Analytics, Culture and Execution, Architecture and Governance.

==== '''Knowledgent Big Data Maturity Assessment'''<ref>{{Cite web|url=https://bigdatamaturity.knowledgent.com|title=Home {{!}} Big Data Maturity Assessment|website=bigdatamaturity.knowledgent.com|access-date=2017-05-21}}</ref> ====

Consisting of an assessment survey, this big data maturity model assesses an organization’s readiness to execute big data initiatives.  Furthermore, the model aims to identify the steps and appropriate technologies that will lead an organization towards big data maturity.

=== Comparative Models ===

Comparative big data maturity models aim to benchmark an organization in relation to its industry peers and normally consist of a survey containing quantitative and qualitative information.

=== CSC big data maturity tool ===

==== '''CSC Big Data Maturity Tool'''<ref>{{Cite web|url=http://csc.bigdatamaturity.com/|title=CSC Big Data Maturity Tool: Business Value, Drivers, and Challenges|last=Inc.|first=Creative services by Cyclone Interactive Multimedia Group, Inc. (www.cycloneinteractive.com) Site designed and hosted by Cyclone Interactive Multimedia Group,|website=csc.bigdatamaturity.com|language=en|access-date=2017-05-21}}</ref> ====

The CSC Bigbig Datadata maturity tool acts as a comparative tool to benchmark an organization’sorganization's big data maturity. A survey is undertaken and the results are then compared to other organizations within a specific industry and within the wider market.<ref>{{Cite web|url=http://csc.bigdatamaturity.com/|title=CSC Big Data Maturity Tool: Business Value, Drivers, and Challenges|last=Inc.|first=Creative services by Cyclone Interactive Multimedia Group, Inc. (www.cycloneinteractive.com) Site designed and hosted by Cyclone Interactive Multimedia Group|website=csc.bigdatamaturity.com|language=en|access-date=2017-05-21}}</ref>

=== TDWI big data maturity model ===

'''TDWI Big Data Maturity Model''' <ref name=":0" />

The TDWI big data maturity model is a model in the current big data maturity area and therefore consists of a significant body of knowledge.<ref name=":0" />

'''Maturity stages'''

The TDWI Big Data Maturity Model is a salient model in the current big data maturity area and therefore consists of a significant body of knowledge. TDWI describes big data maturity as the evolution of an organization to integrate, manage, and leverage all relevant internal and external data sources. The model involves building an ecosystem that includes technologies, data management, analytics, governance, and organizational components. This BDMM provides a methodology to measure and monitor the state of the big data program, identifies the current stage of maturity of an organization, speaks to the effort needed to complete their current stage, as well as the steps required to move to the next stage of maturity. It serves as a kind of odometer to measure and manage the speed of progress and adoption within a company for a big data program.

The assessment criteria against which organizations are measured to assess their maturity level is summarized as follows:

1. '''Organization: '''To what extent does the organizational strategy, culture, leadership, and funding support a successful big data analytics program? What value does the company place on analytics? Additionally, is the company organized for success with data, big data, and analytics? 

2. '''Infrastructure: '''How advanced and coherent is the architecture in support of a big data initiative? To what extent does the infrastructure support all parts of the company and potential users? How effective is the data management approach? What technologies are in place to support this kind of initiative and how are they integrated into the existing environment?

3.  '''Data management: '''How extensive are the variety, volume, and velocity of data used for analytics and how does the company manage its big data in support of analytics? This includes data quality and processing as well as data integration and storage issues. 

4. '''Analytics: '''How advanced is the company in its use of analytics? This includes the kinds of analytics utilized and how the analytics are delivered in the organization. It also includes the skills required to make analytics happen. 

5. '''Governance: '''How coherent is the company’s data governance strategy in support of its big data analytics program?

The different stages of maturity in the TDWI BDMM can be summarized as follows:

'''''Stage 1: Nascent'''''

The nascent stage as a pre–big data environment. During this stage:

* The organization has a low awareness of big data or its value

* There is little to no executive support for the effort and only some people in the organization are interested in potential value of big data

·      The organization has a low awareness of big data or its value;

* The organization understand the benefits of analytics and may have a data warehouse

* An organization's governance strategy is typically more IT-centric rather than being integrative business-and-IT centric

·      There is little to no executive support for the effort and only some people in the organization are interested in potential value of big data;

'''Stage 2: Pre-adoption'''

·             The Organization understand the benefits of analytics and may have a data warehouse but have not started to explore advanced analytics or begun its big data journey;

·             An organization’s governance strategy is typically more IT-centric rather than being integrative business-and-IT centric. 

'''''Stage 2: Pre-Adoption'''''

During the pre-adoption stage:

* The organization start to investigate big data analytics

'''Stage 3: Early adoption'''

'''The "chasm"'''

There is then generally a series of hurdles it needs to overcome. These hurdles include:

* Obtaining the right skill set to support the capability, including Hadoop and advanced analytical skills

* Political issues, i.e. big data projects are conducted in areas within the organization and trying to expand the effort or enforcing more stringent standards and governance lead to issues regarding ownership and control

'''Stage 4: Corporate adoption'''

The corporate adoption stage is characterized by the involvement of end-users, an organization gains further insight and the way of conducting business is transformed. During this stage:

·             The organization start to investigate big data analytics.

* End-users might have started operationalizing big data analytics or changing their decision-making processes

* Most organizations would already have repeatedly addressed certain gaps in their infrastructure, data management, governance and analytics

'''Stage 5: Mature / visionary'''

Only a few organizations can be considered as visionary in terms of big data and big data analytics. During this stage an organization:

·             The organization may have invested in some new technology, such as Hadoop, in support of big data.

* is able to execute big data programs as a well-oiled machine with highly mature infrastructure

* has a well-established big data program and big data governance strategies

* executes its big data program as a budgeted and planned initiative from an organization-wide perspective

* whose employees share a level of excitement and energy around big data and big data analytics

'''Research findings'''

TDWI<ref name=":0" /> did an assessment on 600 organizations and found that the majority of organizations are either in the pre-adoption (50%) or early adoption (36%) stages. Additionally, only 8% of the sample have managed to move past the chasm towards corporate adoption or being mature/visionary.

·      The organization knows that it will be implementing big data analytics in the near future, although the effort is usually limited and departmental in scope.

== Prescriptive ==

'''''Stage 3: Early Adoption'''''

The majority of prescriptive BDMMs follow a similar modus operandi in that the current situation is first assessed followed by phases plotting the path towards increased big data maturity. Examples are:

During=== thisInfo-tech stagebig ofdata maturity, anassessment organization:tool ===

This maturity model is prescriptive in the sense that the model consists of four distinct phases that each plot a path towards big data maturity. Phases are:

* Phase 1, undergo big data education

* Phase 2, assess big data readiness

* Phase 3, pinpoint a killer big data use case

* Phase 4, structure a big data proof-of-concept project

<ref>{{Cite web|url=https://www.infotech.com/research/ss/leverage-big-data-by-starting-small/it-big-data-maturity-assessment-tool|title=Big Data Maturity Assessment Tool|website=www.infotech.com|language=en|access-date=2017-05-21}}</ref>

=== Radcliffe big data maturity model ===

·      Typically conducts one or two proofs of concept (POCs) which become more established and production ready.

The Radcliffe big data maturity model, as other models, also consists of distinct maturity levels ranging from:

* 0 – "In the dark"

* 1 – "Catching up"

* 2 – "First pilot"

* 3 – "Tactical value"

* 4 – "Strategic leverage"

* 5 – "Optimize and extend"

<ref name=":3" />

=== Booz & Company's model ===

·      Tends to spend a long time in this stage, often because it is hard to cross the chasm that leads to corporate wide adoption of big data and big data analytics.

This BDMM provides a framework that not only enables organizations to view the extent of their current maturity, but also to identify goals and opportunities for growth in big data maturity. The model consists of four stages namely,

* Stage 1: Performance management

* Stage 2: Functional area excellence

* Stage 3: Value proposition enhancement

* Stage 4: Business model transformation

<ref name=":2" />

=== Van Veenstra's model ===

'''''Chasm'''''

The prescriptive model proposed by Van Veenstra aims to firstly explore the existing big data environment of the organization followed by exploitation opportunities and a growth path towards big data maturity. The model makes use of four phases namely:

* Efficiency

* Effectiveness

* New solutions

* Transformation

<ref>{{Cite journal|last=van Veenstra|first=Anne Fleur|title=Big Data in Small Steps: Assessing the value of data|url=http://www.idnext.eu/files/TNO-whitepaper--Big-data-in-small-steps.pdf|journal=White Paper}}</ref>

== Critical evaluation ==

As an organization tries to move from early adoption to corporate adoption, there is generally a series of hurdles it needs to overcome. These hurdles include:

Current BDMMs have been evaluated under the following criteria:<ref name=":1" />

* Completeness of the model structure (completeness, consistency)

* The quality of model development and evaluation (trustworthiness, stability)

* Ease of application (ease of use, comprehensibility)

* Big data value creation (actuality, relevancy, performance)

The TDWI and CSC have the strongest overall performance with steady scores in each of the criteria groups. The overall results communicate that the top performer models are extensive, balanced, well-documented, easy to use, and they address a good number of big data capabilities that are utilized in business value creation. The models of Booz & Company and Knowledgent are close seconds and these mid-performers address big data value creation in a commendable manner, but fall short when examining the completeness of the models and the ease of application. Knowledgent suffers from poor quality of development, having barely documented any of its development processes. The rest of the models, i.e. Infotech, Radcliffe, van Veenstra and IBM, have been categorized as low performers. Whilst their content is well aligned with business value creation through big data capabilities, they all lack quality of development, ease of application and extensiveness. Lowest scores were awarded to IBM and Van Veenstra, since both are providing low level guidance for the respective maturity model's practical use, and they completely lack in documentation, ultimately resulting in poor quality of development and evaluation.<ref name=":1" />

== See also ==

·       Obtaining the right skill set to support the capability, including Hadoop and advanced analytical skills;

* [[Capability Maturity Model]]

== References ==

·       Political issues, i.e. big data projects are conducted in areas within the organization and trying to expand the effort or enforcing more stringent standards and governance lead to issues regarding ownership and control.

{{Reflist}}

[[Category:Big data]]

·       In the case of a smaller organization that wants to be nimble, there comes a point after several projects or as they start to grow when they realize they might need to put some structure in place and deal with issues such as data security or management.

'''''Stage 4: Corporate Adoption'''''

The corporate adoption stage is characterized by the involvement of end-users,  an organization gains further insight and the way of conducting business is transformed.  During this stage:

·         End-users might have started operationalizing big data analytics or changing their decision making processes;

·         Most organizations would already have repeatedly addressed certain gaps in their infrastructure, data management, governance and analytics.

'''''Stage 5: Mature / Visionary.'''''

Only a few organizations can be considered as visionary in terms of big data and big data analytics. During this stage an organization:

·       Is able to execute big data programs as a well-oiled machine with highly mature infrastructure

·         Has a well-established big data program and big data governance strategies.

·         Executes its big data program as a budgeted and planned initiative from an organization-wide perspective.

·         shares a level of excitement and energy around big data and big data analytics.

'''Practical Findings'''

TDWI did an assessment on 600 organizations and found that the majority of organizations are either in the Pre-Adoption (50%) or Early Adoption (36%) stages. Additionally, only 8% of the sample have managed to move past the chasm towards corporate adoption or being mature/visionary.

=== Prescriptive Models ===

The majority of prescriptive BDMMs follow a similar modus operandi in that the current situation is first assessed followed by phases plotting the path towards increased big data maturity. Examples are as follows:

==== '''Info-Tech Big Data Maturity Assessment Tool''' <ref>{{Cite web|url=https://www.infotech.com/research/ss/leverage-big-data-by-starting-small/it-big-data-maturity-assessment-tool|title=Big Data Maturity Assessment Tool|website=www.infotech.com|language=en|access-date=2017-05-21}}</ref> ====

This maturity model is prescriptive in the sense that the model consists of four distinct phases that each plot a path towards Big Data Maturity. Phases are: Phase 1, Undergo Big Data Education; Phase 2, Assess Big Data Readiness; Phase 3, Pinpoint a Killer Big Data Use Case; and Phase 4, Structure a Big Data Proof-of-Concept Project.

==== '''Radcliffe Big Data Maturity Model'''<ref name=":3" /> ====

The Radcliffe Big Data Maturity Model, as other models, also consists of distinct maturity levels ranging from; 0 – In the Dark, 1 – Catching up, 2 – First Pilot, 3 - Tactical Value, 4 – Strategic Leverage and 5 – Optimize & Extend.

==== '''Booz & Company'''<ref name=":2" /> ====

This BDMM provides a framework that not only enables organizations to view the extent of their current maturity, but also to identify goals and opportunities for growth in big data maturity. The model consists of four stages namely, Stage 1: Performance Management, Stage 2: Functional Area Excellence, Stage 3: Value Proposition enhancement and finally Stage 4: Business model transformation.

'''Van Veenstra''' <ref>{{Cite journal|last=van Veenstra|first=Anne Fleur|date=|title=Big Data in Small Steps: Assessing the value of data|url=http://www.idnext.eu/files/TNO-whitepaper--Big-data-in-small-steps.pdf|journal=White paper|volume=|pages=|via=}}</ref>

The prescriptive model proposed by Van Veenstra aims to firstly explore the existing big data environment of the organization followed by exploitation opportunities and a growth path towards big data maturity. The model makes use of four phases namely: Efficiency, Effectiveness, New Solutions, and Transformation.

== Critical Evaluation ==

Current BDMMs have been evaluated under the following criteria <ref name=":1" />:

·         Completeness of the model structure (completeness, consistency),

·         The quality of model development and evaluation (trustworthiness, stability),

·         Ease of application (ease of use, comprehensibility), and

·         Big Data value creation (actuality, relevancy, performance).

The TDWI and CSC have the strongest overall performance with steady scores in each of the criteria groups. The overall results communicate that the top performer models are extensive, balanced, well-documented, easy to use, and they address a good number of big data capabilities that are utilized in business value creation. The models of Booz & Company and Knowledgent are close seconds and these mid-performers address big data value creation in a commendable manner, but fall short when examining the completeness of the models and the ease of application. Furthermore, Braun (2015) found that Knowledgent suffers from poor quality of development, having barely documented any of its development processes. The rest of the models, i.e. Infotech, Radcliffe, van Veenstra and IBM, have been categorized as low performers. Whilst their content is well aligned with business value creation through big data capabilities, they all lack quality of development, ease of application and extensiveness. Lowest scores were awarded to IBM and Van Veenstra, since both are providing low level guidance for the respective maturity model’s practical use, and they completely lack in documentation, ultimately resulting in poor quality of development and evaluation<ref name=":1" />.

== Also See ==

[[Big data|Big Data]]

[[Analytics]]

[[Capability Maturity Model]]

[[Digital Mastery]]

== References ==