Rule induction

Rule induction is an area of machine learning in which formal rules are extracted from a set of observations. The rules extracted may represent a full scientific model of the data, or merely represent local patterns in the data.

Example of a decision tree

Data mining in general and rule induction in detail are trying to create algorithms without human programming but with analyzing existing data structures.^[1]: 415- In the easiest case, a rule is expressed with “if-then statements” and was created with the ID3 algorithm for decision tree learning.^{[2]: 7 [1]: 348} Rule learning algorithm are taking training data as input and creating rules by partitioning the table with cluster analysis.^[2]: 7 A possible alternative over the ID3 algorithm is genetic programming which evolves a program until it fits to the data.^[3]: 2

Creating different algorithm and testing them with input data can be realized in the WEKA software.^[3]: 125 Additional tools are machine learning libraries for Python, like scikit-learn.

Paradigms

Some major rule induction paradigms are:

• Association rule learning algorithms (e.g., Agrawal)
• Decision rule algorithms (e.g., Quinlan 1987)
• Hypothesis testing algorithms (e.g., RULEX)
• Horn clause induction
• Version spaces
• Rough set rules
• Inductive Logic Programming
• Boolean decomposition (Feldman)

Algorithms

Some rule induction algorithms are:

• Charade^[4]
• Rulex
• Progol
• CN2

References

1. Evangelos Triantaphyllou; Giovanni Felici (10 September 2006). Data Mining and Knowledge Discovery Approaches Based on Rule Induction Techniques. Springer Science & Business Media. ISBN 978-0-387-34296-2.
2. Alex A. Freitas (11 November 2013). Data Mining and Knowledge Discovery with Evolutionary Algorithms. Springer Science & Business Media. ISBN 978-3-662-04923-5.
3. Gisele L. Pappa; Alex Freitas (27 October 2009). Automating the Design of Data Mining Algorithms: An Evolutionary Computation Approach. Springer Science & Business Media. ISBN 978-3-642-02541-9.
4. Sahami, Mehran. "Learning classification rules using lattices." Machine learning: ECML-95 (1995): 343-346.

• Quinlan, J. R. (1987). "Generating production rules from decision trees" (PDF). In McDermott, John (ed.). Proceedings of the Tenth International Joint Conference on Artificial Intelligence (IJCAI-87). Milan, Italy. pp. 304–307.