Meta-learning (computer science): Difference between revisions

Revision as of 01:45, 14 January 2022

Revision as of 04:40, 30 January 2022

Line 13:

}}</ref> This means that it will only learn well if the bias matches the learning problem. A learning algorithm may perform very well in one ___domain, but not on the next. This poses strong restrictions on the use of [[machine learning]] or [[data mining]] techniques, since the relationship between the learning problem (often some kind of [[database]]) and the effectiveness of different learning algorithms is not yet understood.

By using different kinds of metadata, like properties of the learning problem, algorithm properties (like performance measures), or patterns previously derived from the data, it is possible to learn, select, alter or combine different learning algorithms to effectively solve a given learning problem. Critiques of meta learning approaches bear a strong resemblance to the critique of [[metaheuristic]], a possibly related problem. A good analogy to meta-learning, and the inspiration for [[Jürgen Schmidhuber]]'s early work (1987)<ref name="sch1987" /> and [[Yoshua Bengio]] et al.'s work (1991),<ref>{{cite conference|lastlast1=Bengio|firstfirst1=Yoshua|last2=Bengio|first2=Samy|last3=Cloutier|first3=Jocelyn|conference=IJCNN'91|url=http://bengio.abracadoudou.com/publications/pdf/bengio_1991_ijcnn.pdf|date=1991|title=Learning to learn a synaptic rule}}</ref> considers that genetic evolution learns the learning procedure encoded in genes and executed in each individual's brain. In an open-ended hierarchical meta learning system<ref name="sch1987" /> using [[genetic programming]], better evolutionary methods can be learned by meta evolution, which itself can be improved by meta meta evolution, etc.<ref name="sch1987" />

== Definition ==

A proposed definition<ref>{{Cite journal|lastlast1=Lemke|firstfirst1=Christiane|last2=Budka|first2=Marcin|last3=Gabrys|first3=Bogdan|date=2013-07-20|title=Metalearning: a survey of trends and technologies|journal=Artificial Intelligence Review|language=en|volume=44|issue=1|pages=117–130|doi=10.1007/s10462-013-9406-y|issn=0269-2821|pmc=4459543|pmid=26069389}}</ref> for a meta learning system combines three requirements:

* The system must include a learning subsystem.

* Experience is gained by exploiting meta knowledge extracted

Line 75:

* In the 1990s, Meta [[Reinforcement Learning]] or Meta RL was achieved in Schmidhuber's research group through self-modifying policies written in a universal programming language that contains special instructions for changing the policy itself. There is a single lifelong trial. The goal of the RL agent is to maximize reward. It learns to accelerate reward intake by continually improving its own learning algorithm which is part of the "self-referential" policy.<ref name="sch1994">{{cite journal | last1 = Schmidhuber | first1 = Jürgen | year = 1994| title = On learning how to learn learning strategies | journal = Technical Report FKI-198-94, Tech. Univ. Munich}}</ref><ref name="sch1997">{{cite journal | last1 = Schmidhuber | first1 = Jürgen | last2 = Zhao | first2 = J. | last3 = Wiering | first3 = M. | year = 1997| title = Shifting inductive bias with success-story algorithm, adaptive Levin search, and incremental self-improvement | journal = Machine Learning | volume = 28 | pages = 105–130 | doi=10.1023/a:1007383707642| doi-access = free }}</ref>

* An extreme type of Meta [[Reinforcement Learning]] is embodied by the [[Gödel machine]], a theoretical construct which can inspect and modify any part of its own software which also contains a general [[Automated theorem proving|theorem prover]]. It can achieve [[recursive self-improvement]] in a provably optimal way.<ref name="goedelmachine">{{cite journal | last1 = Schmidhuber | first1 = Jürgen | year = 2006| title = Gödel machines: Fully Self-Referential Optimal Universal Self-Improvers | url=https://archive.org/details/arxiv-cs0309048| journal = In B. Goertzel & C. Pennachin, Eds.: Artificial General Intelligence | pages = 199–226}}</ref><ref name="scholarpedia" />

* ''Model-Agnostic Meta-Learning'' (MAML) was introduced in 2017 by Chelsea Finn et al.<ref name="maml" /> Given a sequence of tasks, the parameters of a given model are trained such that few iterations of gradient descent with few training data from a new task will lead to good generalization performance on that task. MAML "trains the model to be easy to fine-tune."<ref name="maml" /> MAML was successfully applied to few-shot image classification benchmarks and to policy-gradient-based reinforcement learning.<ref name="maml">{{cite arxivarXiv | last1 = Finn | first1 = Chelsea | last2 = Abbeel | first2 = Pieter | last3 = Levine | first3 = Sergey |year = 2017| title = Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks | eprint=1703.03400|class=cs.LG }}</ref>

* ''Discovering [[meta-knowledge]]'' works by inducing knowledge (e.g. rules) that expresses how each learning method will perform on different learning problems. The metadata is formed by characteristics of the data (general, statistical, information-theoretic,... ) in the learning problem, and characteristics of the learning algorithm (type, parameter settings, performance measures,...). Another learning algorithm then learns how the data characteristics relate to the algorithm characteristics. Given a new learning problem, the data characteristics are measured, and the performance of different learning algorithms are predicted. Hence, one can predict the algorithms best suited for the new problem.

* ''Stacked generalisation'' works by combining multiple (different) learning algorithms. The metadata is formed by the predictions of those different algorithms. Another learning algorithm learns from this metadata to predict which combinations of algorithms give generally good results. Given a new learning problem, the predictions of the selected set of algorithms are combined (e.g. by (weighted) voting) to provide the final prediction. Since each algorithm is deemed to work on a subset of problems, a combination is hoped to be more flexible and able to make good predictions.

Line 82:

* ''[[Inductive transfer]]'' studies how the learning process can be improved over time. Metadata consists of knowledge about previous learning episodes and is used to efficiently develop an effective hypothesis for a new task. A related approach is called [[learning to learn]], in which the goal is to use acquired knowledge from one ___domain to help learning in other domains.

* Other approaches using metadata to improve automatic learning are [[learning classifier system]]s, [[case-based reasoning]] and [[constraint satisfaction]].

* Some initial, theoretical work has been initiated to use ''[[Applied Behavioral Analysis]]'' as a foundation for agent-mediated meta-learning about the performances of human learners, and adjust the instructional course of an artificial agent.<ref name="Begoli, PRS-ABA, ABA Ontology">{{cite bookjournal|last1=Begoli|first1=Edmon|title=Procedural-Reasoning Architecture for Applied Behavior Analysis-based Instructions|journal=Doctoral Dissertations|date=May 2014|publisher=University of Tennessee, Knoxville|___location=Knoxville, Tennessee, USA|pages=44–79|url=http://trace.tennessee.edu/utk_graddiss/2749|access-date=14 October 2017}}</ref>

* [[AutoML]] such as Google Brain's "AI building AI" project, which according to Google briefly exceeded existing [[ImageNet]] benchmarks in 2017.<ref>{{cite news|title=Robots Are Now 'Creating New Robots,' Tech Reporter Says|url=https://www.npr.org/2018/03/15/593863645/robots-are-now-creating-new-robots-tech-reporter-says|access-date=29 March 2018|work=NPR.org|date=2018|language=en}}</ref><ref>{{cite news|title=AutoML for large scale image classification and object detection|url=https://research.googleblog.com/2017/11/automl-for-large-scale-image.html|access-date=29 March 2018|work=Google Research Blog|date=November 2017}}</ref>