Developmental robotics: Difference between revisions

Revision as of 00:41, 16 October 2015

Latest revision as of 13:54, 19 August 2025

(39 intermediate revisions by 26 users not shown)

Line 1:

{{Use mdy dates|date=October 2014}}

'''Developmental Roboticsrobotics''' ('''DevRob'''), sometimes called '''[[epigenetics|epigenetic]] robotics''', is a scientific field which aims at studying the developmental mechanisms, architectures and constraints that allow lifelong and open-ended learning of new skills and new knowledge in embodied [[machine]]s. As in human children, [[learning]] is expected to be cumulative and of progressively increasing complexity, and to result from self-exploration of the world in combination with [[social relation|social interaction]]. The typical methodological approach consists in starting from theories of human and animal development elaborated in fields such as [[developmental psychology]], [[neuroscience]], [[developmental biology|developmental]] and [[evolutionary biology]], and [[linguistics]], then to formalize and implement them in robots, sometimes exploring extensions or variants of them. The experimentation of those models in robots allows researchers to confront them with reality, and as a consequence, developmental robotics also provides feedback and novel hypotheses on theories of human and animal development.

Developmental robotics is related to, but differs from, [[evolutionary robotics]] (ER). ER uses populations of robots that evolve over time, whereas DevRob is interested in how the organization of a single robot's control system develops through experience, over time.

DevRob is also related to work done in the domains of [[robotics]] and [[artificial life]].

Line 9:

Can a robot learn like a child? Can it learn a variety of new skills and new knowledge unspecified at design time and in a partially unknown and changing environment? How can it discover its body and its relationships with the physical and social environment? How can its cognitive capacities continuously develop without the intervention of an engineer once it is "out of the factory"? What can it learn through natural social interactions with humans? These are the questions at the center of developmental robotics. Alan Turing, as well as a number of other pioneers of cybernetics, already formulated those questions and the general approach in 1950,<ref name="Turing50">{{cite journal

| last = Turing | first = A.M. | date = 1950 | url = http://www.csee.umbc.edu/courses/471/papers/turing.pdf | title = Computing machinery and intelligence | journal = Mind | publisher = LIX | issue = 236 | pages = 433–460 | volume=LIX| doi = 10.1093/mind/LIX.236.433 }}</ref>

but it is only since the end of the 20th century that they began to be investigated systematically.<ref name="Weng01Lungarella03">{{cite journal

| last1 = WengLungarella | first1 = JM. | last2 = McClellandMetta | last3 = Pentland | first3first2 = AG. | last4last3 = SpornsPfeifer | first4first3 = OR. | last5first4 = Stockman | first5 = IG. | last6last4 = SurSandini | first6date = M.2003 | first7title = E. |Developmental last7robotics: =a Thelensurvey | dateciteseerx = 2001 | url = http://www10.cse1.msu1.edu/dl/SciencePaper83.pdf7615 | titlejournal = AutonomousConnection mental development by robots and animalsScience | journalvolume = Science15 | volumeissue = 2914 | pages = 599–600151–190 | doi=10.11261080/science09540090310001655110| bibcode = 2003ConSc.291.550415.599.151L | s2cid = 1452734 }}</ref><ref name="Lungarella03Asada09">{{cite journal

| last1 = LungarellaAsada | first1 = M. | last2 = MettaHosoda | first2 = GK. | last3 = PfeiferKuniyoshi | first3 = RY. | last4 = Ishiguro | first4 = GH. | last4last5 = SandiniInui | datefirst5 = 2003T. | titlelast6 = Yoshikawa Developmental| robotics:first6 a= surveyY. | idlast7 = {{citeseerxOgino |10 first7 = M.1 | first8 = C.1.83.7615}} | last8 = Yoshida | date = 2009 | title = Cognitive developmental robotics: a survey | journal = ConnectionIEEE Transactions on Autonomous Mental ScienceDevelopment | volume = 151 | issue = 1 | pages = 151–19012–34 | doi=10.10801109/09540090310001655110tamd.2009.2021702| bibcode = 2009ITAMD...1...12A | s2cid = 10168773 }}</ref><ref name="Asada09Oudeyer10">{{cite journal

| last1authorlink1 = AsadaPierre-Yves | first1 = M.Oudeyer | last2last1 = HosodaOudeyer | first2 = K. | last3 = Kuniyoshi | first3first1 = P-Y. | last4date = Ishiguro2010 | first4url = Hhttp://www. | last5 = Inui | first5 = Tpyoudeyer. | last6 = Yoshikawa | first6 = Ycom/IEEETAMDOudeyer10.pdf | last7title = OginoOn |the first7impact =of M.robotics |in first8behavioral =and C.cognitive | last8 = Yoshida | date = 2009 | url = httpsciences://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=4895715&url=http%3A%2F%2Fieeexplore.ieee.org%2Fiel5%2F4563672%2F5038478%2F04895715.pdf%3Farnumber%3D4895715 |from titleinsect =navigation Cognitiveto developmental robotics:human acognitive surveydevelopment | journal = IEEE Transactions on Autonomous Mental Development | volume = 12 | issue = 1 | pages = 12–342–16 | doi=10.1109/tamd.2009.20217022039057| bibcode = 2010ITAMD...2....2O | s2cid = 6362217 }}</ref><ref name="Oudeyer10">{{cite journal

| authorlink1 = Pierre-Yves Oudeyer | last1 = Oudeyer | first1 = P-Y. | date = 2010 | url = http://www.pyoudeyer.com/IEEETAMDOudeyer10.pdf | title = On the impact of robotics in behavioral and cognitive sciences: from insect navigation to human cognitive development | journal = IEEE Transactions on Autonomous Mental Development | volume = 2 | issue = 1 | pages = 2–16 | doi=10.1109/tamd.2009.2039057}}</ref>

Because the concept of adaptive intelligent machinemachines is central to developmental robotics, isit has relationships with fields such as artificial intelligence, machine learning, [[cognitive robotics]] or [[computational neuroscience]]. Yet, while it may reuse some of the techniques elaborated in these fields, it differs from them from many perspectives. It differs from classical artificial intelligence because it does not assume the capability of advanced symbolic reasoning and focuses on embodied and situated sensorimotor and social skills rather than on abstract symbolic problems. It differs from traditional machine learning because it targets task- independent self-determined learning rather than task-specific inference over "spoon fed human-edited sensori data" (Weng et al., 2001). It differs from cognitive robotics because it focuses on the processes that allow the formation of cognitive capabilities rather than these capabilities themselves. It differs from computational neuroscience because it focuses on functional modeling of integrated architectures of development and learning. More generally, developmental robotics is uniquely characterized by the following three features:

# It targets task-independent architectures and learning mechanisms, i.e. the machine/robot has to be able to learn new tasks that are unknown by the engineer;

# It emphasizes open-ended development and lifelong learning, i.e. the capacity of an organism to acquire continuously novel skills. This should not be understood as a capacity for learning "anything" or even “everything”, but just that the set of skills that is acquired can be infinitely extended at least in some (not all) directions;

# The complexity of acquired knowledge and skills shall increase (and the increase be controlled) progressively.

Developmental robotics emerged at the crossroads of several research communities including embodied artificial intelligence, enactive and dynamical systems cognitive science, connectionism. Starting from the essential idea that learning and development happen as the self-organized result of the dynamical interactions among brains, bodies and their physical and social environment, and trying to understand how this self- organization can be harnessed to provide task-independent lifelong learning of skills of increasing complexity, developmental robotics strongly interacts with fields such as developmental psychology, developmental and cognitive neuroscience, developmental biology (embryology), evolutionary biology, and [[cognitive linguistics]]. As many of the theories coming from these sciences are verbal and/or descriptive, this implies a crucial formalization and computational modeling activity in developmental robotics. These computational models are then not only used as ways to explore how to build more versatile and adaptive machines, but also as a way to evaluate their coherence and possibly explore alternative explanations for understanding biological development.<ref name="Oudeyer10" />

== Research directions ==

=== Skill domains ===

Due to the general approach and methodology, developmental robotics projects typically focus on having robots develop the same types of skills as human infants. A first category that is importantlyimportant being investigated is the acquisition of sensorimotor skills. These include the discovery of one's own body, including its structure and dynamics such as hand–eyehand-eye coordination, locomotion, and interaction with objects as well as tool use, with a particular focus on the discovery and learning of affordances. A second category of skills targeted by developmental robots are social and linguistic skills: the acquisition of simple social behavioural games such as turn-taking, coordinated interaction, lexicons, syntax and grammar, and the grounding of these linguistic skills into sensorimotor skills (sometimes referred as symbol grounding). In parallel, the acquisition of associated cognitive skills are being investigated such as the emergence of the self/non-self distinction, the development of attentional capabilities, of categorization systems and higher-level representations of affordances or social constructs, of the emergence of values, empathy, or theories of mind.

=== Mechanisms and constraints ===

Line 32 ⟶ 31:

# Motivational systems, generating internal reward signals that drive exploration and learning, which can be of two main types:

#* extrinsic motivations push robots/organisms to maintain basic specific internal properties such as food and water level, physical integrity, or light (e.g. in phototropic systems);

#* [[intrinsic motivation (artificial intelligence) | intrinsic motivations]] push robot to search for novelty, challenge, compression or learning progress per se, thus generating what is sometimes called curiosity-driven learning and exploration, or alternatively active learning and exploration;

#Social guidance: as humans learn a lot by interacting with their peers, developmental robotics investigates mechanisms whichthat can allow robots to participate to human-like social interaction. By perceiving and interpreting social cues, this may allow robots both to learn from humans (through diverse means such as imitation, emulation, stimulus enhancement, demonstration, etc. ...) and to trigger natural human pedagogy. Thus, social acceptance of developmental robots is also investigated;

# Statistical inference biases and cumulative knowledge/skill reuse: biases characterizing both representations/encodings and inference mechanisms can typically allow considerable improvement of the efficiency of learning and are thus studied. Related to this, mechanisms allowing to infer new knowledge and acquire new skills by reusing previously learnt structures is also an essential field of study;

#The properties of embodiment, including geometry, materials, or innate motor primitives/synergies often encoded as dynamical systems, can considerably simplify the acquisition of sensorimotor or social skills, and is sometimes referred as morphological computation. The interaction of these constraints with other constraints is an important axis of investigation;

#Maturational constraints: In human infants, both the body and the neural system grow progressively, rather than being full-fledged already at birth. This implies, for example, that new degressdegrees of freedom, as well as increases of the volume and resolution of available sensorimotor signals, may appear as learning and development unfold. Transposing these mechanisms in developmental robots, and understanding how it may hinder or on the contrary ease the acquisition of novel complex skills is a central question in developmental robotics.

=== From bio-mimetic development to functional inspiration. ===

While most developmental robotics projects strongly interact closely with theories of animal and human development, the degrees of similarities and inspiration between identified biological mechanisms and their counterpart in robots, as well as the abstraction levels of modeling, may vary a lot. While some projects aim at modeling precisely both the function and biological implementation (neural or morphological models), such as in neurorobotics[[Neurorobotics]], some other projects only focus on functional modeling of the mechanisms and constraints described above, and might for example reuse in their architectures techniques coming from applied mathematics or engineering fields.

== Open questions ==

As developmental robotics is a relatively novelnew research field and at the same time very ambitious, many fundamental open challenges remain to be solved.

First of all, existing techniques are far from allowing real-world high-dimensional robots to learn an open- ended repertoire of increasingly complex skills over a life-time period. High-dimensional continuous sensorimotor spaces areconstitute a majorsignificant obstacle to be solved. Lifelong [[cumulative learning]] is another one. Actually, no experiments lasting more than a few days have been set up so far, which contrasts severely with the time period needed by human infants to learn basic sensorimotor skills while equipped with brains and morphologies which are tremendously more powerful than existing computational mechanisms.

Among the strategies to explore in order to progress towards this target, the interaction between the mechanisms and constraints described in the previous section shall be investigated more systematically. Indeed, they have so far mainly been studied in isolation. For example, the interaction of intrinsically motivated learning and socially guided learning, possibly constrained by maturation, is an essential issue to be investigated.

Another important challenge is to allow robots to perceive, interpret and leverage the diversity of [[Multimodal_interaction|multimodal]] social cues provided by non-engineer humans during human-robot interaction. These capacities are so far, mostly too limited to allow efficient general -purpose teaching from humans.

A fundamental scientific issue to be understood and resolved, which applied equally to human development, is how compositionality, functional hierarchies, primitives, and modularity, at all levels of sensorimotor and social structures, can be formed and leveraged during development. This is deeply linked with the problem of the emergence of symbols, sometimes referred to as the "[[symbol grounding problem]]" when it comes to language acquisition. Actually, the very existence and need for symbols in the brain isare actively questioned, and alternative concepts, still allowing for compositionality and functional hierarchies are being investigated.

During biological epigenesis, morphology is not fixed but rather develops in constant interaction with the development of sensorimotor and social skills. The development of morphology poses obvious practical problems with robots, but it may be a crucial mechanism that should be further explored, at least in simulation, such as in morphogenetic robotics.

Another open problem is the understanding of the relation between the key phenomena investigated by developmental robotics (e.g., hierarchical and modular sensorimotor systems, intrinsic/extrinsic/social motivations, and open-ended learning) and the underlying brain mechanisms.

Similarly, in biology, developmental mechanisms (operating at the ontogenetic time scale) strongly interact with evolutionary mechanisms (operating at the phylogenetic time scale) as shown in the flourishing "evo- devo" scientific literature.<ref name="Muller07">{{cite journal

| last1 = Müller | first1 = G. B. | date = 2007 | url = http://www.nature.com/nrg/journal/v8/n12/full/nrg2219.html | title = Evo-devo: extending the evolutionary synthesis | journal = Nature Reviews Genetics | volume = 8 | pages = 943–949 | doi=10.1038/nrg2219}}</ref>

Similarly, in biology, developmental mechanisms (operating at the ontogenetic time scale) strongly interact closely with evolutionary mechanisms (operating at the phylogenetic time scale) as shown in the flourishing "[[evo- devo]]" scientific literature.<ref name="Muller07">{{cite journal

However, the interaction of those mechanisms in artificial organisms, developmental robots in particular, is still vastly understudied. The interaction of evolutionary mechanisms, unfolding morphologies and developing sensorimotor and social skills will thus be a highly stimulating topic for the future of developmental robotics.

| last1 = Müller | first1 = G. B. | date = 2007 | url = http://www.nature.com/nrg/journal/v8/n12/full/nrg2219.html | title = Evo-devo: extending the evolutionary synthesis | journal = Nature Reviews Genetics | volume = 8 | issue = 12 | pages = 943–949 | doi=10.1038/nrg2219 | pmid=17984972| s2cid = 19264907 }}</ref>

However, the interaction of those mechanisms in artificial organisms, developmental robots, in particular, is still vastly understudied. The interaction of evolutionary mechanisms, unfolding morphologies and developing sensorimotor and social skills will thus be a highly stimulating topic for the future of developmental robotics.

==Main journals==

* IEEE Transactions on Cognitive and Developmental Systems (previously known as IEEE Transactions on Autonomous Mental Development): httphttps://wwwcis.ieee-cis.org/pubs/tamdpublications/t-cognitive-and-developmental-systems

* AMD Newsletter: http://www.cse.msu.edu/amdtc/amdnl/

==Main conferences==

* International Conference on Development and Learning: http://www.cogsci.ucsd.edu/~triesch/icdl/

* Epigenetic Robotics: httphttps://www.epigenetic-roboticslucs.orglu.se/epirob/

* ICDL-EpiRob: http://www.icdl-epirob.org/ (the two above joined since 2011)

* Developmental Robotics: http://cs.brynmawr.edu/DevRob05/

The NSF/DARPA funded [http://www.cse.msu.edu/dl/ Workshop on Development and Learning] was held April 5–7, 2000 at Michigan State University. It was the first international meeting devoted to computational understanding of mental development by robots and animals. The term "by" was used since the agents are active during development.

==See also==

* [[Evolutionary developmental robotics]]

* [[Robot learning]]

== References ==

{{reflist}}

==External links==

=== Technical committees ===

*IEEE Technical committeeCommittee on Cognitive and Developmental Systems (CDSTC), previously known as IEEE Technical Committee on Autonomous Mental Development, http[https://wwwweb.icdl-epirobarchive.org/amdtcweb/20190822214757/https://cis.ieee.org/technical-committees/cognitive-and-developmental-systems-technical-committee]

*IEEE Technical Committee on RobotCognitive LearningRobotics, httphttps://www.learningieee-robotsras.deorg/cognitive-robotics

*IEEE Technical Committee on Robot Learning, https://www.ieee-ras.org/robot-learning/

=== Academic institutions and researchers in the field ===

*[https://www.lucs.lu.se/lucs-robotics-group/ Lund University Cognitive Science - Robotics Group]

* [http://www.iub.edu/~cogdev/ Cognitive Development Lab, University of Indiana, US]

* [[Michigan State University]] – [http://www.cse.msu.edu/ei Embodied Intelligence Lab]

Line 82 ⟶ 91:

* [http://www.isi.imi.i.u-tokyo.ac.jp/ University of Tokyo—Intelligent Systems and Informatics Lab]

* [http://www.idsia.ch/~juergen/cogbotlab.html Cognitive Robotics Lab] of [[Juergen Schmidhuber]] at [[IDSIA]] and [[Technical University of Munich]]

* [https://web.archive.org/web/20190420191303/http://www.liralab.it/ LIRA-Lab], University of Genova, Italy

* [https://www.cit-ec.de/ CITEC at University of Bielefeld, Germany]

* [http://matthew.siu.edu/ Vision Lab], Psychology Department, Southern Illinois University Carbondale

* [http://fias.uni-frankfurt.de/~triesch/ FIAS (J. Triesch lab.)]

* [http://nivea.psycho.univ-paris5.fr/ LPP, CNRS (K. Oregan lab.)]

* [http://a-labs.aldebaran.com/labs/ai-lab AI Lab], AldebaranSoftBank Robotics Europe, France

* [http://homepages.abdn.ac.uk/f.guerin/pages/ Departement of Computer Science, University of Aberdeen]

* [https://web.archive.org/web/20130601122743/http://www.er.ams.eng.osaka-u.ac.jp/asadalab/index_en.html Asada Laboratory], Department of Adaptive Machine Systems, Graduate School of Engineering, Osaka University, Japan

* The University of Texas at Austin, [http://www.cs.utexas.edu/users/qr/robotics/bootstrap-learning.html UTCS Intelligent Robotics Lab]

* [[Bryn Mawr College]]'s [http://cs.brynmawr.edu/devrob/ Developmental Robotics Project]: research projects by faculty and students at Swarthmore and Bryn Mawr Colleges, Philadelphia, PA, USA

* [https://web.archive.org/web/20070222045437/http://eksl.isi.edu/cgi-bin/page.cgi?page=project-jean.html Jean Project]: Information Sciences Institute of the University of Southern California

* [http://www.nrl.navy.mil/aic/iss/aas/CognitiveRobots.php Cognitive Robotics (including Hide and Seek) at the Naval Research Laboratory] {{Webarchive|url=https://web.archive.org/web/20100808015544/http://www.nrl.navy.mil/aic/iss/aas/CognitiveRobots.php |date=August 8, 2010 }}

* [http://www-robotics.cs.umass.edu/index.php The Laboratory for Perceptual Robotics], [[University of Massachusetts Amherst]] Amherst, USA

* [https://web.archive.org/web/20170714151253/http://www.tech.plym.ac.uk/SOCCESoCCE/CRNS/ Centre for Robotics and Neural Systems], [http://www.plymouth.ac.uk/ Plymouth University] Plymouth, United Kingdom

* [http://www.istc.cnr.it/group/locen Laboratory of Computational Embodied Neuroscience], [http://www.istc.cnr.it/ Institute of Cognitive Science and Technologies] [https://web.archive.org/web/20140209072327/http://www.cnr.it/sitocnr/home.html National Research Council], Rome, Italy

* [https://web.archive.org/web/20141006074808/http://www-etis.ensea.fr/index.php/neuro-neurocybernetics.html Neurocybernetic team], ETIS Lab., ENSEA – University of Cergy-Pontoise – CNRS, France

* [http://mpcrlab.com Machine Perception and Cognitive Robotics Lab], Florida Atlantic University, Boca Raton, Florida

* [http://adapt.informatik.hu-berlin.de/ Adaptive Systems Group], Department of Computer Science, Humboldt University of Berlin, Germany

* [http://developmental-robotics.jp/en/ Cognitive Developmental Robotics Lab (Nagai Lab)], The University of Tokyo, Japan

=== Related large-scale projects ===

* [http://www.robotdoc.org RobotDoC Project] (funded by European Commission)

* [http://www.italkproject.org/ Italk Project] (funded by European Commission)

* [https://web.archive.org/web/20130728113632/http://www.im-clever.eu/ IM-CLeVeR Project] (funded by European Commission)

* [http://flowers.inria.Fr ERC Grant EXPLORERS Project] (funded by European Research Council)

* [http://www.robotcub.org/ RobotCub Project] (funded by European Commission)

Line 108 ⟶ 120:

=== Courses ===

The first undergraduate [https://web.archive.org/web/20061013103459/http://dangermouse.brynmawr.edu/cs380/ courses] in DevRob were offered at [[Bryn Mawr College]] and [[Swarthmore College]] in the Spring of 2003 by Douglas Blank and Lisa Meeden, respectively.

The [https://web.archive.org/web/20061012010522/http://www.cs.iastate.edu/~alex/classes/2005_Fall_610as/ first graduate course] in DevRob was offered at [[Iowa State University]] by Alexander Stoytchev in the Fall of 2005.

===Blogs and other links===

* The Mental Development Repository: http://www.mentaldev.org

* Developing Intelligence: http://develintel.blogspot.com

* Developmental Robotics: http://developmentalrobotics.org : general information about developmental robotics

== References ==

{{reflist}}

{{Robotics}}