Volume 9 · Number 1 ·
Pages 46–56
A Computational Constructivist Model as an Anticipatory Learning Mechanism for Coupled Agent–Environment Systems
Log in to download the full text for free
Abstract
Context: The advent of a general artificial intelligence mechanism that learns like humans do would represent the realization of an old and major dream of science. It could be achieved by an artifact able to develop its own cognitive structures following constructivist principles. However, there is a large distance between the descriptions of the intelligence made by constructivist theories and the mechanisms that currently exist. Problem: The constructivist conception of intelligence is very powerful for explaining how cognitive development takes place. However, until now, no computational model has successfully demonstrated the underlying mechanisms necessary to realize it. In other words, the artificial intelligence (AI) community has not been able to give rise to a system that convincingly implements the principles of intelligence as postulated by constructivism, and that is also capable of dealing with complex environments. Results: This paper presents the constructivist anticipatory learning mechanism (CALM), an agent learning mechanism based on the constructivist approach of AI. It is designed to deal dynamically and interactively with environments that are at the same time partially deterministic and partially observable. CALM can model the regularities experienced in the interaction with the environment, on the sensorimotor level as well, as by constructing abstract or high-level representational concepts. The created model provides the knowledge necessary to generate the agent behavior. The paper also presents the coupled agent environment system (CAES) meta-architecture, which defines a conception of an autonomous agent, situated in the environment, embodied and intrinsically motivated. Implications: The paper can be seen as a step towards a computational implementation of constructivist principles, on the one hand suggesting a further perspective of this refreshing movement on the AI field (which is still too steeped in a behaviorist influence and dominated by probabilistic models and narrow applied approaches), and on the other hand bringing some abstract descriptions of the cognitive process into a more concrete dimension, in the form of algorithms. Constructivist content: The connection of this paper with constructivism is the proposal of a computational and formally described mechanism that implements important aspects of the subjective process of knowledge construction based on key ideas proposed by constructivist theories.
Key words: Factored partially observable Markov decision process (FPOMDP), computational constructivist learning mechanisms, anticipatory learning, model-based learning.
Citation
Perotto F. S. (2013) A computational constructivist model as an anticipatory learning mechanism for coupled agent–environment systems. Constructivist Foundations 9(1): 46–56. http://constructivist.info/9/1/046
Export article citation data: Plain Text · BibTex · EndNote · Reference Manager (RIS)
Similar articles
References
Allen J. W. P. & Bickhard M. H. (2011) Normativity: A crucial kind of emergence. Human Development 54: 106–112. ▸︎ Google︎ Scholar
Allen J. W. P. & Bickhard M. H. (2013) Beyond principles and programs: An action framework for modeling development. Human Development 56: 171–177. ▸︎ Google︎ Scholar
Allen J. W. P. & Bickhard M. H. (2013) Stepping off the pendulum: Why only an action-based approach can transcend the nativist-empiricist debate. Cognitive Development 28: 96–133. ▸︎ Google︎ Scholar
Allen J. W. P. & Bickhard M. H. (2013) The pendulum still swings. Cognitive Development 28: 164–174. ▸︎ Google︎ Scholar
Anderson M. (2003) Embodied cognition: A field guide. Artificial Intelligence 149(1): 91–130. ▸︎ Google︎ Scholar
Ashby W. R. (1952) Design for a brain. First Edition. Chapman & Hall, London. ▸︎ Google︎ Scholar
Barandiaran X. & Moreno A. (2006) On what makes certain dynamical systems cognitive. Adaptive Behavior 14(2): 171–185. ▸︎ Google︎ Scholar
Barandiaran X. & Moreno A. (2008) Adaptivity: From metabolism to behavior. Adaptive Behavior 16(5): 325–344. ▸︎ Google︎ Scholar
Barsalou L. W. (1999) Perceptual symbol systems. Behavioral and Brain Sciences 22: 577–600. ▸︎ Google︎ Scholar
Barsalou L. W. (2008) Grounded cognition. Annual Review of Psychology 59: 617–645. ▸︎ Google︎ Scholar
Beer R. D. (1995) A dynamical systems perspective on agent-environment interactions. Artificial Intelligence 72: 173–215. ▸︎ Google︎ Scholar
Beer R. D. (2004) Autopoiesis and cognition in the game of life. Artificial Life 10(3): 309–326. ▸︎ Google︎ Scholar
Beetz M., Jain D., Mösenlechner L. & Tenorth M. (2010) Towards performing everyday manipulation activities. Robotics and Autonomous System 58(9): 1085–1095. ▸︎ Google︎ Scholar
Bellman R. A. (1957) Markovian decision process. Journal of Mathematics and Mechanics 6: 679–684. ▸︎ Google︎ Scholar
Bickhard M. H. & Campbell R. L. (1996) Topologies of learning and development. New Ideas in Psychology 14(2): 111–156. ▸︎ Google︎ Scholar
Bickhard M. H. & Richie D. M. (1983) On the nature of representation: A case study of James Gibson’s theory of perception. Praeger Publishers, New York. ▸︎ Google︎ Scholar
Bickhard M. H. (2009) Interactivism: A manifesto. New Ideas in Psychology 27: 85–95. ▸︎ Google︎ Scholar
Bickhard M. H. (2009) The interactivist model. Synthese 166(3), 547–591. ▸︎ Google︎ Scholar
Blum A. L. & Langley P. (1997) Selection of relevant features and examples in machine learning. Artificial Intelligence 97: 245–271. ▸︎ Google︎ Scholar
Boutilier C., Dearden R. & Goldszmidt M. (2000) Stochastic dynamic programming with factored representations. Artificial Intelligence 121(1–2): 49–107. ▸︎ Google︎ Scholar
Bril B., Roux V., Dietrich G. (2005) Stone knapping: Khambhat (India), a unique opportunity? In: Roux V. & Bril B. (eds.) Stone Knapping: The necessary conditions for a uniquely hominid behaviour. McDonald Press, Cambridge MA: 53–72. ▸︎ Google︎ Scholar
Brooks R. A. (1991) Intelligence without representation. Artificial Intelligence 47: 139–159. ▸︎ Google︎ Scholar
Buisson J.-C. (2004) A rhythm recognition computer program to advocate interactivist perception. Cognitive Science 28(1): 75–87. ▸︎ Google︎ Scholar
Butz M. V. (2002) Anticipatory learning classifier systems. Kluwer, Boston MA. ▸︎ Google︎ Scholar
Butz M. V. (2008) How and why the brain lays the foundations for a conscious self. Constructivist Foundations 4(1): 1–14 & 32–37. http://www.univie.ac.at/constructivism/journal/4/1/001.butz
Butz M. V. (2013) Separating goals from behavioral control: Implications from learning predictive modularizations. New Ideas in Psychology 31: 302–312. ▸︎ Google︎ Scholar
Byrne R. W. (2005) The maker not the tool: The cognitive significance of great ape manual skills. In: Roux V. & Bril B. (eds.) Stone Knapping: The necessary conditions for a uniquely hominid behaviour. McDonald Press, Cambridge MA: 159–169. ▸︎ Google︎ Scholar
Campbell D. T. (1974) Evolutionary epistemology. In: Schilpp P. A. (ed.) The philosophy of Karl Popper. Open Court, LaSalle IL: 413–463. ▸︎ Google︎ Scholar
Campbell R. L. & Bickhard M. H. (1993) Knowing levels and the child’s understanding of mind. Behavioral and Brain Sciences 16(1): 33–34. ▸︎ Google︎ Scholar
Cassandra A. R., Kaelbling L. P. & Littman M. L. (1998) Planning and acting in partially observable stochastic domains. Artificial Intelligence 101: 99–134. ▸︎ Google︎ Scholar
Chaput H. (2004) The constructivist learning architecture. Unpublished PhD Thesis, University of Texas. ▸︎ Google︎ Scholar
Conant R. C. & Ashby W. R. (1970) Every good regulator of a system must be a model of that system. International Journal of Systems Science 1(2): 89–97. ▸︎ Google︎ Scholar
Crook P. & Hayes G. (2003) Could active perception aid navigation of partially observable grid worlds? In: Proceedings of the 14th European Conference on Machine Learning (ECML). Lecture Notes in Artificial Intelligence 2837. Springer-Verlag, Berlin: 72–83. ▸︎ Google︎ Scholar
Degris T. & Sigaud O. (2010) Factored markov decision processes. In: Buffet O. & Sigaud O. (eds.). Markov decision processes in artificial intelligence. Wiley-ISTE, London: 99–125. ▸︎ Google︎ Scholar
Degris T., Sigaud O. & Wuillemin P.-H. (2006) Learning the structure of factored markov decision processes in reinforcement learning problems. In: Proceedings of the 23rd International Conference on Machine Learning. ACM Press: 257–264. ▸︎ Google︎ Scholar
Drescher G. (1991) Made-up minds: A constructivist approach to artificial intelligence. MIT Press, Cambridge MA. ▸︎ Google︎ Scholar
Feinberg E. A. & Shwartz A. (2002) Handbook of Markov decision processes: Methods and applications. Kluwer, Norwell MA. ▸︎ Google︎ Scholar
Foerster H. von (2003) Understanding understanding: Essays on cybernetics and cognition. Springer, New York. ▸︎ Google︎ Scholar
Foner L. N. & Maes P. (1994) Paying attention to what’s important: Using focus of attention to improve unsupervised learning. In: Cliff D., Husbands P., Meyer J. & Wilson S. (eds.) From animals to animats 3. Proceedings of the Third International Conference on Simulation of Adaptive Behavior. MIT Press, Cambridge MA: 256–265. ▸︎ Google︎ Scholar
Glasersfeld E. von (1974) Piaget and the radical constructivist epistemology. In: Smock C. D. & Glasersfeld E. von (eds.) Epistemology and education. Follow Through Publications, Athens GA: 1–24. http://www.vonglasersfeld.com/034
Glasersfeld E. von (1979) Cybernetics, experience, and the concept of self. In: Ozer M. N. (ed.) A cybernetic approach to the assessment of children: Toward a more humane use of human beings. Westview Press: Boulder, CO, pp. 67–113. http://www.vonglasersfeld.com/056/
Glasersfeld E. von (2005) 30 years of radical constructivism. Constructivist Foundations 1(1): 9–12. http://www.univie.ac.at/constructivism/journal/1/1/009.glasersfeld
Goldstein J. (1999) Emergence as a construct: History and issues. Emergence: Complexity and Organization 1: 49–72. ▸︎ Google︎ Scholar
Guerin F. (2011) Learning like baby: A survey of AI approaches. Knowledge Engineering Review 26(2): 209–236. ▸︎ Google︎ Scholar
Guestrin C., Koller D. & Parr R. (2001) Solving factored POMDPs with linear value functions. In: Proceedings of the Workshop on Planning under Uncertainty and Incomplete Information. Morgan Kaufmann, San Francisco CA: 67–75. ▸︎ Google︎ Scholar
Guestrin C., Koller D., Parr R. & Venkataraman S. (2003) Efficient solution algorithms for factored MDPs. Journal of Artificial Intelligence Research 19: 399–468. ▸︎ Google︎ Scholar
Hansen E. A. & Feng Z. (2000) Dynamic programming for POMDPs using a factored state representation. In: Proceedings of the Fifth International Conference on Artificial Intelligence, Planning and Scheduling. AAAI Press, Menlo Park CA:130–139. ▸︎ Google︎ Scholar
Hauskrecht M. (2000) Value-function approximations for partially observable Markov decision processes. Journal of Artificial Intelligence Research 13: 33–94. ▸︎ Google︎ Scholar
Hoffmann J. (2003) Anticipatory behavioral control. In: Butz M. V., Sigaud O. & Gérard P. (eds.) Anticipatory behavior in adaptive learning systems: Foundations, theories, and systems. Springer, Berlin: 44–65. ▸︎ Google︎ Scholar
Holmes M. & Isbell C. (2005) Schema learning: Experience-based construction of predictive action models. Advances in Neural Information Processing Systems 17: 562–585. ▸︎ Google︎ Scholar
Holmes M. & Isbell C. (2006) Looping suffix tree-based inference of partially observable hidden state. In: Cohen W. W. & Moore A. (eds.) Proceedings of the 23rd International Conference on Machine Learning (ICML 2006) ACM Press, New York: 409–416. ▸︎ Google︎ Scholar
Hommel B., Müsseler J., Aschersleben G. & Prinz W. (2001) The theory of event coding (TEC): A framework for perception and action planning. Behavioral and Brain Sciences 24: 849–878. ▸︎ Google︎ Scholar
Indurkhya B. (1992) Metaphor and cognition: An interactionist approach. Kluwer, Dordrecht. ▸︎ Google︎ Scholar
Joas H. (1993) American pragmatism and German thought: A history of misunderstandings. In: Joas H., Pragmatism and social theory. University of Chicago Press, Chicago: 94–121. ▸︎ Google︎ Scholar
Jonsson A. & Barto A. (2005) A causal approach to hierarchical decomposition of factored MDPs. In: De Raedt L. & Wrobel S.(eds.) Proceedings of the 22nd International Conference on Machine Learning (ICML 2005) ACM Press, New York: 401–408. ▸︎ Google︎ Scholar
Kitchener F. K. (1986) Piaget’s theory of knowledge. Yale University Press, New Haven. ▸︎ Google︎ Scholar
Maturana H. R. & Varela F. J. (1980) Autopoiesis and cognition. Reidel, Dordrecht. ▸︎ Google︎ Scholar
Minsky M. (1988) The society of mind. Simon & Schuster, New York. ▸︎ Google︎ Scholar
Modayil J., White A. & Sutton R. S. (2012) Multi-timescale nexting in a reinforcement learning robot. In: Ziemke T., Balkenius C. & Hallam J. (eds.) Proceedings of the Conference on the Simulation of Adaptive behavior (SAB’12) Springer, Heidelberg: 299–309. ▸︎ Google︎ Scholar
Morrison C., Oates T. & King G. (2001) Grounding the unobservable in the observable: The role and representation of hidden state in concept formation and refinement. In: Working notes of AAAI Spring Symposium. AAAI Press, Menlo Park CA: 45–49. ▸︎ Google︎ Scholar
Nivel E. & Thórisson K. R. (2013) Towards a programming paradigm for control systems with high levels of existential autonomy. In: Kühnberger K.-U., Rudolph S. & Wang P. (eds.) Artificial General Intelligence. Proceedings of the Sixth International Conference, AGI-13, Beijing, China, July/August 2013. Springer, Berlin: 78–87. ▸︎ Google︎ Scholar
Nivel E., Thórisson K. R., Dindo H., Pezzulo G., Rodriguez M., Corbato C., Steunebrink B., Ognibene D., Chella A., Schmidhuber J., Sanz R. & Helgason H. P. (2013) Autocatalytic endogenous reflective architecture. Reykjavik University School of Computer Science Technical Report, RUTR-SCS13002. ▸︎ Google︎ Scholar
Parisi D. (2004) Internal robotics. Connection Science 16(4): 325–338. ▸︎ Google︎ Scholar
Pask G. (1981) Organisational closure of potentially conscious systems In: Zelany M. (ed.) Autopoiesis. North Holland Elsevier: New York: 265–307. ▸︎ Google︎ Scholar
Pask G., Scott B. & Kallikourdis D. (1973) A theory of conversations and individuals (exemplified by the learning process on CASTE). International Journal of Man-Machine Studies 5: 443–566. ▸︎ Google︎ Scholar
Pastra K. & Aloimonos Y. (2012) The minimalist grammar of action. Philosophical Transactions of the Royal Society B: Biological Sciences 367: 103–117. ▸︎ Google︎ Scholar
Perotto F. S. (2010) Un mécanisme constructiviste d’apprentissage automatique d’anticipations pour des agents artificiels situés. Unpublished Ph.D Thesis at the Institut National Polytechnique de Toulouse, France. ▸︎ Google︎ Scholar
Piaget J. (1953) The origin of intelligence in the child. Routledge & Kegan Paul. London. ▸︎ Google︎ Scholar
Piaget J. (1954) The construction of reality in the child. Routledge & Kegan Paul, London. Originally published as: Piaget J. (1937) La construction du réel chez l’enfant. Delachaux et Niestlé, Neuchâtel. ▸︎ Google︎ Scholar
Poupart P. & Boutilier C. (2004) VDCBPI: An approximate scalable algorithm for large scale POMDPs. In: Proceedings of the 17th Advances in Neural Information Processing Systems (NIPS). MIT Press, Cambridge MA: 1081–1088. ▸︎ Google︎ Scholar
Prigogine I. (1980) From being to becoming. Freeman, San Francisco CA. ▸︎ Google︎ Scholar
Puterman M. L. (1994) Markov decision processes: Discrete stochastic dynamic programming. Wiley, New York. ▸︎ Google︎ Scholar
Rivest R. & Schapire R. (1994) Diversity-based inference of finite automata. ACM Journal 43(3): 555–589. ▸︎ Google︎ Scholar
Schachner W. (ed.) (1996) Constructivism, interactionism and their applications. CEPIAG’s Symposium Proceeding. CEPIAG, Geneva. ▸︎ Google︎ Scholar
Schachner W., Real del Sarte O. & León C. (1999) Valore, indice e rappresentazione nella guida dei valori elementari. Metis 1: 7–22. ▸︎ Google︎ Scholar
Scott B. & Shurville S. (2011) What is a symbol? Kybernetes 48(1/2): 12–22. ▸︎ Google︎ Scholar
Scott B. (2007) The co-emergence of parts and wholes in psychological individuation. Constructivist Foundations 2(2–3): 65–71. http://www.univie.ac.at/constructivism/journal/2/2–3/065.scott
Searle J. (1980) Minds, brains, and programs. Behavioural and Brain Sciences 3: 417–424. ▸︎ Google︎ Scholar
Shani G., Brafman R. I. & Shimony S. E. (2005) Model-based online learning of POMDPs. In: Proceedings of the 16th European Conference on Machine Learning (ECML). Springer, Berlin: 353–364. ▸︎ Google︎ Scholar
Shani G., Poupart P., Brafman R. I. & Shimony S. E. (2008) Efficient ADD operations for point-based algorithms. In: Proceedings of the 8th International Conference on Automated Planning and Scheduling (ICAPS). AAAI Press, Menlo Park CA: 330–337. ▸︎ Google︎ Scholar
Sigaud O., Butz M. V., Kozlova O. & Meyer C. (2009) Anticipatory learning classifier systems and factored reinforcement learning. In: Pezzulo G., Butz M. V., Sigaud O. & Baldassarre G. (eds.). Anticipatory behavior in adaptive learning systems: From psychological theories to artificial cognitive systems. Springer, Berlin: 321–333. ▸︎ Google︎ Scholar
Sim H. S., Kim K.-E., Kim J. H., Chang D.-S. & Koo M.-W. (2008) Symbolic heuristic search value iteration for factored POMDPs. In: Proceedings of the 23rd National Conference on Artificial Intelligence. AAAI Press, Menlo Park CA: 1088–1093. ▸︎ Google︎ Scholar
Singh S., James M. & Rudary M. (2004) Predictive state representations: A new theory for modeling dynamical systems. In Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence. AUAI Press, Banff: 512–519. ▸︎ Google︎ Scholar
Singh S., Littman M., Jong N., Pardoe D & Stone P. (2003) Learning predictive state representations. In: Proceedings of the 20th International Conference on Machine Learning (ICML). AAAI Press, Menlo Park CA: 712–719. ▸︎ Google︎ Scholar
Stojanov G. & Kulakov A. (2011) Modeling attention within a complete cognitive architecture. In: Roda C. (ed.) Attention support in digital environments. Cambridge University Press, Cambridge: 210–241. ▸︎ Google︎ Scholar
Stojanov G. (2009) History of usage of Piaget’s theory of cognitive development in AI and robotics: A look backwards for a step forwards. In: Cañamero L., Oudeyer P.-Y. & Balkenius C. (eds.) Proceedings of the Ninth International Conference on Epigenetic Robotics. LUCS, Lund: 243–245. http://www.lucs.lu.se/LUCS/146/
Stojanov G., Bozinovski S. & Trajkovski G. (1997) Interactionist-expectative view on agency and learning. IMACS Journal for Mathematics and Computers in Simulation 44: 295–310. ▸︎ Google︎ Scholar
Stojanov G., Kulakov A. & Clauzel D. (2006) On curiosity in intelligent robotic systems. In: Proceedings AAAI 2006 Fall Symposium on Interaction and Emergent Phenomena in Societies of Agents, Arlington, Virginia. AAAI Press, Cambridge: 183–189. ▸︎ Google︎ Scholar
Stolzmann W. (2000) An Introduction to anticipatory classifier systems. In: Lanzi P. L., Stolzmann W. & Wilson S. W. (eds.) Learning classifier systems: From foundations to applications. Springer, Berlin: 175–194. ▸︎ Google︎ Scholar
Stoytchev A. (2009) Some basic principles of developmental robotics. IEEE Transactions on Autonomous Mental Development 1(2): 122–130. ▸︎ Google︎ Scholar
Strehl A. L., Diuk C. & Littman M. L. (2007) Efficient structure learning in factored-state MDPs. In: Proceedings of the 22nd National Conference on Artificial Intelligence, AAAI Press, Menlo Park CA: 645–650. ▸︎ Google︎ Scholar
Suchman L. A. (1987) Plans and situated actions: The problem of human-machine communication. Cambridge University Press, Cambridge. ▸︎ Google︎ Scholar
Sutton R. S. & Barto A. G. (1998) Reinforcement learning: An introduction. MIT Press, Cambridge MA. ▸︎ Google︎ Scholar
Sutton R. S., Modayil J., Delp M., Degris T., Pilarski P. M. White A. & Precup D. (2011) Horde: A scalable real-time architecture for learning knowledge from unsupervised sensorimotor interaction. In: Proceedings of the Tenth International Conference on Autonomous Agents and Multiagent Systems (AAMAS’11), Volume 2. IFAAMAS, Taipei: 761–776. ▸︎ Google︎ Scholar
Sutton R. S., Precup D. & Singh S. (1999) Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence 112: 181–211. ▸︎ Google︎ Scholar
Tani J. (1996) Model-based learning for mobile robot navigation from the dynamical systems. Perspective. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, Special Issue on learning autonomous robots 26(3): 421–436. ▸︎ Google︎ Scholar
Thórisson K. R. (2012) A new constructivist AI: From manual construction to self-constructive systems. In: Wang P. & Goertzel B. (eds.) Theoretical foundations of artificial general intelligence. Atlantis Press, Paris: 145–171. ▸︎ Google︎ Scholar
Thórisson K. R. (2013) Reductio ad absurdum: On oversimplification in computer science and its pernicious effect on artificial intelligence research. In: Abdel-Fattah A. H. M. & Kühnberger K.-U. (eds.) Proceedings of the Workshop Formalizing Mechanisms for Artificial General Intelligence and Cognition (Formal MAGiC). Institute of Cognitive Science, Osnabrück: 31–35. ▸︎ Google︎ Scholar
Thornton C. (2003) Indirect sensing through abstractive learning. Intelligent Data Analysis 7(3): 1–16. ▸︎ Google︎ Scholar
Tortora G. J. & Derrickson B. H. (2012) Principles of anatomy and physiology. 13th Edition. Wiley, New York. ▸︎ Google︎ Scholar
Varela F., Thompson E. & Rosch E. (1991) The embodied mind: Cognitive science and human experience. MIT Press, Cambridge MA. ▸︎ Google︎ Scholar
Wilson R. & Clark A. (2008) How to situate cognition: Letting nature take its course. In: Aydene M. & Robbins P. (eds.). Cambridge handbook of situated cognition. Cambridge University Press, New York. ▸︎ Google︎ Scholar
Wörgötter F., Aksoy E. E., Krüger N., Piater J., Ude A. & Tamosiunaite M. (2013) A simple ontology of manipulation actions based on hand-object relations. IEEE Transactions on Autonomous Mental Development 5: 117–134. ▸︎ Google︎ Scholar
Yavuz A. & Davenport D. (1997) PAL: A constructivist model of cognitive activity. In: Riegler A., Peschl M. F. & von Stein A. (eds.) Proceedings of the international conference “New Trends in Cognitive Science: Does Representation Need Reality?” Vienna, Austria. ▸︎ Google︎ Scholar
Ziemke T. (2003) What’s that thing called embodiment? In: Alterman R. & Kirsh D. (eds.) Proceedings of the 25th Annual Meeting of the Cognitive Science Society. Lawrence Erlbaum, Mahwah NJ: 1305–1310. ▸︎ Google︎ Scholar
Comments: 0
To stay informed about comments to this publication and post comments yourself, please log in first.