Program Advisory Committee
Andreas K. Engel, Dept. of Neurophysiology and Pathophysiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
Karl Friston, Wellcome Trust Centre for Neuroimaging, University College London, London, UK
Bernhard Hommel, Faculty of Social Sciences, Institute of Psychology, University of Leiden, The Netherlands
Danica Kragic, School of Computer Science and Communication, Centre for Autonomous Systems, Royal Institute of Technology, Stockholm, Sweden
Julia Lupp, Ernst Strüngmann Forum, Frankfurt, Germany
Paul Verschure, Center of Autonomous Systems and Neurorobotics, Technology Department, Universitat Pompeu Fabra, Barcelona, Spain
Gottfried Vosgerau, Institute of Philosophy, University of Düsseldorf, Düsseldorf, Germany
In cognitive science, we are currently witnessing a "pragmatic turn" away from the traditional representation-centered framework toward a paradigm that focuses on understanding cognition as "enactive," as a form of practice. A key premise of this view holds that cognition should not be understood as serving to make models of the world, but rather as subserving action and being grounded in sensorimotor skills. Accordingly, cognitive states and their associated neural activity patterns should be studied primarily with respect to their functional role in action generation. Such an action-oriented paradigm seems not only conceptually viable, but is already supported by much experimental evidence. Numerous neurobiological findings demonstrate either overtly the action-relatedness of cognitive processing or can be reinterpreted in this new framework. Moreover, new vistas on the functional relevance and the presumed "representational" nature of neural processes are likely to emerge from this paradigm.
The goal of the proposed Forum would be to examine the key concepts of an emerging action-oriented view of cognition and the consequences of such a paradigm shift.
This Forum would contribute to this paradigm shift by enabling a collective discussion and elaboration of key concepts involved in this pragmatic turn, which in turn could be applied and ultimately result in novel approaches in a number of fields, including cognitive science, neuroscience, psychology, robotics, and philosophy of mind. While action-oriented views are currently beginning to emerge in these individual fields, strong links across these domains are mostly lacking. Bringing together leading proponents from these fields to a Strüngmann Forum would substantially speed up interdisciplinary interactions and, by confronting different approaches and types of data, enable novel and more integrated perspectives.Top of page
Since its formation as a discipline aimed at providing a naturalistic account of the mind and its processes, cognitive science has been dominated by a computational-representational view of cognition. The following key assumptions characterize the classical representation-centered paradigm in cognitive science:
These key assumptions, which go back to the work of Fodor (1981), Newell and Simon (1972), and other protagonists of the representational theory of mind, seem to be present, albeit with different emphasis, in most theoretical accounts on cognition. While this paradigm has been highly fruitful in stimulating important research in the early decades of cognitive science, massive criticisms have been summoned, claiming that the classical view may be highly biased if not misleading in nature (Winograd and Flores 1986; Brooks 1991; Varela et al 1991; Dreyfus 1992; Clark 1995, 1998; O'Regan and Noe 2001; Noe 2004; Engel et al 2013).
In response, an action-oriented paradigm is emerging (Varela et al 1991; Clark 1998; O'Regan and Noe 2001; Noe 2004, 2009, O'Regan 2011). Initially, this paradigm shift began, and has most explicitly been developed, in robotics (Winograd and Flores 1986; Brooks 1991; Dreyfus 1992; Pfeifer and Bongard 2007). More recently it has begun to gain impact on cognitive psychology (Hommel et al 2001; O'Regan and Noe 2001; Schütz-Bosbach and Prinz 2007) and neuroscience (Jeannerod 2001; Beauchamp and Martin 2007; Friston 2010; Friston et al 2010; Pulvermüller and Fadiga 2010; Engel 2010; Engel et al 2013).
The basic concept of the action-oriented paradigm is that cognition should not be understood as a capacity of deriving world models, which then might provide a database for thinking, planning, and problem-solving. Rather, it emphasizes that cognitive processes are closely intertwined with action and that cognition can best be understood as "enactive"; that is, as a form of practice itself. The view, which is advocated by proponents of the pragmatic turn, can be summarized as follows (Varela et al 1991; Clark 1998; Noe 2009; Engel 2010; O'Regan 2011; Engel et al 2013):
It should be noted that the concept of action, as used here, is neither coextensive with that of behavior nor with that of movement (Mead 1938; Engel et al 2013). The expression of action is used in a wider sense, which also includes acts not involving any overt movements like, e.g., thinking, calculating, imagining or deciding. The description of acts or actions typically makes references to goals, whereas behavior can be described without making any reference to mental states.
An action-oriented paradigm is supported by a number of prominent and highly discussed conceptual approaches. The notion that cognition can only by understood by considering its inherent action-relatedness is a key postulate of the "enactive" approach developed by Varela, Thompson and Rosch (1991). According to their view, cognition should not be considered as producing veridical representations of the environment but, rather, as the capacity of generating structure by action (Varela et al 1991). A related, strongly action-oriented view of cognition has also been advocated by Clark (1995, 1998).
Of particular relevance in this context is the sensorimotor contingency theory put forward by O’Regan and Noë (2001). This approach builds on earlier approaches in explaining the fundamental role of action for perception and awareness like, e.g., Gibson's affordances (Gibson 1979). It also relates to older neurobiological concepts such as the "reafference principle" of von Holst and Mittelstaedt (1950), who discovered that an efference copy is needed for the unambiguous interpretation of sensory signals (Wolpert and Flanagan 2001; Wolpert et al 2011; Friston 2011). According to O’Regan and Noe (2001), the agent’s sensorimotor contingencies (SMCs) are constitutive for cognitive processes. In this framework, SMCs are defined as law-like relations between movements and associated changes in sensory inputs that are produced by the agent’s actions. Once acquired, an agent can use these SMCs to predict consequences of its own actions.
Recent work in cognitive robotics suggests that learning of such predictions could, for instance, mediate the acquisition of object concepts (Krüger et al 2007; Bergstrom et al 2011; Maye and Engel 2012), grounding knowledge of objects in repertoires of actions that can be executed upon them. This theoretical perspective is also closely related to the active inference approach to action and perception (Friston and Stephan 2007; Friston 2010; Friston et al 2010) as well as to models of predictive coding (Rao and Ballard 1999).
An action-oriented paradigm has the potential of changing our view of the brain and its function profoundly. If mapped to the neuroscientific level of description, the conceptual premises of the pragmatic stance may lead to a redefinition of some of the basic explananda. Then, neuroscience would not need to explain how brains act as world-mirroring devices (Marr 1982; Churchland 1994), but rather how they can serve as "vehicles of world-making" (Varela et al 1991) that support, based on individual learning history, the construction of the experienced world and the guidance of action. Data from developmental and cognitive neuroscience seem to support such a departure from more classical views on cognition and brain function. These findings and their potential implications would constitute the starting point for the proposed Forum.
The time is ripe for a discussion and elaboration of such action-oriented approaches in a number of fields including cognitive science, neuroscience, psychology, robotics, and philosophy of mind. The Ernst Strüngmann Forum is an ideal setting in which to explore the preconditions and possible consequences of such a paradigm shift. Due to its highly interdisciplinary nature and the openness of the discussions that are key elements of the Forum, action-oriented views emerging in these different fields could cross-fertilize and start to integrate, paving the way toward a novel theory of cognition.Top of page
Key Controversial Issues to be Discussed at the Forum
Challenging representations: The functional role and the semantics of neural states is one of the key topics for the planned Forum. The insight that cognition may be fundamentally grounded in action seems to enforce a radical change in how we conceive of the functional significance of neural activity patterns. Rather than describing states of the outside world, brain states prescribe possible actions (Clark 1998). Thus, they might better be understood as “directives” guiding action (Engel et al 2013), rather than as “representations.”
Cognitive role of motor brain structures: A key question is to what degree the function of motor regions can be understood as directly supporting cognition, as opposed to a view which assigns merely “output” functions to these circuits. For instance, attention and decision making may rely much more on motor regions than previously assumed (Engel et al 2013). From the viewpoint of an integrated sensorimotor approach, it may be questioned whether it makes sense to use the classical categories of “motor” and “sensory” cortex. Rather, these cortical regions might by viewed as proprioceptive and exteroceptive sensorimotor areas encoding SMCs. For example, visual cortex might be considered as the recipient of top-down predictions about the consequences of oculomotor acts.
Role of sensorimotor contingencies: While the role of SMCs in basic sensorimotor integration seems undisputed, a critical question is to what extent more complex cognitive functions, such as object recognition or action planning, can be achieved by learning SMCs (Maye and Engel 2012). The challenging question is whether SMC knowledge is sufficient for implementing complex cognitive functions or whether higher cognitive processes based on other principles are required.
Predictive coding and active inference: A key mechanism for cognitive processing seems to be the optimization of predictions and the minimization of prediction errors. It has been suggested that new views unifying perception, cognition and motor control may emerge from this basic principle (Friston 2010; Friston et al 2010). The implications of this principle for a novel understanding of the relation between action and cognition would be another key topic for the Forum.
Action-relatedness of phenomenal states: In more classical accounts, conscious awareness is largely detached from action and the activation of motor circuits. An important question is whether implications of the pragmatic turn would also concern current models of consciousness. The sensorimotor account (O’Regan and Noe 2001)rt the notion of the pragmatic turn. For developmental robotics this issue is equally relevant to answer questions like how robots can achieve mastery of high-dimensional action spaces.
Joint action and social cognition: Social aspects of cognitive processing have, in many accounts, been conceptualized as mainly involving “theory-of-mind” type representations in the individual brain. A key question is whether this largely disembodied approach fully captures the nature of social interactions. Alternatively, a perspective aiming at grounding social cognition in joint action, including e.g. synchronized movements, might have large potential. Ambitious questions are whether enactive approaches to social cognition would also extend to interactions between humans and robots and what mechanisms might establish social cognition in artificial agents.
Agenda for pragmatic neuroscience: A key question is whether the conceptual shifts implied by the pragmatic turn may lead to the development of novel experimental paradigms and strategies. A radical action-orientedness would violate many practical constraints and theoretical premises of neuroscience as they have been functioning in the past decades.Top of page
Working Structure and Key Questions for the Discussion Groups
We have defined four themes for the group discussions of the proposed Forum. The first group will address action-oriented views in relation to the development of cognition. The second group will focus on novel perspectives emerging for the understanding of consciousness. The third group will discuss action-oriented models for other cognitive processes. Finally, the fourth discussion group will address, at a more general level, the potential contributions of action-oriented views to a paradigm shift in cognitive science. A number of overarching questions for the discussion in these four groups are suggested below.
Issues to be addressed by all four discussion groups:
Group 1: Development, acquisition, and adaptation of action-oriented processing
Group 2: Action-oriented models of cognitive and functional processing
Group 3: Action-oriented understanding of consciousness and the structure of experience
Group 4: Implications of action-oriented paradigm shifts in cognitive science
Beauchamp MS, Martin A (2007) Grounding object concepts in perception and action: evidence from fMRI studies of tools. Cortex 43: 461-468
Bergstrom N, Bjorkman M, Kragic D (2011) Generating object hypotheses in natural scenes through human-robot interaction. In: Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ International Conference on, pp 827-833
Brooks RA (1991) Intelligence without representation. Artif Intell 47: 139-159
Churchland PS et al (1994) A critique of pure vision. In Large-scale Neuronal Theories of the Brain (Koch C, Davis J, eds), pp 23-60, The MIT Press
Clark A (1995) Moving minds: situating content in the service of real-time success. Phil Perspect 9: 89-104
Clark A (1998) Being There: Putting Brain, Body, and World Together Again. The MIT Press
Dreyfus HL (1992) What Computers Still Can't Do: A Critique of Artificial Reason. The MIT Press
Engel AK (2010) Directive minds: how dynamics shapes cognition. In Enaction: Toward a New Paradigm for Cognitive Science (Stewart J et al, eds), pp 219-243, The MIT Press
Engel AK, Maye A, Kurthen M, König P (2013) Where‘s the action? The pragmatic turn in cognitive science. Trends Cogn Sci 17: 8-15
Fodor JA (1981) Representations: Essays on the Foundations of Cognitive Science. The MIT Press
Friston KJ, Stephan KE (2007) Free-energy and the brain. Synthese 159: 417-458
Friston KJ (2010) The free-energy principle: a unified brain theory? Nat Rev Neurosci 11: 127-138
Friston KJ, Daunizeau J, Kilner J, Kiebel SJ (2010) Action and behaviour: a free-energy formulation. Biol Cybern 102: 227-260
Gibson JJ (1979) The Ecological Approach to Visual Perception. Houghton Mifflin, Boston
Hommel B et al (2001) The Theory of Event Coding (TEC): a framework for perception and action planning. Behav Brain Sci 24: 849-878
Jeannerod M (2001) Neural simulation of action: a unifying mechanism for motor cognition. Neuroimage. 14: S103-S109
Krüger V, Kragic D, Ude A, Geib C (2007) The meaning of action: a review on action recognition and mapping. Advanced Robotics 21: 1473-1501
Marr D (1982) Vision: A Computational Approach. Freeman & Co
Maye A, Engel AK (2012) Time scales of sensorimotor contingencies. In: Advances in Brain Inspired Cognitive Systems (BICS) 2012 (Zhang H, eds), pp. 240-249. Springer-Verlag
Mead GH (1938) The Philosophy of the Act. University of Chicago Press
Newell A, Simon HA (1972) Human Problem Solving. Prentice-Hall
Noe A (2004) Action in Perception. The MIT Press
Noe A (2009) Out of Our Heads. Hill & Wang
O'Regan JK, Noe A (2001) A sensorimotor account of vision and visual consciousness. Behav Brain Sci 24: 939-973
O'Regan JK (2011) Why Red Doesn't Sound Like a Bell: Understanding the Feel of Consciousness. Oxford University Press
Pfeifer R, Bongard J (2007) How the Body Shapes the Way We Think: A New View of Intelligence. Bradford Books
Pulvermüller F, Fadiga L (2010) Active perception: sensorimotor circuits as a cortical basis for language. Nat Rev Neurosci 11: 351-360
Rao RP, Ballard DH (1999) Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat Neurosci 2: 79-87
Schütz-Bosbach S, Prinz W (2007) Perceptual resonance: action-induced modulation of perception. Trends Cogn Sci 11: 349-355
Varela FJ et al (1991) The Embodied Mind: Cognitive Science and Human Experience. The MIT Press
von Holst E, Mittelstaedt H (1950) Das Reafferenzprinzip. Die Naturwissenschaften 37: 464-476
Winograd T, Flores F (1986) Understanding Computers and Cognition: A New Foundation for Design. Ablex Publishing Corporation
Wolpert DM, Flanagan JR (2001) Motor prediction. Curr Biol 11: 729-732
Wolpert DM, Diedrichsen J, Flanagan JR (2011) Principles of sensorimotor learning. Nat Rev Neurosci 12: 739-751Top of page
(GZ S1 505/15-1) http://www.dfg.de/en/
This Forum is supported by the eSMCs project, which is funded by an EU 7th Framework Programme grant. The eSMCs project focuses on the role of sensorimotor predictions for cognitive processing, which is addressed in studies on humans and robots.
The European research project "Convergence Science Network of Biomimetics and NeuroTechnology"
(FP7-601167) has provided scholarships to enable the participation of 4 European Young Scientists active in the Neuro and Bio-Inspired Systems area.