Browsing by Author "Uğur, E."

Now showing 1 - 11 of 11

Open Access
ACNMP: skill transfer and task extrapolation through learning from demonstration and reinforcement learning via representation sharing
(ML Research Press, 2020) Akbulut, M. T.; Öztop, Erhan; Xue, H.; Tekden, A. E.; Şeker, M. Y.; Uğur, E.; Computer Science; ÖZTOP, Erhan
To equip robots with dexterous skills, an effective approach is to first transfer the desired skill via Learning from Demonstration (LfD), then let the robot improve it by self-exploration via Reinforcement Learning (RL). In this paper, we propose a novel LfD+RL framework, namely Adaptive Conditional Neural Movement Primitives (ACNMP), that allows efficient policy improvement in novel environments and effective skill transfer between different agents. This is achieved through exploiting the latent representation learned by the underlying Conditional Neural Process (CNP) model, and simultaneous training of the model with supervised learning (SL) for acquiring the demonstrated trajectories and via RL for new trajectory discovery. Through simulation experiments, we show that (i) ACNMP enables the system to extrapolate to situations where pure LfD fails; (ii) Simultaneous training of the system through SL and RL preserves the shape of demonstrations while adapting to novel situations due to the shared representations used by both learners; (iii) ACNMP enables order-of-magnitude sample-efficient RL in extrapolation of reaching tasks compared to the existing approaches; (iv) ACNMPs can be used to implement skill transfer between robots having different morphology, with competitive learning speeds and importantly with less number of assumptions compared to the state-of-the-art approaches. Finally, we show the real-world suitability of ACNMPs through real robot experiments that involve obstacle avoidance, pick and place and pouring actions.
Metadata only
Adaptive shared control with human intention estimation for human agent collaboration
(IEEE, 2022) Amirshirzad, Negin; Uğur, E.; Bebek, Özkan; Öztop, Erhan; Computer Science; Mechanical Engineering; BEBEK, Özkan; ÖZTOP, Erhan; Amirshirzad, Negin
In this paper an adaptive shared control frame-work for human agent collaboration is introduced. In this framework the agent predicts the human intention with a confidence factor that also serves as the control blending parameter, that is used to combine the human and agent control commands to drive a robot or a manipulator. While performing a given task, the blending parameter is dynamically updated as the result of the interplay between human and agent control. In a scenario where additional trajectories need to be taught to the agent, either new human demonstrations can be generated and given to the learning system, or alternatively the aforementioned shared control system can be used to generate new demonstrations. The simulation study conducted in this study shows that the latter approach is more beneficial. The latter approach creates improved collaboration between the human and the agent, by decreasing the human effort and increasing the compatibility of the human and agent control commands.
Metadata only
Deep multi-object symbol learning with self-attention based predictors
(IEEE, 2023) Ahmetoğlu, A.; Öztop, Erhan; Uğur, E.; Computer Science; ÖZTOP, Erhan
This paper proposes an architecture that can learn symbolic representations from the continuous sensorimotor experience of a robot interacting with a varying number of objects. Unlike previous works, this work aims to remove constraints on the learned symbols such as a fixed number of interacted objects or pre-defined symbolic structures. The proposed architecture can learn symbols for single objects and relations between them in a unified manner. The architecture is an encoder-decoder network with a binary activation layer followed by self-attention layers. Experiments are conducted in a robotic manipulation setup with a varying number of objects. Results showed that the robot successfully encodes the interaction dynamics between a varying number of objects using the discovered symbols. We also showed that the discovered symbols can be used for planning to reach symbolic goal states by training a higher-level neural network.
Metadata only
Discovering predictive relational object symbols with symbolic attentive layers
(IEEE, 2024-02-01) Ahmetoglu, A.; Celik, B.; Öztop, Erhan; Uğur, E.; Computer Science; ÖZTOP, Erhan
In this letter, we propose and realize a new deep learning architecture for discovering symbolic representations for objects and their relations based on the self-supervised continuous interaction of a manipulator robot with multiple objects in a tabletop environment. The key feature of the model is that it can take a changing number of objects as input and map the object-object relations into symbolic domain explicitly. In the model, we employ a self-attention layer that computes discrete attention weights from object features, which are treated as relational symbols between objects. These relational symbols are then used to aggregate the learned object symbols and predict the effects of executed actions on each object. The result is a pipeline that allows the formation of object symbols and relational symbols from a dataset of object features, actions, and effects in an end-to-end manner. We compare the performance of our proposed architecture with state-of-the-art symbol discovery methods in a simulated tabletop environment where the robot needs to discover symbols related to the relative positions of objects to predict the action's result. Our experiments show that the proposed architecture performs better than other baselines in effect prediction while forming not only object symbols but also relational symbols.
Metadata only
An ecologically valid reference frame for perspective invariant action recognition
(IEEE, 2021) Bayram, Berkay; Uğur, E.; Asada, M.; Öztop, Erhan; Computer Science; ÖZTOP, Erhan; Bayram, Berkay
In robotics, objects and body parts can be represented in various coordinate frames to ease computation. In biological systems, body or body part centered coordinate frames have been proposed as possible reference frames that the brain uses for interacting with the environment. Coordinate transformations are standard tools in robotics and can facilitate perspective invariant action recognition and action prediction based on observed actions of other agents. Although it is known that human adults can do explicit coordinate transformations, it is not clear whether this capability is used for recognizing and understanding the actions of others. Mirror neurons, found in the ventral premotor cortex of macaque monkeys, seem to undertake action understanding in a perspective invariant way, which may rely on lower level perceptual mechanisms. To this end, in this paper we propose a novel reference frame that is ecologically plausible and can sustain basic action understanding and mirror function. We demonstrate the potential of this representation by simulation of an upper body humanoid robot with an action repertoire consisting of push, poke, move-away and bring-to-mouth actions. The simulation experiments indicate that the representation is suitable for action recognition and effect prediction in a perspective invariant way, and thus can be deployed as an artificial mirror system for robotic applications.
Metadata only
Exploration with intrinsic motivation using object–action–outcome latent space
(IEEE, 2023-06) Sener, M. İ.; Nagai, Y.; Öztop, Erhan; Uğur, E.; Computer Science; ÖZTOP, Erhan
One effective approach for equipping artificial agents with sensorimotor skills is to use self-exploration. To do this efficiently is critical, as time and data collection are costly. In this study, we propose an exploration mechanism that blends action, object, and action outcome representations into a latent space, where local regions are formed to host forward model learning. The agent uses intrinsic motivation to select the forward model with the highest learning progress to adopt at a given exploration step. This parallels how infants learn, as high learning progress indicates that the learning problem is neither too easy nor too difficult in the selected region. The proposed approach is validated with a simulated robot in a table-top environment. The simulation scene comprises a robot and various objects, where the robot interacts with one of them each time using a set of parameterized actions and learns the outcomes of these interactions. With the proposed approach, the robot organizes its curriculum of learning as in existing intrinsic motivation approaches and outperforms them in learning speed. Moreover, the learning regime demonstrates features that partially match infant development; in particular, the proposed system learns to predict the outcomes of different skills in a staged manner.
Metadata only
High-level features for resource economy and fast learning in skill transfer
(Taylor & Francis) Ahmetoglu, A.; Uğur, E.; Asada, M.; Öztop, Erhan; Computer Science; ÖZTOP, Erhan
Abstraction is an important aspect of intelligence which enables agents to construct robust representations for effective and efficient decision making. Although, deep neural networks are proven to be effective learning systems due to their ability to form increasingly complex abstractions at successive layers these abstractions are mostly distributed over many neurons, making the re-use of a learned skill costly and blind to the insights that can be obtained on the emergent representations. For avoiding designer bias and unsparing resource use, we propose to exploit neural response dynamics to form compact representations to use in skill transfer. For this, we consider two competing methods based on (1) maximum information compression principle and (2) the notion that abstract events tend to generate slowly changing signals, and apply them to the neural signals generated during task execution. To be concrete, in our simulation experiments, we either apply principal component analysis (PCA) or slow feature analysis (SFA) on the signals collected from the last hidden layer of a deep neural network while it performs a source task, and use these features for skill transfer in a new, target, task. We then compare the generalization and learning performance of these alternatives with the baselines of skill transfer with full layer output and no-transfer settings. Our experimental results on a simulated tabletop robot arm navigation task show that units that are created with SFA are the most successful for skill transfer. SFA as well as PCA, incur less resources compared to usual skill transfer where full layer outputs are used in the new task learning, whereby many units formed show a localized response reflecting end-effector-obstacle-goal relations. Finally, SFA units with the lowest eigenvalues resemble symbolic representations that highly correlate with high-level features such as joint angles and end-effector position which might be thought of as precursors for fully symbolic systems.
Metadata only
High-level representations through unconstrained sensorimotor learning
(IEEE, 2020-10-26) Öztürkçü, Özgür Baran; Uğur, E.; Öztop, E.; Öztürkçü, Özgür Baran
How the sensorimotor experience of an agent can be organized into abstract symbol-like structures to enable effective planning and control is an open question. In the literature, there are many studies that start by assuming the existence of some symbols and 'ground' those onto continuous sensorimotor signals. There are also works that aim to facilitate the emergence of symbol-like representations by using specially designed machine learning architectures. In this paper, we investigate whether a deep reinforcement learning system that learns a dynamic task would facilitate the formation of high-level neural representations that might be considered as precursors of symbolic representation, which could be exploited by higher level neural circuits for better control and planning. The results indicate that without even explicit design to promote such representations, neural responses emerge that may serve as the basis of abstract symbol-like representations.
Metadata only
Learning system dynamics via deep recurrent and conditional neural systems
(IEEE, 2021) Pekmezci, Mehmet; Uğur, E.; Öztop, Erhan; Computer Science; ÖZTOP, Erhan; Pekmezci, Mehmet
Although there are various mathematical methods for modeling system dynamics, more general solutions can be achieved using deep learning based on data. Alternative deep learning methods are presented in parallel with the improvements in artificial neural networks. In this study, both LSTM-based recurrent deep learning method and CNMP-based conditional deep learning method were used to learn the system dynamics of the selected system using time series data. The effects of the amount of time series data needed for training and the initial input length needed for predictions made using the learned system model on both methods were analyzed.
Metadata only
Self-discovery of motor primitives and learning grasp affordances
(IEEE, 2012) Uğur, E.; Şahin, E.; Öztop, Erhan; Computer Science; ÖZTOP, Erhan
Human infants practice their initial, seemingly random arm movements for transforming them into voluntary reaching and grasping actions. With the developing perceptual abilities, infants further explore their environment using the behavior repertoire they have developed, and learn causality relations in the form of affordances, which they use for goal satisfaction and motor planning. This study proposes and implements a developmental progression on a robotic system mimicking the aforementioned infant development stages: An anthropomorphic robot hand with one basic action of swing-hand and the palmar reflex (i.e. the enclosure of the fingers upon contact) at its disposal, executes swing-hand action targeted to a salient object with different hand speeds. During the executions, it monitors the changes in its sensors, automatically forming behavior primitives such as `grasp', `hit', `carry-object' and `drop' by segmenting and differentiating the initial swing-hand action. The study then focuses on one of these behaviors, namely grasping, and shows how further practice allows the robot to learn affordances of more complex objects, which can be further used to make plans to achieve desired goals using the discovered behavior repertoire.
Open Access
Symbol emergence in cognitive developmental systems: A survey
(IEEE, 2019-12) Taniguchi, T.; Uğur, E.; Hoffmann, M.; Jamone, L.; Nagai, T.; Rosman, B.; Matsuka, T.; Iwahashi, N.; Öztop, Erhan; Piater, J.; Worgotter, F.; Computer Science; ÖZTOP, Erhan
Humans use signs, e.g., sentences in a spoken language, for communication and thought. Hence, symbol systems like language are crucial for our communication with other agents and adaptation to our real-world environment. The symbol systems we use in our human society adaptively and dynamically change over time. In the context of artificial intelligence (AI) and cognitive systems, the symbol grounding problem has been regarded as one of the central problems related to symbols. However, the symbol grounding problem was originally posed to connect symbolic AI and sensorimotor information and did not consider many interdisciplinary phenomena in human communication and dynamic symbol systems in our society, which semiotics considered. In this paper, we focus on the symbol emergence problem, addressing not only cognitive dynamics but also the dynamics of symbol systems in society, rather than the symbol grounding problem. We first introduce the notion of a symbol in semiotics from the humanities, to leave the very narrow idea of symbols in symbolic AI. Furthermore, over the years, it became more and more clear that symbol emergence has to be regarded as a multifaceted problem. Therefore, second, we review the history of the symbol emergence problem in different fields, including both biological and artificial systems, showing their mutual relations. We summarize the discussion and provide an integrative viewpoint and comprehensive overview of symbol emergence in cognitive systems. Additionally, we describe the challenges facing the creation of cognitive systems that can be part of symbol emergence systems.