Browsing by Author "Ugur, E."

Now showing 1 - 12 of 12

Open Access
Affordance-based altruistic robotic architecture for human–robot collaboration
(Sage, 2019-08) Imre, M.; Öztop, Erhan; Nagai, Y.; Ugur, E.; Computer Science; ÖZTOP, Erhan
This article proposes a computational model for altruistic behavior, shows its implementation on a physical robot, and presents the results of human-robot interaction experiments conducted with the implemented system. Inspired from the sensorimotor mechanisms of the primate brain, object affordances are utilized for both intention estimation and action execution, in particular, to generate altruistic behavior. At the core of the model is the notion that sensorimotor systems developed for movement generation can be used to process the visual stimuli generated by actions of the others, infer the goals behind, and take the necessary actions to help achieving these goals, potentially leading to the emergence of altruistic behavior. Therefore, we argue that altruistic behavior is not necessarily a consequence of deliberate cognitive processing but may emerge through basic sensorimotor processes such as error minimization, that is, minimizing the difference between the observed and expected outcomes. In the model, affordances also play a key role by constraining the possible set of actions that an observed actor might be engaged in, enabling a fast and accurate intention inference. The model components are implemented on an upper-body humanoid robot. A set of experiments are conducted validating the workings of the components of the model, such as affordance extraction and task execution. Significantly, to assess how human partners interact with our altruistic model deployed robot, extensive experiments with naive subjects are conducted. Our results indicate that the proposed computational model can explain emergent altruistic behavior in reference to its biological counterpart and moreover engage human partners to exploit this behavior when implemented on an anthropomorphic robot.
Metadata only
Bimanual rope manipulation skill synthesis through context dependent correction policy learning from human demonstration
(IEEE, 2023) Akbulut, B.; Girgin, T.; Mehrabi, Arash; Asada, M.; Ugur, E.; Öztop, Erhan; Computer Science; ÖZTOP, Erhan; Mehrabi, Arash
Learning from demonstration (LfD) with behavior cloning is attractive for its simplicity; however, compounding errors in long and complex skills can be a hindrance. Considering a target skill as a sequence of motor primitives is helpful in this respect. Then the requirement that a motor primitive ends in a state that allows the successful execution of the subsequent primitive must be met. In this study, we focus on this problem by proposing to learn an explicit correction policy when the expected transition state between primitives is not achieved. The correction policy is learned via behavior cloning by the use of Conditional Neural Motor Primitives (CNMPs) that can generate correction trajectories in a context-dependent way. The advantage of the proposed system over learning the complete task as a single action is shown with a table-top setup in simulation, where an object has to be pushed through a corridor in two steps. Then, the applicability of the proposed method to bi-manual knotting in the real world is shown by equipping an upper-body humanoid robot with the skill of making knots over a bar in 3D space.
Open Access
Deepsym: Deep symbol generation and rule learning for planning from unsupervised robot interaction
(AI Access Foundation, 2022) Ahmetoglu, A.; Seker, M. Y.; Piater, J.; Öztop, Erhan; Ugur, E.; Computer Science; ÖZTOP, Erhan
Symbolic planning and reasoning are powerful tools for robots tackling complex tasks. However, the need to manually design the symbols restrict their applicability, especially for robots that are expected to act in open-ended environments. Therefore symbol formation and rule extraction should be considered part of robot learning, which, when done properly, will offer scalability, flexibility, and robustness. Towards this goal, we propose a novel general method that finds action-grounded, discrete object and effect categories and builds probabilistic rules over them for non-trivial action planning. Our robot interacts with objects using an initial action repertoire that is assumed to be acquired earlier and observes the effects it can create in the environment. To form action-grounded object, effect, and relational categories, we employ a binary bottleneck layer in a predictive, deep encoder-decoder network that takes the image of the scene and the action applied as input, and generates the resulting effects in the scene in pixel coordinates. After learning, the binary latent vector represents action-driven object categories based on the interaction experience of the robot. To distill the knowledge represented by the neural network into rules useful for symbolic reasoning, a decision tree is trained to reproduce its decoder function. Probabilistic rules are extracted from the decision paths of the tree and are represented in the Probabilistic Planning Domain Definition Language (PPDDL), allowing off-the-shelf planners to operate on the knowledge extracted from the sensorimotor experience of the robot. The deployment of the proposed approach for a simulated robotic manipulator enabled the discovery of discrete representations of object properties such as 'rollable' and 'insertable'. In turn, the use of these representations as symbols allowed the generation of effective plans for achieving goals, such as building towers of the desired height, demonstrating the effectiveness of the approach for multi-step object manipulation. Finally, we demonstrate that the system is not only restricted to the robotics domain by assessing its applicability to the MNIST 8-puzzle domain in which learned symbols allow for the generation of plans that move the empty tile into any given position.
Metadata only
Developmental scaffolding with large language models
(IEEE, 2023) Çelik, B.; Ahmetoglu, A.; Ugur, E.; Öztop, Erhan; Computer Science; ÖZTOP, Erhan
Exploration and self-observation are key mechanisms of infant sensorimotor development. These processes are further guided by parental scaffolding to accelerate skill and knowledge acquisition. In developmental robotics, this approach has been adopted often by having a human acting as the source of scaffolding. In this study, we investigate whether Large Language Models (LLMs) can act as a scaffolding agent for a robotic system that aims to learn to predict the effects of its actions. To this end, an object manipulation setup is considered where one object can be picked and placed on top of or in the vicinity of another object. The adopted LLM is asked to guide the action selection process through algorithmically generated state descriptions and action selection alternatives in natural language. The simulation experiments that include cubes in this setup show that LLM-guided (GPT3.5-guided) learning yields significantly faster discovery of novel structures compared to random exploration. However, we observed that GPT3.5 fails to effectively guide the robot in generating structures with different affordances such as cubes and spheres. Overall, we conclude that even without fine-tuning, LLMs may serve as a moderate scaffolding agent for improving robot learning, however, they still lack affordance understanding which limits the applicability of the current LLMs in robotic scaffolding tasks.
Metadata only
Effect regulated projection of robot’s action space for production and prediction of manipulation primitives through learning progress and predictability based exploration
(IEEE, 2021-06) Bugur, S.; Öztop, Erhan; Nagai, Y.; Ugur, E.; Computer Science; ÖZTOP, Erhan
In this study, we propose an effective action parameter exploration mechanism that enables efficient discovery of robot actions through interacting with objects in a simulated table-top environment. For this, the robot organizes its action parameter space based on the generated effects in the environment and learns forward models for predicting consequences of its actions. Following the Intrinsic Motivation approach, the robot samples the action parameters from the regions that are expected to yield high learning progress (LP). In addition to the LP-based action sampling, our method uses a novel parameter space organization scheme to form regions that naturally correspond to qualitatively different action classes, which might be also called action primitives. The proposed method enabled the robot to discover a number of lateralized movement primitives and to acquire the capability of prediction the consequences of these primitives. Furthermore our results suggest the reasons behind the earlier development of grasp compared to push action in infants. Finally, our findings show some parallels with data from infant development where correspondence between action production and prediction is observed.
Open Access
Imitation and mirror systems in robots through Deep Modality Blending Networks
(Elsevier, 2022-02) Seker, M. Y.; Ahmetoglu, A.; Nagai, Y.; Asada, M.; Öztop, Erhan; Ugur, E.; Computer Science; ÖZTOP, Erhan
Learning to interact with the environment not only empowers the agent with manipulation capability but also generates information to facilitate building of action understanding and imitation capabilities. This seems to be a strategy adopted by biological systems, in particular primates, as evidenced by the existence of mirror neurons that seem to be involved in multi-modal action understanding. How to benefit from the interaction experience of the robots to enable understanding actions and goals of other agents is still a challenging question. In this study, we propose a novel method, deep modality blending networks (DMBN), that creates a common latent space from multi-modal experience of a robot by blending multi-modal signals with a stochastic weighting mechanism. We show for the first time that deep learning, when combined with a novel modality blending scheme, can facilitate action recognition and produce structures to sustain anatomical and effect-based imitation capabilities. Our proposed system, which is based on conditional neural processes, can be conditioned on any desired sensory/motor value at any time step, and can generate a complete multi-modal trajectory consistent with the desired conditioning in one-shot by querying the network for all the sampled time points in parallel avoiding the accumulation of prediction errors. Based on simulation experiments with an arm-gripper robot and an RGB camera, we showed that DMBN could make accurate predictions about any missing modality (camera or joint angles) given the available ones outperforming recent multimodal variational autoencoder models in terms of long-horizon high-dimensional trajectory predictions. We further showed that given desired images from different perspectives, i.e. images generated by the observation of other robots placed on different sides of the table, our system could generate image and joint angle sequences that correspond to either anatomical or effect-based imitation behavior. To achieve this mirror-like behavior, our system does not perform a pixel-based template matching but rather benefits from and relies on the common latent space constructed by using both joint and image modalities, as shown by additional experiments. Moreover, we showed that mirror learning (in our system) does not only depend on visual experience and cannot be achieved without proprioceptive experience. Our experiments showed that out of ten training scenarios with different initial configurations, the proposed DMBN model could achieve mirror learning in all of the cases where the model that only uses visual information failed in half of them. Overall, the proposed DMBN architecture not only serves as a computational model for sustaining mirror neuron-like capabilities, but also stands as a powerful machine learning architecture for high-dimensional multi-modal temporal data with robust retrieval capabilities operating with partial information in one or multiple modalities.
Metadata only
Inferring cost functions using reward parameter search and policy gradient reinforcement learning
(IEEE, 2021) Arditi, Emir; Kunavar, T.; Ugur, E.; Babic, J.; Öztop, Erhan; Computer Science; ÖZTOP, Erhan
This study focuses on inferring cost functions of obtained movement data using reward parameter search and policy gradient based Reinforcement Learning (RL). The behavior data for this task is obtained through a series of squat-to-stand movements of human participants under dynamic perturbations. The key parameter searched in the cost function is the weight of total torque used in performing the squat-to-stand action. An approximate model is used to learn squat-to-stand movements via a policy gradient method, namely Proximal Policy Optimization(PPO). A behavioral similarity metric based on Center of Mass(COM) is used to find the most likely weight parameter. The stochasticity in the training result of PPO is dealt with multiple runs, and as a result, a reasonable and a stable Inverse Reinforcement Learning(IRL) algorithm is obtained in terms of performance. The results indicate that for some participants, the reward function parameters of the experts were inferred successfully.
Metadata only
Learning to grasp with parental scaffolding
(IEEE, 2011) Ugur, E.; Celikkanat, H.; Şahin, E.; Nagai, Y.; Öztop, Erhan; Computer Science; ÖZTOP, Erhan
Parental scaffolding is an important mechanism utilized by infants during their development. Infants, for example, pay stronger attention to the features of objects highlighted by parents and learn the way of manipulating an object while being supported by parents. In this paper, a robot with the basic ability of reaching for an object, closing fingers and lifting its hand lacks knowledge of which parts of the object affords grasping, and in which hand orientation should the object be grasped. During reach and grasp attempts, the movement of the robot hand is modified by the human caregiver's physical interaction to enable successful grasping. The object regions that the robot fingers contact first are detected and stored as potential graspable object regions along with the trajectory of the hand. In the experiments, we showed that although the human caregiver did not directly show the graspable regions, the robot was able to find regions such as handles of the mugs after its action execution was partially guided by the human. Later, this experience was used to find graspable regions of never seen objects. At the end, the robot was able to grasp objects based on the position of the graspable part and stored action execution trajectories.
Metadata only
Modeling the development of infant imitation using inverse reinforcement learning
(IEEE, 2018-09) Tekden, A. E.; Ugur, E.; Nagai, Y.; Öztop, Erhan; Computer Science; ÖZTOP, Erhan
Little is known about the computational mechanisms of how imitation skills develop along with infant sensorimotor learning. In robotics, there are several well developed frameworks for imitation learning or so called learning by demonstration. Two paradigms dominate: Direct Learning (DL) and Inverse Reinforcement Learning (IRL). The former is a simple mechanism where the observed state and action pairs are associated to construct a copy of the action policy of the demonstrator. In the latter, an optimality principle or reward structure is sought that would explain the observed behavior as the optimal solution governed by the optimality principle or the reward function found. In this study, we explore the plausibility of whether some form of IRL mechanism in infants can facilitate imitation learning and understanding of others' behaviours. We propose that infants project the events taking place in the environment into their internal representations through a set of features that evolve during development. We implement this idea on a grid world environment, which can be considered as a simple model for reaching with obstacle avoidance. The observing infant has to imitate the demonstrator's reaching behavior through IRL by using various set of features that correspond to different stages of development. Our simulation results indicate that the U-shape performance change during imitation development observed in infants can be reproduced with the proposed model.
Metadata only
Parental scaffolding as a bootstrapping mechanism for learning grasp affordances and imitation skills
(Cambridge University Press, 2015-06) Ugur, E.; Nagai, Y.; Celikkanat, H.; Öztop, Erhan; Computer Science; ÖZTOP, Erhan
Parental scaffolding is an important mechanism that speeds up infant sensorimotor development. Infants pay stronger attention to the features of the objects highlighted by parents, and their manipulation skills develop earlier than they would in isolation due to caregivers' support. Parents are known to make modifications in infant-directed actions, which are often called “motionese”. The features that might be associated with motionese are amplification, repetition and simplification in caregivers' movements, which are often accompanied by increased social signalling. In this paper, we extend our previously developed affordances learning framework to enable our hand-arm robot equipped with a range camera to benefit from parental scaffolding and motionese. We first present our results on how parental scaffolding can be used to guide the robot learning and to modify its crude action execution to speed up the learning of complex skills. For this purpose, an interactive human caregiver-infant scenario was realized with our robotic setup. This setup allowed the caregiver's modification of the ongoing reach and grasp movement of the robot via physical interaction. This enabled the caregiver to make the robot grasp the target object, which in turn could be used by the robot to learn the grasping skill. In addition to this, we also show how parental scaffolding can be used in speeding up imitation learning. We present the details of our work that takes the robot beyond simple goal-level imitation, making it a better imitator with the help of motionese.
Open Access
A soft+rigid hybrid exoskeleton concept in scissors-pendulum mode: A suit for human state sensing and an exoskeleton for assistance
(IEEE, 2019-06) Uğurlu, Regaip Barkan; Acer, M.; Barkana, D. E.; Gocek, I.; Kucukyilmaz, A.; Arslan, Y. Z.; Basturk, H.; Samur, E.; Ugur, E.; Ünal, Ramazan; Bebek, Özkan; Mechanical Engineering; UĞURLU, Regaip Barkan; ÜNAL, Ramazan; BEBEK, Özkan
In this paper, we present a novel concept that can enable the human aware control of exoskeletons through the integration of a soft suit and a robotic exoskeleton. Unlike the state-of-the-art exoskeleton controllers which mostly rely on lumped human-robot models, the proposed concept makes use of the independent state measurements concerning the human user and the robot. The ability to observe the human state independently is the key factor in this approach. In order to realize such a system from the hardware point of view, we propose a system integration frame that combines a soft suit for human state measurement and a rigid exoskeleton for human assistance. We identify the technological requirements that are necessary for the realization of such a system with a particular emphasis on soft suit integration. We also propose a template model, named scissor pendulum, that may encapsulate the dominant dynamics of the human-robot combined model to synthesize a controller for human state regulation. A series of simulation experiments were conducted to check the controller performance. As a result, satisfactory human state regulation was attained, adequately confirming that the proposed system could potentially improve exoskeleton-aided applications.
Open Access
Staged development of robot skills: behavior formation, affordance learning and imitation
(IEEE, 2015-06) Ugur, E.; Nagai, Y.; Sahin, E.; Öztop, Erhan; Computer Science; ÖZTOP, Erhan
Inspired by infant development, we propose a three staged developmental framework for an anthropomorphic robot manipulator. In the first stage, the robot is initialized with a basic reach-and- enclose-on-contact movement capability, and discovers a set of behavior primitives by exploring its movement parameter space. In the next stage, the robot exercises the discovered behaviors on different objects, and learns the caused effects; effectively building a library of affordances and associated predictors. Finally, in the third stage, the learned structures and predictors are used to bootstrap complex imitation and action learning with the help of a cooperative tutor. The main contribution of this paper is the realization of an integrated developmental system where the structures emerging from the sensorimotor experience of an interacting real robot are used as the sole building blocks of the subsequent stages that generate increasingly more complex cognitive capabilities. The proposed framework includes a number of common features with infant sensorimotor development. Furthermore, the findings obtained from the self-exploration and motionese guided human-robot interaction experiments allow us to reason about the underlying mechanisms of simple-to-complex sensorimotor skill progression in human infants.