Subgoal Discovery and Language Learning in Reinforcement Learning Agents
University of Maryland
April 9, 2015
Bailey 201
Lunch will be served at noon.


As intelligent agents and robots become more commonly used, methods to make interaction with the agents more accessible will become increasingly important. In this talk, I will present a system for intelligent agents to learn task descriptions from linguistically annotated demonstrations, using a reinforcement learning framework based on object-oriented Markov decision processes (OO-MDPs). Our framework learns how to ground natural language commands into reward functions, using as input demonstrations of different tasks being carried out in the environment. Because language is grounded to reward functions, rather than being directly tied to the actions that the agent can perform, commands can be high-level and can be carried out autonomously in novel environments. Our approach has been empirically validated in a simulated environment with both expert-created natural language commands and commands gathered from a user study.

I will also describe a related, ongoing project to develop novel option discovery methods for OO-MDP domains. These methods permit agents to identify new subgoals in complex environments that can be transferred to new tasks. We have developed a framework called Portable Option Discovery (POD), which uses novel abstraction methods to extend several existing option discovery approaches (PolicyBlocks and Transfer Options (TOPs)) to OO-MDPs.

This work is collaborative research with Dr. Michael Littman and Dr. James MacGlashan of Brown University, Dr. Smaranda Muresan of Columbia University. A number of UMBC students have contributed to the project: Shawn Squire, Nicholay Topin, Nick Haltemeyer, John Winder, Tenji Tembo, Michael Bishoff, Rose Carignan, and Nathaniel Lam.

Dr. Marie desJardins is currently an American Council of Education Fellow, spending the spring semester at Worcester Polytechnic Institute studying academic leadership, institutional transformation, and data-driven decision making. She is a Professor in the Department of Computer Science and Electrical Engineering at the University of Maryland, Baltimore County, and was recently appointed as Associate Dean for Academic Affairs in UMBC's College of Engineering and Information Technology. She is the 2014-17 UMBC Presidential Teaching Professor and an inaugural Hrabowski Academic Innovation Fellow. Her research is in artificial intelligence, focusing on the areas of machine learning, multi-agent systems, planning, interactive AI techniques, information management, reasoning with uncertainty, and decision theory. Current research projects include learning in the context of planning and decision making, analyzing and visualizing uncertainty in machine learning, trust modeling in multiagent systems, and computer science education.

Dr. desJardins has published over 120 scientific papers in journals, conferences, and workshops. She is an Associate Editor of the Journal of Artificial Intelligence Research, is a member of the editorial board of AI Magazine, and was the Program Cochair for AAAI-13. She has previously served as AAAI Liaison to the Board of Directors of the Computing Research Association, Vice-Chair of ACM's SIGART, and AAAI Councillor. She is an ACM Distinguished Member, is a AAAI Senior Member, is a member and former chair of UMBC's Honors College Advisory Board, is the former chair of UMBC's Faculty Affairs Committee, and serves on the advisory board of UMBC's Center for Women in Technology.

Please email Kristina Striegnitz ( if you have any questions concerning the seminar series or if you would like to receive the seminar announcements by email.