BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//TU/e ME//Events//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
BEGIN:VTIMEZONE
TZID:Europe/Amsterdam
BEGIN:STANDARD
DTSTART:19701025T030000
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:19700329T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
UID:event-72@tuemeche.nl
DTSTAMP:20261007T234653Z
DTSTART;TZID=Europe/Amsterdam:20261215T133000
DTEND;TZID=Europe/Amsterdam:20261215T150000
SUMMARY:Human-Guided Robot Task and Motion Adaptation through Demonstrati
 ons\, Language\, and Physical Corrections
DESCRIPTION:Speaker: Busra Sen\nHost: René van de Molengraft\n\nRobots a
 re increasingly deployed beyond traditional industrial settings\, extendi
 ng their use to homes\, hospitals\, agriculture\, and other everyday envi
 ronments. In these applications\, robots must perform diverse tasks under
  varying conditions while accounting for user preferences and expectation
 s. Traditional robot programming is effective for repetitive tasks\, but 
 it may be insufficient when robot behavior must adapt to end-user needs a
 nd safety concerns. This motivates intuitive human-robot interaction meth
 ods that allow users to guide\, correct\, and personalize robot behavior 
 without programming expertise.\n\nThe first part of this thesis addresses
  this need through learning from demonstration (LfD)\, which enables non-
 expert users to teach robot tasks without explicit programming. While man
 y LfD approaches focus on reproducing demonstrated motions or task sequen
 ces\, demonstrations can also implicitly convey why particular objects ar
 e selected. To capture this intention\, we propose a keyframe-based metho
 d that infers task goals from object-attribute constraints. During demons
 trations\, the system records the robot state\, object poses\, and object
  attributes\, and identifies object-centric keyframes based on meaningful
  environmental changes. It then extracts constraints over discrete and co
 ntinuous object attributes across and within demonstrations. These constr
 aints represent the semantic rationale behind the user’s object choices
  and are used during reproduction to select suitable objects in new envir
 onments. In this way\, the robot can generalize demonstrated tasks to nov
 el scenes and unseen objects while preserving the user’s intended objec
 t-level semantics.\n\nAlthough demonstrations can teach the robot what ta
 sk to perform\, the resulting motion may still conflict with the user’s
  preferences or safety expectations. Natural language offers an intuitive
  way to correct robot motion\, such as asking the robot to move slower or
  keep a larger distance from a fragile item. However\, existing low-level
  correction methods often treat feedback as environment-specific and main
 ly focus on unconditional corrections that specify only the desired motio
 n change. Some corrections are valid only under specific conditions\; ign
 oring these conditions may cause unnecessary detours or overly conservati
 ve behavior. Conditional corrections therefore introduce an additional ch
 allenge because they specify both what the robot should do and when the c
 orrection should be applied\, as in “stay away from the open book while
  carrying a full cup.” The second part of this thesis therefore represe
 nts such corrections as reusable user preferences. A fine-tuned language 
 model converts each correction into a structured preference\, which is st
 ored in memory and later selected based on the current scene and robot st
 ate. The selected preferences are activated over relevant trajectory segm
 ents and converted into constraints to generate corrected motion in new e
 nvironments.\n\nWhile natural language corrections are semantically infor
 mative\, they may lack precise physical grounding about the desired motio
 n change. In contrast\, physical corrections provide direct geometric inf
 ormation but can be semantically ambiguous\, as the same intervention may
  reflect different intentions. The third part of this thesis exploits the
 se complementary modalities to construct and refine structured task-and-m
 otion plans. Verbal cues clarify the semantic target of the correction\, 
 while physical interaction grounds the correction in the robot’s motion
 \, allowing the model to infer the corresponding update to the task-and-m
 otion plan. The plan is represented as a sequence of task-level objective
 s\, such as grasping or placing an object\, together with motion-level pr
 eferences between consecutive objectives. A transformer-based model condi
 tions the current multimodal feedback on the previous plan to support bot
 h plan construction and incremental revision. We compare two architecture
 s that differ in how the previous plan is incorporated.\n\nMore info: htt
 ps://research.tue.nl/en/persons/busra-sen/
LOCATION:Atlas 0.710
URL:https://tuemeche.nl/peoplepages/event.php?id=72
CATEGORIES:PhD Defense
END:VEVENT
END:VCALENDAR