BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Computer Science and Engineering - ECPv6.13.0//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:Computer Science and Engineering
X-ORIGINAL-URL:https://homecse.iitd.ac.in
X-WR-CALDESC:Events for Computer Science and Engineering
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:Asia/Kolkata
BEGIN:STANDARD
TZOFFSETFROM:+0530
TZOFFSETTO:+0530
TZNAME:IST
DTSTART:20260101T000000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=Asia/Kolkata:20260520T160000
DTEND;TZID=Asia/Kolkata:20260520T170000
DTSTAMP:20261010T121545
CREATED:20260513T060719Z
LAST-MODIFIED:20260513T060719Z
UID:2520-1779292800-1779296400@homecse.iitd.ac.in
SUMMARY:Algorithmic Behaviours in In-Context Learning by Dr. Aditya Gangrade
DESCRIPTION:Venue: Bharti-501/ MS Teams \nAbstract: In-Context Learning (ICL) is a remarkable phenomenon whereby transformer-based LLMs can use data contained within their prompts to adapt their responses\, without changing their weights. This suggests that such models encode learning mechanisms. The recent literature has used statistical learning problems as a test-bed to investigate ICL\, and established that ICL can be realised for a wide range of function classes. However\, the mechanisms these models use to learn are poorly characterised. \nI will describe work on extracting and analysing learning algorithms embedded in the weights of transformers trained to perform ICL in two settings: linear-activation transformers for linear regression\, and softmax-activation transformers for linear classification. Through the former\, I will illustrate a high-level ‘simplify and validate’ strategy that allows extraction\, and through the latter\, I will describe a symmetry-driven strategy for evoking structure in these weights. In these settings\, we recover concrete iterative procedures that use existing ideas (Newton-Schulz; mean-shift methods) in new ways that are distinct from gradient descent. Further\, we show that transformers trained on variations of these problems implement modified versions of the same dynamics. This suggests that such models recover certain `stable’ algorithmic motifs\, and adapt them in response to problem structure. \nBased on work done jointly with Patrick Lutz\, Themistoklis Haris\, Arjun Chandra\, Hadi Daneshmand\, and Venkatesh Saligrama. \nBio:  Aditya Gangrade is a research scientist at the ECE department at Boston University. He obtained his Ph. D. in Systems Engineering from Boston University\, and previously held postdoctoral positions at Carnegie Mellon University and the University of Michigan. His research interests span theoretical and methodological aspects of machine learning\, with recent focus on safety in sequential decision making\, and on in-context learning phenomena.
URL:https://homecse.iitd.ac.in/event/algorithmic-behaviours-in-in-context-learning-by-dr-aditya-gangrade/
LOCATION:Bharti 501\, IIT Campus\, Hauz Khas\, New Delhi
END:VEVENT
END:VCALENDAR