Sudipta Kar
Senior Applied Scientist at Amazon AGI
Sudipta Kar is a Senior Applied Scientist at Amazon AGI. He received his Ph.D. in Computer Science from the University of Houston in 2020 under the supervision of Thamar Solorio. His doctoral research focused on creative text analysis. Currently, he works on developing intelligent systems to enable seamless proactivity in smart voice assistants such as Alexa. His research interests include computational systems for low-resource languages, language models, and information extraction. He has co-organized multiple natural language processing workshops and shared tasks, including BLP, CALCS, MultiCoNER, and SentiMix. Additionally, in 2023 he led the first NLP hackathon held in Bangladesh.
Tutorial title: "The Power of Rewards - Reinforcing Language Models with Reinforcement Learning"
Language models ranging from BERT to GPT have shown impressive performance on many natural language tasks through first self-supervised pre-training on large text corpora and then fine-tuning on downstream tasks or even with zero-shot or few-shot approaches. These models are also very powerful in solving multiple tasks. However, their capabilities are still limited as they lack a built-in reward signal to directly optimize for the end task objective or align to certain human preferences.On the other hand, Reinforcement Learning (RL) is another area in machine learning which is commonly applied to develop systems that improve over real-time feedback loops (such as games). Because it provides a framework for optimizing goal-directed behavior through rewards. In this tutorial, we will explore how reinforcement learning came into the play of language modeling and suddenly changed the game by reinforcing the representational power of large language models with the ability to more efficiently solve tasks requiring reasoning, planning, and knowledge. We will first provide background on contemporary language models and reinforcement learning fundamentals. Next, we will discuss techniques for applying policy gradient methods to fine-tune language models to maximize rewards from an environment. Attendees will learn how to practically apply RL to language tasks, understand tradeoffs between different algorithms, and gain insight into state-of-the-art research in this emerging field.Danae Sánchez
Researcher at the University of Copenhagen
Danae Sánchez Villegas is a postdoctoral researcher at the University of Copenhagen. She holds a Ph.D. and a Master's degree in Computer Science from the University of Sheffield and a Bachelor's degree in Computer Engineering from the Instituto Tecnológico Autónomo de México. Her research interests include multilingual natural language understanding, vision and language modeling, and computational social science. Danae has worked as a Research Associate in the Natural Language Processing Group at the University of Sheffield and as an Applied Scientist Intern at Amazon Alexa.
Tutorial title: "Exploring Transformers and Limitations in Language Modeling"
This tutorial explores language modeling techniques in Natural Language Processing (NLP), covering key concepts from traditional approaches to Transformer architectures. Beginning with an introduction to NLP and language modeling, it delves into probabilistic language models and progresses to neural language models, emphasizing the significance of embeddings for semantic representation. Moving on to Transformer models, we will discuss key concepts such as multi-head attention mechanisms, masked language modeling, and encoder models. Additionally, the tutorial addresses the limitations of large language models, providing insights into challenges and considerations for leveraging these models effectively in practical applications.Alham Fikri Aji
Assistant Professor at MBZUAI, UAE.
Alham Fikri Aji is an Assistant Professor at MBZUAI, holding a Ph.D. from the University of Edinburgh's Institute for Language, Cognition, and Computation. His doctoral research, supervised by Dr. Kenneth Heafield and Dr. Rico Sennrich, focused on enhancing the training and inference speed of machine translation. Dr. Aji's current research centers around multilingual, low-resource, and low-compute Natural Language Processing (NLP). His recent work has been in developing diverse multilingual large language models. and multilingual NLP resources, particularly for underrepresented languages, with a specific emphasis on Indonesian. He has worked at Amazon, Google, and Apple in the past.
Tutorial title: "Training Lightweight Model via Knowledge Distillation and Parameter Efficient Finetuning"
Language models can be resource-hungry and prohibitive to train and deploy for many people due to a lack of proper computing resources. In this tutorial, we delve into how to build smaller, more lightweight models without sacrificing quality via knowledge distillation. Additionally, we delve into training your model with less memory usage through parameter-efficient finetuning. In this tutorial, we will delve into the hands-on implementation. It is ideal for those just delving into the field of AI/NLP.Víctor Mijangos
Professor at UNAM, Mexico.
Víctor Mijangos de la Cruz is a Full-Time Professor at the Faculty of Sciences, where he teaches subjects related to Artificial Intelligence, Neural Networks, and Computational Linguistics. In addition to teaching, he develops research projects aimed at exploring inductive biases in neural networks and the application of deep learning models for the development of technologies in indigenous languages. His interests lie in the study of the capabilities and limitations of deep learning, computational linguistics, and the creation of technologies for indigenous languages, particularly in the Otomí language.
Tutorial title: "Introduction to Attention Mechanisms in Transformers"
Attention layers are currently central mechanisms in language models. Transformers, which represent the state of the art in this field, rely on the use of attention layers in combination with other strategies. Attention has also been used in models based on sequence-to-sequence recurrent networks, providing significant improvements in natural language processing tasks such as machine translation and text generation. Understanding how these mechanisms work is essential to comprehend current language models. This workshop aims to present a first approach to the attention mechanisms used in neural networks. Firstly, the basic theoretical concepts to understand attention and its operation will be presented, other attention mechanisms, mainly sparse attention, will be reviewed, and the relationship of attention with auto-encoded and auto-regressive language models will be discussed. Finally, its relationship with other mechanisms such as convolutional layers and graph layers, highlighting their advantages and disadvantages, will be addressed. Secondly, the technical principles for the implementation of attention mechanisms in Pytorch and their incorporation within the architecture of Transformers will be covered.Diyi Yang
Assistant Professor at Stanford
Diyi Yang is an assistant professor in the Computer Science Department at Stanford University, also affiliated with the Stanford NLP Group, Stanford HCI Group and Stanford Human Centered AI Institute. Her research focuses on human-centered natural language processing and computational social science. She is a recipient of IEEE “AI 10 to Watch” (2020), Microsoft Research Faculty Fellowship (2021), NSF CAREER Award (2022), an ONR Young Investigator Award (2023), and a Sloan Research Fellowship (2024). Her work has received multiple paper awards or nominations at top NLP and HCI conferences, (e.g., Best Paper Honorable Mention at ICWSM 2016, Best Paper Honorable Mention at SIGCHI 2019, and Outstanding Paper at ACL 2022).
Keynote: "Human Centered NLP for Positive Impact"
Large language models have revolutionized the way humans interact with AI systems, transforming a wide range of fields and disciplines. However, there is a growing amount of evidence and concern about the negative aspects of NLP systems such as biases and the lack of input from users. How can we build NLP systems that are more aware of human factors? In this talk, we will present two case studies on how human-centered design can be leveraged to build responsible NLP applications. The first one presents a participatory design approach to develop dialect-inclusive language tools and adaptation techniques for low-resourced language and dialect. The second part looks at social skill training with LLMs by demonstrating how we use LLMs to teach conflict resolution skills through simulated practice. We conclude by discussing how human-AI interaction via LLMs can empower individuals and foster positive change.Veronica Perez Rosas
Researcher at the University of Michigan
Veronica Perez Rosas is an Assistant Research Scientist at the University of Michigan. Her research interests include Natural Language Processing, Machine Learning, Affect Recognition, and Multimodal Processing of Human Behavior. Her research focuses on developing computational methods to analyze, recognize, and predict human behaviors during social interactions. She has authored papers in leading conferences and journals in Natural Language Processing and Multimodal Processing, has mentored numerous students in these research areas, and has served as workshop chair or area chair for multiple international conferences in the field.
Keynote: "NLP for Timely and Actionable Feedback in Healthcare Conversations"
Effective communication is essential in mental healthcare, as it influences patient responses, treatment decisions, and outcomes. In this presentation, I'll discuss computational methods aimed at helping clinicians enhance their communication skills during patient interactions, with a particular focus on feedback and coaching in counseling. Firstly, I'll explore the generation of language cues to assist counselors in crafting responses that align with counseling guidelines, using language models augmented with context and common sense. Secondly, I will introduce language-based coaching systems that use contrastive learning and language models for response scoring and rewriting. These tools aim to empower clinicians to refine their communication skills by providing detailed feedback and enabling them to adapt their style to better suit patients' needs. Finally, I'll share a case study demonstrating the application of these tools in educational settings, highlighting the potential of AI in mental healthcare.Alexis Palmer
Assistant Professor at the University of Colorado Boulder
Alexis Palmer is an assistant professor in Linguistics at CU Boulder, a fellow of the ICS, and affiliated faculty with the department of Computer Science. Her primary research focus is on computational linguistics for low-resource and endangered languages. She also works in computational semantics, computational discourse, and computational analysis of toxic language. Her current work includes incorporating linguistic insights into cross-lingual transfer learning in order to more rapidly build tools to support low-resource and endangered languages. She is a member at large of the ACL Special Interest Group on Typology and Multilingual NLP, and a founding member of the ACL Special Interest Group on Endangered Languages. Alexis grew up in Northern Michigan, did her PhD at the University of Texas in Austin, and made her way to Boulder by way of several years in German universities and research institutions, as well as the University of North Texas.
Keynote: "A tricky intersection: Natural language processing and endangered language documentation"
Of the roughly 7000 languages spoken across the globe, more than half have been categorized as endangered, threatened, or of decreasing vitality. Only a handful of the world’s languages have anywhere near the amount of digitally-available data needed to train large language models such as ChatGPT, leading to a dramatic disparity in access to cutting-edge technologies. At the same time, for several decades now, linguists, language activists, language community members, and other researchers have recognized language endangerment as a pressing issue and have devoted considerable time to the incredibly time-consuming work of language documentation and revitalization. And yet, as a field, we have not yet realized the potential of natural language processing (NLP) technologies to support language documentation, description, and revitalization. In this talk, I will discuss some of the structural barriers inherent to the intersection of NLP and endangered language documentation, as well as some important ethical considerations that arise at this intersection. I will also present our work on developing models to partially automate the complex linguistic analysis work involved with producing interlinear glossed text, an important format in language documentation.Pablo Rivas
Assistant Professor at Baylor University
Pablo Rivas is an Assistant Professor of Computer Science at Baylor University, where he directs the NSF IUCRC Center for Standards and Ethics in Artificial Intelligence (planning). He holds a Ph.D. in Electrical and Computer Engineering from the University of Texas at El Paso and completed his masters and undergraduate studies at the Tecnologico Nacional de Mexico. His research primarily focuses on natural language processing (NLP), responsible AI, and machine learning. Rivas' recent contributions to the field of NLP include aligning word embeddings from BERT to vocabulary-free representations and developing automated assessment systems for SQL statements. His work also extends to combating human trafficking through NLP by analyzing online advertisements. A Senior Member of IEEE, ACM, and SIAM, Rivas is committed to advancing responsible AI practices and has been recognized with numerous awards and honors for his contributions.
Keynote: "Combatting Human Trafficking in the Cyberspace: An NLP-Based Methodology to Analyze Online Advertisements"
Human trafficking is a grave issue proliferating in online marketplaces. This keynote will discuss an advanced NLP methodology designed to combat this problem by analyzing the language used in online advertisements. The approach leverages pseudo-labeled datasets and minimal supervision to train state-of-the-art NLP models, focusing on Human Trafficking Risk Prediction (HTRP) and Organized Activity Detection (OAD). Cutting-edge transformer models and an interpretability framework based on integrated gradients are utilized to provide explainable insights crucial for law enforcement. This work, funded by an NSF multidisciplinary award, addresses a critical gap in the literature and offers a scalable, machine learning-driven solution to combat online human trafficking. The keynote will explore the intersection of NLP and social good, demonstrating how technology can be pivotal in addressing complex social challenges.From "publish or perish" to "publicize or perish": Navigating challenges and opportunities for career advancement in Latin American communities in the current era of scientific publication trends.
Diyi Yang | Assistant Professor at Stanford
Diyi Yang is an assistant professor in the Computer Science Department at Stanford University, also affiliated with the Stanford NLP Group, Stanford HCI Group and Stanford Human Centered AI Institute. Her research focuses on human-centered natural language processing and computational social science. She is a recipient of IEEE “AI 10 to Watch” (2020), Microsoft Research Faculty Fellowship (2021), NSF CAREER Award (2022), an ONR Young Investigator Award (2023), and a Sloan Research Fellowship (2024). Her work has received multiple paper awards or nominations at top NLP and HCI conferences, (e.g., Best Paper Honorable Mention at ICWSM 2016, Best Paper Honorable Mention at SIGCHI 2019, and Outstanding Paper at ACL 2022).
Manuel Montes y Gomez | Full Professor at the National Institute of Astrophysics, Optics and Electronics (INAOE) of Mexico
Manuel Montes-y-Gómez is Full Professor at the National Institute of Astrophysics, Optics and Electronics (INAOE) of Mexico. His research is on automatic text processing. He is author of more than 250 journal and conference papers in the fields of information retrieval, text mining and authorship analysis.
He has been visiting professor at the Polytechnic University of Valencia (Spain), and the University of Alabama (USA). He is also a member of the Mexican Academy of Sciences (AMC), and founding member of the Mexican Academy of Computer Science (AMEXCOMP), the Mexican Association of Natural Language Processing (AMNLP), and of the Language Technology Network of CONACYT. In the context of them, he has been the organizer of the National Workshop on Language Technologies (from 2004 to 2016), the Mexican Workshop on Plagiarism Detection and Authorship Analysis (2016-2020), the Mexican Autumn School on Language Technologies (2015 and 2016), and a shared task on author profiling, aggressiveness analysis and fake news detection in Mexican Spanish at IberLEF (2018-2021).
Manuel Mager | Applied Scientist at Amazon AWS
Manuel Mager is an Applied Scientist at AWS Bedrock. He did his Ph.D. at the University of Stuttgart, Germany, graduated in informatics from the National Autonomous University of Mexico (UNAM) and did a Master's in Computer Science at the Metropolitan Autonomous University, Mexico (UAM). His research is focused on Natural Language Processing for low resource languages, mainly indigenous languages of the American continent that are polysynthetic. He also worked on Graph-to-text generation and information extraction, and LLM alignment. He is currently working on responsible AI..
Perspectivas de NLP desde Latinoamérica / NLP perspectives, an overview from Latin America.
Luciana Benotti | Associate Professor at the National University of Córdoba and AI Researcher at CONICET, Argentina.
Luciana Benotti is an Associate Professor in Computer Science at the National University of Córdoba and a Researcher in Artificial Intelligence at CONICET, Argentina. Her research interests include different aspects of situated and interactive NLP, such as interpreting instructions in a dialogue, generating contextualized questions, and deciding when to speak in a dialogue system, among others. She is particularly interested in how linguistic and non-linguistic features contribute to the meaning conveyed during a conversation. These features include what the conversational participants are doing while they talk, the visual context, temporal aspects, etc. She has been a visiting scientist at the University of Trento (2019), Stanford University (2018), Roskilde University (2014), the University of Costa Rica (2012), and the University of Southern California (2010). She holds a joint MSc Erasmus Mundus from the Free University of Bolzano and the Polytechnic University of Madrid and a PhD from the Université de Lorraine. This year, she was chosen as the Latin American representative for the North American Association for Computational Linguistics (NAACL).
Helena Gomez-Adorno | Associate Professor at IIMAS-UNAM, Mexico.
Helena Gómez Adorno is a professor of Artificial Intelligence at IIMAS-UNAM. Among her research interests areas are natural language processing, computational linguistics, and information retrieval; specifically, question answering, semantic similarity, authorship attribution, and author profiling. Lead a research and development team to implement intelligent systems with the help of machine learning and neural networks.
Jocelyn Dunstan | Assistant Professor at the Pontifical Catholic University of Chile.
Jocelyn Dunstan is an Assistant Professor at the Pontifical Catholic University of Chile. She holds a Ph.D. in Applied Mathematics and Theoretical Physics from the University of Cambridge in the UK. She specializes in leveraging machine learning and natural language processing to address key challenges. Her research primarily revolves around clinical text mining and patient prioritization. In addition to her academic role at the Catholic University of Chile, she is actively engaged as a researcher at prominent institutions such as the Millenium Institute for Foundational Research on Data (IMFD) and the Advanced Center for Electrical and Electronic Engineering (AC3E). Further information about her group's work can be found on their webpage at pln.cmm.uchile.cl.
Aidé Paola Ríos Rivero | Consultant at Deloitte
Aidé Ríos Rivero studied Linguistics at the National Autonomous University of Mexico. She currently works as a consultant at Deloitte in the Conversational AI area. Aide has worked on several linguistic projects such as transcription, analysis of lexical and argument structure of verbs, annotation of automatic translation errors, research into dialectal variants of Spanish, etc. In recent years, she has worked mainly on the design, deployment, and improvement of voice portals for telephony, banking, and finance companies Her main interests are Natural Language Processing and Conversational Design.
Getting into NLP: Insights from experts and peers.
Alexis Palmer | Assistant Professor at the University of Colorado Boulder
Alexis Palmer is an assistant professor in Linguistics at CU Boulder, a fellow of the ICS, and affiliated faculty with the department of Computer Science. Her primary research focus is on computational linguistics for low-resource and endangered languages. She also works in computational semantics, computational discourse, and computational analysis of toxic language. Her current work includes incorporating linguistic insights into cross-lingual transfer learning in order to more rapidly build tools to support low-resource and endangered languages. She is a member at large of the ACL Special Interest Group on Typology and Multilingual NLP, and a founding member of the ACL Special Interest Group on Endangered Languages. Alexis grew up in Northern Michigan, did her PhD at the University of Texas in Austin, and made her way to Boulder by way of several years in German universities and research institutions, as well as the University of North Texas.
Roma Patel | Research Scientist at Google DeepMind
Roma Patel is a Senior Research Scientist at Deepmind, where her research focuses on grounded language learning i.e., teaching agents to understand and use language for more intelligent behaviour, interpretability (mechanistically understanding how language models work) and safety (building more inclusive, safer and less biased language models). Given the interdisciplinary nature of this research agenda, Roma's research has spanned the fields of robotics, multi-agent reinforcement learning and natural language processing. She finished her PhD at Brown University in 2018 advised by Ellie Pavlick.
Emilio Villa-Cueva | Master’s student at CIMAT, Mexico.
Emilio Villa-Cueva holds a degree in Engineering Physics from the University of Guanajuato. Currently, he is a master’s student in computer science at CIMAT, specializing in Natural Language Processing. He has previously worked on domain adaptation, question-answering, few-shot learning, and cross-lingual transfer. His current interests are multilingual LLMs and their applications.
Selene Báez Santamaría | PhD candidate at Vrije Universiteit Amsterdam
Selene Báez Santamaría is a PhD candiate at Vrije Universiteit Amsterdam. Her research interests are natural language processing, and knowledge representation, focusing on the design of conversational social agents.
Horacio Jarquin | PhD candidate at INAOE, Mexico.
Horacio Jarquin is a PhD student at the National Institute of Astrophysics, Optics and Electronics, (INAOE). He holds the a bachelor degree from the Universidad Popular Autónoma del Estado de Puebla in 2018 a MSc degree from INAOE in 2020. His research interests are on machine learning and natural language processing, with emphasis in supervised classification; with particular interest on the development and application of deep learning approaches for vision and language representation.