|
NLP 280: Seminar in Natural Language Processing |
|
|
|
|
|
About |
|
NLP 280 is a seminar course that features talks from industry experts in the natural language processing (NLP) and artificial intelligence (AI) areas. The speaker schedule may change without notice, due to changes in speaker availability. Titles, abstracts, and speaker bios will be made available as the talk date approaches. Some seminar slots do not have a speaker. Instead, the seminar time will be used for discussion. Unless noted otherwise, the seminar meets weekly on Friday at 2:40 PM.
|
|
Seminar Schedule |
|
Date: 9/24/2021 Time: 2:40 PM PDT Title: Orientation
Date: 10/1/2021 Time: 2:40 PM PDT Speaker: Zornitsa Kozareva Affiliation: Facebook AI Title: On-device Neural Conversational AI and Natural Language Understanding Abstract: Deep neural networks reach state-of-the-art performance for a wide range of Natural Language Understanding, Conversational AI, Computer Vision and Speech applications. Yet, one of the biggest challenges is to run these complex networks on devices with tiny memory footprint and low computational capacity such as mobile phones, smart watches and Internet of Things. In this talk, I will introduce novel on-device Self-Governing Neural Networks (SGNNs), which learn compact projection vectors with local sensitive hashing. The key advantage of SGNNs is that they surmount the need for pre-trained word embeddings and complex networks with huge parameters. Then, I will dive into SGNN++ networks, which can further improve SGNN via structured and context partitioned projections. In a series of Conversational AI intent prediction evaluations and text classification tasks, we will see how the impact of the partitioned projects and structured information leads to 10% quality improvement. I will also discuss the impact of the model size on accuracy and introduce quantization-aware training to further reduce the model size. Finally, I will introduce a new on-device neural sequence labeling model SoDA, which uses embedding-free projections and character information to construct compact word representations to learn a sequence model using a combination of bidirectional LSTM with self-attention and CRF. Unlike typical dialog models that rely on huge, complex neural network architectures and large-scale pre-trained Transformers to achieve state-of-the-art results, our method achieves comparable results to BERT and even outperforms its smaller variant DistilBERT on conversational slot extraction tasks. Our SoDA method is faster than BERT models while achieving significant model size reduction–our model SoDA requires 135x and 81x fewer model parameters than BERT and DistilBERT, respectively. Bio: Dr. Zornitsa Kozareva is a Senior Manager at Facebook AI Research leading teams on building the next generation large language models. Prior to that, Dr. Kozareva was at Google leading and managing Search and Intelligence efforts. She led and managed Amazon’s AWS Deep Learning group that built and launched the first Natural Language Processing and Dialog services Amazon Comprehend and Amazon Lex. Dr. Kozareva was a Senior Manager at Yahoo! leading the Query Processing group that powered Mobile Search and Advertisement. From 2009 to 2014, Dr. Kozareva wore an academic hat as a Research Professor at the University of Southern California CS Department with affiliation to Information Sciences Institute, where she spearheaded research funded by DARPA and IARPA on learning to read, interpreting metaphors and building knowledge bases from the Web. Dr. Kozareva regularly serves as Senior Area Chair and PC of top tier NLP and ML conferences such as ACL, NAACL, EMNLP, WSDM, AAAI. She is the Co-Chair for EMNLP 2021 and ICML 2019 On-device ML, and ACL 2021, EMNLP 2020 & NAACL 2019 Structured prediction workshops. Dr. Kozareva has organized four SemEval scientific challenges and has published over 100 research papers and patents. Her work has been featured in the press such as Forbes, VentureBeat, GizBot, NextBigWhat. Dr. Kozareva was invited speaker for 2019 National Academy of Engineering (NAE) German-American Frontiers of Engineering symposium. Dr. Kozareva is a recipient of the John Atanasoff Award given by the President of the Republic of Bulgaria in 2016 for her contributions and impact in science, education and industry; the Yahoo! Labs Excellence Award in 2014 and the RANLP Young Scientist Award in 2011.
Date: 10/8/2021 Time: 2:40 PM PDT Speaker: Radu Florian Affiliation: IBM Title: The Road Towards Language Agnostic Information Extraction Abstract: In this talk I will present my view on the evolution of Information Extraction, in particular mention detection, coreference resolution, relation extraction, and entity linking across multiple languages, going from language specific in CoNLL'02 and CoNLL'03 up until current research that produces models that can process a large set of languages with one engine. If time permits, I will also present some newer experiments that allow a user to take a system in English (for instance) and produce good models in other languages, further enabling true multi-language Information Extraction. Bio: Radu Florian wears two hats as Distinguished Research Scientist and Senior Manager, managing the Multilingual Natural Language Processing Group in IBM Watson Research Center in Yorktown Heights, NY. His research interests include multi-language statistical information extraction, question answering, semantic parsing, and machine learning. He has participated and lead teams in several competitions, including CoNLL information extraction, ACE, TAC-KBP, and DARPA projects such as GALE, BOLT, MRP, and KAIROS. One of the recent focus in Radu's research involves building models that can operate cross-language -- using multilingual language models such as multilingual BERT or XLMRoberta to build information extraction, dependency parsing, and question answering models that can take a wide variety of languages as input and produce good output, even on languages that were not trained on. Not only these models can perform the extraction on any of the language input (unfortunately, not on Klingon yet), but they usually work better than the models built on one language alone, and do degrade gracefully when tested on completely new languages.
Date: 10/15/2021 Time: 2:40 PM PDT Title: Discussion
Date: 10/22/2021 Time: 2:40 PM PDT
Speaker: Daniel Preotiuc-Pietro Affiliation: Bloomberg
Title: Applied Named Entity Recognition @ Bloomberg
Abstract: Bloomberg deals with a wealth of heterogeneous data, including (but not limited to): reports from financial analysts, earnings releases, company filings, news stories, web scrapes, social media posts, ticker symbols, and pricing information of a wide variety of securities. With 80% of financial data in unstructured formats, the ability to quickly identify named entities automatically is essential to understanding this content. We will briefly introduce Bloomberg's products that use this technology and present the unique challenges related to Bloomberg's data and business. We will then present three of our recent publications that aim to tackle the following challenges: heterogeneity of content, temporal data drift, and ensuring high precision in tagging.
Bio: Daniel Preoțiuc-Pietro is a Senior Research Scientist at Bloomberg, where he leads the core NLP group that powers models for processing news, social media and financial documents. His research interests are focused on understanding the social and temporal aspects of text, especially from social media, with applications in domains such as Social Psychology, Law, Political Science and Journalism. Several of his research studies were featured in popular press including the Washington Post, BBC, New Scientist, Scientific American or FiveThirtyEight.
He is a co-organizer of the Natural Legal Language Processing workshop series since its inception. Prior to joining Bloomberg, Daniel was a postdoctoral researcher at the University of Pennsylvania with the interdisciplinary World Well Being Project and obtained his PhD in Natural Language Processing and Machine Learning at the University of Sheffield, UK.
Pengxiang Cheng is a Senior Research Scientist at Bloomberg, where he works on named entity analysis on news, social media, and financial documents. Prior to joining Bloomberg, he obtained his PhD in computer science at the University of Texas of Austin, working on Natural Language Processing and Computational Semantics. His research interests are focused on integrating structural semantic knowledge into end-to-end neural models for better natural language understanding and reasoning.
Date: 10/29/2021 Time: 2:40 PM PDT
Speaker: Srinivas Bangalore Affiliation: Interactions LLC
Title: Who is responsible for the success of Conversational AI?
Abstract: Advances in conversational technologies including speech recognition, natural language understanding, dialog management, language generation and text-to-speech synthesis have accelerated the adoption of conversational systems by businesses and consumers alike for everyday tasks. Even so, architecting and engineering a complex enterprise-grade conversational system to deliver an effective and efficient user experience involves a delicate orchestration between technologies and personnel. From right-sizing the use cases to designing an efficient dialog management for those use cases, customizing the conversational technologies and evolving the system for enterprise business, the success of a conversational system requires a judicious balance of several metrics, often along competing dimensions. In this talk, I will highlight the complexity of building an enterprise-grade conversational system, and discuss some of the research directions being pursued at Interactions.
Bio: Srinivas Bangalore is currently the Vice President of AI Research technologies at Interactions LLC and a visiting lecturer at Princeton University. He has made significant contributions to many areas of natural language processing including Natural Language Understanding, Spoken Language Translation, Multimodal Understanding, Language Generation and Question-Answering. He has co-edited three books on Supertagging, Natural Language Generation, and Language Translation, authored over 150 research publications and holds over 100 patents in these areas. He is the recipient of the Morris and Dorothy Rubinoff award for outstanding dissertation, the AT&T Outstanding Mentor Award, and the AT&T Science & Technology Medal.
Date: 11/5/2021 Time: 2:40 PM PDT Title: Discussion
Date: 11/12/2021 Time: 2:40 PM PDT
Speaker: Yunyao Li Affiliation: IBM
Title: Towards Universal Natural Language Understanding: Recent Progress and Open Challenges
Abstract: Natural language understanding is a fundamental task in artificial intelligence. English understanding has reached a mature state and successfully deployed in multiple IBM AI products and services, such as Watson Natural Language Understanding and Watson Discovery. However, scaling existing products/services to support additional languages remain an open challenge. In this talk, we will discuss the open challenges in supporting universal natural language understanding. We will share our work in the past few years in addressing these challenges. We will also showcase how universal semantic representation of natural languages can enable cross-lingual information extraction in concrete domains (e.g. compliance) and show ongoing efforts towards seamless scaling existing NLP capabilities across languages with minimal efforts.
Date: 11/19/2021 Time: 2:40 PM PDT
Speaker: Ananth Sankar Affiliation: LinkedIn
Title: Deep neural networks for search and recommendation systems at LinkedIn
Abstract: Deep neural networks, like convolutional neural networks (CNN), recurrent neural networks (RNN), and attention-based encoder-decoder networks have made a big impact in several natural language processing (NLP) applications, such as sentence classification, part of speech tagging, and machine translation. In recent years, transfer learning methods using models like BERT and its variants have improved the state of the art in NLP through contextual word embeddings, and sentence embeddings.
In this talk, I will give a high-level overview of transfer learning in NLP using deep neural networks, and give examples of its successful use in search and recommendation systems at LinkedIn.
Date: 12/3/2021 Time: 2:40 PM PDT Title: Discussion
|