Zhou Yu 俞舟
Zhou (pronounced similar to Jo)
Computer Science Department
University of California, Davis
Academic Surge 2085
1 Shields Ave, Davis, CA 95616
Twitter: @zhouYu26514074 (I tweet more frequently than updating this webpage)
I am an Assistant Professor at the Computer Science Department, University of California, Davis. I am the director of the Davis NLP Lab. I recieved my PhD at Language Technology Institute under School of Computer Science, Carnegie Mellon University 2017, working with Prof. Alan W Black and Prof. Alexander I. Rudnicky. 2015 summer and 2016 summer, I interned with Prof. David Suendermann-Oeft at ETS San Francisco Office on cloud based mulitmodal dialog systems. 2014 Fall, I interned with Dan Bohus and Eric Horvitz at Microsoft Research on situated multimodal dialogue systems.
Prior to CMU, I received a B.S. in Computer Science and a B.A. in Linguistics under English Language Major from Zhejiang University in 2011. I worked with Prof. Xiaofei He and Prof. Deng Cai there on Machine Learning and Computer Vision. I also worked with Prof. Yunhua Qu on Machine Translation.
I design algorithms for real-time intelligent interactive systems that coordinate with user actions that are beyond spoken languages, including non-verbal behaviors to achieve effective and natural communications. In particular, I optimize human-machine communication via studies of multimodal sensing and analysis, speech and natural language processing, machine learning and human-computer interaction. The central focus of my dissertation research is to bring together all the areas above to design, implement and deploy end-to-end real-time interactive intelligent systems that are able to plan globally considering interaction history and current user actions to achieve better user experience and task performance. Meanwhile, I enjoy collaborating with researchers with different backgrounds on interdisciplinary research in all area of science, such as health care, education and robotics.
I am also interested in various tasks in natural language processing, such as language understanding, language generation and common sense reasoning.
New WeCNLP invited talk video on Builidng Dialog Systems with Less Supervsion video
Recent talk: Building Dialog Systems with Less Supervision slides
Recent talk: Gunrock, 2018 Amazon Alexa Socialbot Winner slides
Recent talk: Grounding Reinforcement Learning with Real-World Dialog Applications slides
Here is a YouTube video on Multimodal Dialog Systems at AI2 (Thanks AI2 for the recording) video
Here is a Chinese verison of Multimodal Dialog Systems talk that targets to general public. video
Our paper Persuasion For Good was nominated for ACL 2019 best paper.
I was featured on Forbes 2018 30 under 30 in Science webpage If you want to try our chatbot, just say "Alexa Let's chat!" to any Alexa device or download the App Amazon Alexa on your phone.
I am recruiting postdocs who have interests and background knowledge on areas, such as Robotics, NLP, ML, and AI in general.
If you are thinking of doing a PhD with me, please read Mor's advice Applying PhD Due to the large volum of incoming email, I would not be able to reply to individual student. Please apply to our UC Davis PhD program
Weixin Liang, Youzhi Tian, Chengcai Chen, Zhou Yu,MOSS: End-to-End Dialog System Framework with Modular Supervision, arXiv [pdf]
Dian Yu & Zhou Yu, MIDAS: A Dialog Act Annotation Scheme for Open Domain Human Machine Spoken Conversations, arXiv [pdf]
Mingyang Zhou, Joshua Arnold, Zhou Yu, Building Task-Oriented Visual Dialog Systems Through Alternative Optimization Between Dialog Policy and Language Generation, EMNLP 2019 [pdf]
Weiyan Shi, Kun Qian, Xuewei Wang, Zhou Yu, How to Build User Simulators to Train RL-based Dialog Systems, EMNLP 2019 [pdf]
Sam Davidson, Dian Yu, Zhou Yu, Dependency Parsing for Spoken Dialog Systems, EMNLP 2019 [pdf]
Dian Yu, et al., Gunrock: A Social Bot for Complex and Engaging Long Conversation, EMNLP 2019 demo [pdf]
Michelle Cohn, Chun-Yen Chen and Zhou Yu, A Large-Scale User Study of an Alexa Prize Chatbot: Effect of TTS Dynamism on Perceived Quality of Social Dialog, SIGDIAL 2019 [pdf]
Kun Qian & Zhou Yu, Domain Adaptive Dialog Generation via Meta Learning, ACL 2019 [pdf]
Xuewei Wang, Weiyan Shi, Richard Kim, Yoo Jung Oh, Sijia Yang, Jingwen Zhang, Zhou Yu, Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good, ACL 2019[pdf], [data and code] ** best paper nomination
Weiyan Shi, Tiancheng Zhao, Zhou Yu, Unsupervised Dialog Structure Learning, NAACL 2019 [pdf]
Chen et al., Gunrock: Building A Human-Like Social Bot By Leveraging Large Scale Real User Data [pdf]
Jiaao Chen, Jianshu Chen and Zhou Yu, Incorporating Structured Commonsense Knowledge in Story Completion, AAAI 2019 [pdf]
Youzhi Tian, Zhiting Hu and Zhou Yu, Structured Content Preservation for Unsupervised Text Style Transfer, arXiv 2018 [pdf]
Zijie Zhu, Xuewei Wang, Aakaash Kapoor, Tingrui Pan and Zhou Yu, EIS: A Wearable Device for Epidermal American Sign Language Recognition, IMWUT 2018 (Ubicomp 2019)[pdf]
Jiaping Zhang, Tiancheng Zhao and Zhou Yu, Multimodal Hierarchical Reinforcement Learning Policy for Task-Oriented Visual Dialog, SIGDIAL 2018 [pdf]
Weiyan Shi and Zhou Yu, Sentiment Adaptive End-to-End Dialog Systems, ACL 2018 [pdf]
Ryant et al., Enhancement and Analysis of Conversational Speech: JSALT 2017, ICAASP 2018 [pdf]
Zhou Yu, Alan W Black and Alexander I. Rudnicky, Learning Conversational Systems that Interleave Task and Non-Task Content, IJCAI 2017 [pdf]
Zhou Yu, Vikram Ramanarayanan, Patrick Lange, and David Suendermann-Oeft. An open-source multimodal dialog system with real-time engagement tracking for job interview trainingapplications.In IWSDS, 2017 [pdf]
Zhou Yu, Ziyu Xu, Alan W Black and Alexander Rudnicky, Strategy and Policy Learning for Non-Task-Oriented Conversational Systems, SIGDIAL 2016. [pdf]
Zhou Yu, Leah Nicolich-Henkin, Alan W Black and Alexander Rudnicky, A Wizard-of-Oz Study on A Non-Task-Oriented Dialog Systems that Reacts to User Engagement, SIGDIAL 2016. [pdf]
Sean Andrist, Dan Bohus, Zhou Yu, Eric Horvitz, Are You Messing with Me?: Querying about the Sincerity of Interactions in the Open World. HRI 2016. [pdf]
Zhou Yu, Dan Bohus and Eric Horvitz, Incremental Coordination: Attention-Centric Speech Production in a Physically Situated Conversational Agent, SIGDIAL 2015. [pdf]
TickTock: a multimodal chatbot with user engagement coordination
- below is a demo of using automatically generated conversational strategy to improve user engagement.
Direction-giving Robot: a direction-giving humanoid robot with user attention coordination
- below is a demo and some real user cases of people interacting with the robot.
HALEF: a distributed web-based multimodal dialog system with user engagement coordination
- below is a demo of a Amazon Turker interacting with our job interview training application via a web browser. It live-streams videos from usrs' local webcam to the server.