Cognitive Computing & Conversational AI

In this paper we describe the requirements and early system design for a smart conversational agent that can assist older adults in the reminiscence process. The practice of reminiscence has well documented benefits for the mental, social and emotional well-being of older adults. However, the technology support, valuable in many different ways, is still limited in terms of need of co-located human presence, data collection capabilities, and ability to support sustained engagement, thus missing key opportunities to improve care practices, facilitate social interactions, and bring the reminiscence practice closer to those with less opportunities to engage in co-located sessions with a (trained) companion. We discuss conversational agents and cognitive services as the platform for building the next generation of reminiscence applications, and introduce the concept application of a smart reminiscence agent.

This paper lays the foundation for a new delivery paradigm for web-accessible content and functionality, i.e., conversational interaction. Instead of asking users to read text, click through links and type on the keyboard, the vision is to enable users to “speak to a website” and to obtain natural language, spoken feedback. The paper describes how state-of-the-art chatbot technology can enable a dialog between the user and the website, proposes a reference architecture for the automated inference of site-specific chatbots able to mediate between the user and the website, and discusses open challenges and research questions. The envisioned, bidirectional dialog paradigm advances current screen reader technology and aims to benefit both regular users in eyes-free usage scenarios as well as visually impaired users in everyday scenarios.

In this paper, we describe the foundations for generating a chatbot out of a website equipped with simple, bot-specific HTML annotations. The approach is part of what we call conversational web browsing, i.e., a dialog-based, natural language interaction with websites. The goal is to enable users to use content and functionality accessible through rendered UIs by “talking to websites” instead of by operating the graphical UI using keyboard and mouse. The chatbot mediates between the user and the website, operates its graphical UI on behalf of the user, and informs the user about the state of interaction. We describe the conceptual vocabulary and annotation format, the supporting conversational middleware and techniques, and the implementation of a demo able to deliver conversational web browsing experiences through Amazon Alexa.

Chatbots are software agents that are able to interact with humans in natural language. Their intuitive interaction paradigm is expected to significantly reshape the software landscape of tomorrow, while already today chatbots are invading a multitude of scenarios and contexts. This article takes a developer’s perspective, identifies a set of architectural patterns that capture different chatbot integration scenarios, and reviews state-of-the-art development aids.

In this paper we explore the opportunities brought by cognitive augmentation to provide a more natural and accessible web browsing experience. We explore these opportunities through conversational web browsing, an emerging interaction paradigm for the Web that enables blind and visually impaired users (BVIP), as well as regular users, to access the contents and features of websites through conversational agents. Informed by the literature, our previous work and prototyping exercises, we derive a conceptual framework for supporting BVIP conversational web browsing needs, to then focus on the challenges of automatically providing this support, describing our early work and prototype that leverage heuristics that consider structural and content features only.

Systematic literature reviews (SLRs) are at the heart of evidence-based research, setting the foundation for future research and practice. However, producing good quality timely contributions is a challenging and highly cognitive endeavor, which has lately motivated the exploration of automation and support in the SLR process. In this paper we address an often overlooked phase in this process, that of planning literature reviews, and explore under the lenses of cognitive process augmentation how to overcome its most salient challenges. In doing so, we report on the insights from 24 SLR authors on planning practices, its challenges as well as feedback on support strategies inspired by recent advances in cognitive computing. We frame our findings under the cognitive augmentation framework, and report on a prototype implementation and evaluation focusing on further informing the technical feasibility.

Chatbots are emerging as a promising platform for accessing and delivering healthcare services. The evidence is in the growing number of publicly available chatbots aiming at taking an active role in the provision of prevention, diagnosis, and treatment services. This article takes a closer look at how these emerging chatbots address design aspects relevant to healthcare service provision, emphasizing the human–AI interaction aspects and the transparency in AI automation and decision making.

We present a vision for the next generation of process technology based on cognitive augmentation. Starting from current process technology, we show how by augmenting layers of cognitive intelligence to combine advances in machine-automation, crowdsourcing and more importantly adaptation and reasoning, we can advance support for emerging requirements of highly changing environments. We believe the challenges lie in the synergy between human and machine, in understanding how to orchestrate and combine their contributions. This vision paper sets forth a roadmap for future research by introducing a framework for cognitive augmentation, identifying the relevant research and technologies, and discussing its application amidst real-world use cases.

Attracted by their easy-to-use interfaces and captivating benefits, conversational systems have been widely embraced by many individuals and organizations as side-by-side digital co-workers. They enable the understanding of user needs, expressed in natural language, and on fulfilling such needs by invoking the appropriate backend services (e.g., APIs). Controlling the conversation flow, known as Dialogue Management, is one of the essential tasks in conversational systems and the key to its success and adoption as well. Nevertheless, designing scalable and robust dialogue management techniques to effectively support intelligent conversations remains a deeply challenging problem. This article studies dialogue management from an in-depth design perspective. We discuss the state-of-the-art approaches, identify their recent advances and challenges, and provide an outlook on future research directions. Thus, we contribute to guiding researchers and practitioners in selecting the appropriate dialogue management approach aligned with their objectives, among the variety of approaches proposed so far.

Task-oriented conversational bots allow users to access services and perform tasks through natural language conversations. However, integrating these bots and software-enabled services has not kept pace with our ability to deploy individual devices and services. The main drawbacks of current bots and services integration techniques stem from the inherent development and maintenance cost. In addition, existing Natural Language Processing (NLP) techniques automate various tasks but the synthesis of API calls to support broad range of potentially complex user intents is still largely a manual and costly process. In this paper, we propose three types of reusable patterns for recognising compositional conversational flows and therefore automatically support increased complexity and expressivity during the conversation.

In this demonstration paper we showcase an extensible and reusable pipeline for automatic paraphrase generation, i.e., reformulating sentences using different words. Capturing the nuances of human language is fundamental to the effectiveness of Conversational AI systems, as it allows them to deal with the different ways users can utter their requests in natural language. Traditional approaches to utterance paraphrasing acquisition, such as hiring experts or crowd-sourcing, involve processes that are often costly or time consuming, and with their own trade-offs in terms of quality. Automatic paraphrasing is emerging as an attractive alternative that promises a fast, scalable and cost-effective process. In this paper we showcase how our extensible and reusable pipeline for automated utterance paraphrasing can support the development of Conversational AI systems by integrating and extending existing techniques under an unified and configurable framework.

A prominent approach to build datasets for training task-oriented bots is crowd-based paraphrasing. Current approaches, however, assume the crowd would naturally provide diverse paraphrases or focus only on lexical diversity. In this WiP we addressed an overlooked aspect of diversity, introducing an approach for guiding the crowdsourcing process towards paraphrases that are syntactically diverse.

Chatbots are increasingly becoming important gateways to digital services and information—taken up within domains such as customer service, health, education, and work support. However, there is only limited knowledge concerning the impact of chatbots at the individual, group, and societal level. Furthermore, a number of challenges remain to be resolved before the potential of chatbots can be fully realized. In response, chatbots have emerged as a substantial research area in recent years. To help advance knowledge in this emerging research area, we propose a research agenda in the form of future directions and challenges to be addressed by chatbot research. This proposal consolidates years of discussions at the CONVERSATIONS workshop series on chatbot research. Following a deliberative research analysis process among the workshop participants, we explore future directions within six topics of interest: (a) users and implications, (b) user experience and design, (c) frameworks and platforms, (d) chatbots for collaboration, (e) democratizing chatbots, and (f) ethics and privacy. For each of these topics, we provide a brief overview of the state of the art, discuss key research challenges, and suggest promising directions for future research. The six topics are detailed with a 5-year perspective in mind and are to be considered items of an interdisciplinary research agenda produced collaboratively by avid researchers in the field.

Conversational services are emerging as a new paradigm for accessing information by simply uttering questions in natural language, posing a whole new set of challenges to the design and engineering of information systems. Training conversational services to deal with the nuances of natural language often requires collecting a high-quality and diverse set of training samples (i.e., paraphrases). Traditional approaches such as hiring an expert or crowdsourcing involve data collection processes that are often costly and time-consuming. Automated paraphrase generation is a promising cost-effective and scalable approach to generating training samples. Current automatic techniques, however, tend to specialise in specific types of lexical or syntactic variations. As a result, generated paraphrases may not perform well in relevant quality aspects such as diversity and semantic relatedness. In this paper, we follow an approach inspired by services integration to address these issues and generate paraphrases in English that are semantically relevant and diverse. We propose an extensible and reusable pipeline that combines automatic paraphrasing techniques in a two-step process that first focus on i) leveraging the strengths of multiple techniques to generate the most diverse (and possibly noisy) set of paraphrases, to then ii) address common quality issues in a separate step. Through empirical evaluations we show the benefits of the two-step process design and of combining techniques for more balancing relevance and diversity.

Conversational AI is changing the way we interact with digital services. However, there is still a lack of conversational paradigms facilitating access to the Web. This paper discusses a new approach for Conversational Web Browsing, and introduces a design space identified through a user-centered process that involved 26 blind and visually impaired users. The paper also illustrates the conceptual architecture of a software framework that can automatically generate conversational agents for the Conversational Web.

Task-oriented dialogue systems employ third-party APIs to serve end-users via natural language interactions. while existing advances in Natural Language Processing (NLP) and Machine Learning (ML) techniques have produced promising and useful results to recognize user intents, the synthesis of API calls to support a broad range of potentially complex user intents is still largely a manual and costly process. In this paper, we propose a new approach to recognize and realize complex user intents. Our approach relies on a new rule-based technique that leverages both (i) natural language features extracted using existing NLP and ML techniques and (ii) contextual knowledge to capture the different classes of complex intents. We devise a context knowledge service to capture the requisite contextual knowledge.

Systematic literature reviews (SLRs) are at the heart of evidence-based research, collecting and integrating empirical evidence regarding specific research questions. A leading step in the search for relevant evidence is composing Boolean search queries, which are still at the core of how information retrieval systems work to perform an advanced literature search. Building these queries thus requires going from the general aims of the research questions into actionable search terms that are combined into potentially complex Boolean expressions. Researchers are thus tasked with the daunting and challenging task of building and refining search queries in their quest for sufficient coverage and proper representation of the literature. In this paper, we propose an adaptive Boolean query generation and refinement pipeline for SLR search. Our approach utilizes a reinforcement learning technique to learn the optimal modifications for a query based on the feedback collecting from the researchers about the query retrieval performance. Empirical evaluations with 10 SLR datasets showed our approach to achieve comparable performance to that of queries manually composed by SLR authors.

Task-oriented bots (or simply bots) enable humans to perform tasks in natural language. For example, to book a restaurant or check the weather. Crowdsourcing has become a prominent approach to build datasets for training and evaluating task-oriented bots, where the crowd grows an initial seed of utterances through paraphrasing, i.e., reformulating a given seed into semantically equivalent sentences. In this context, the resulting diversity is a relevant dimension of high-quality datasets, as diverse paraphrases capture the many ways users may express an intent. Current techniques, however, are either based on the assumption that crowd-powered paraphrases are naturally diverse or focus only on lexical diversity. In this paper, we address an overlooked aspect of diversity and introduce an approach for guiding the crowdsourcing process towards paraphrases that are syntactically diverse. We introduce a workflow and novel prompts that are informed by syntax patterns to elicit paraphrases avoiding or incorporating desired syntax. Our empirical analysis indicates that our approach yields higher syntactic diversity, syntactic novelty and more uniform pattern distribution than state-of-the-art baselines, albeit incurring on higher task effort.

Conversational AI is disrupting the way information is accessed. However, there is still a lack of conversational technologies leveraging the Web. This paper introduces an approach to support the notion of Conversational Web Browsing. It illustrates design patterns for navigating websites through conversation and shows how such patterns are sustained by a Web architecture that integrates NLP technologies.

Task-oriented conversational assistants are in very high demand these days. They employ third-party APIs to serve end-users via natural language interactions and improve their productivity. Recently, the augmentation of process-enabled automation with conversational assistants emerged as a promising technology to make process automation closer to users. This paper focuses on the superimposition of task-oriented assistants over composite services. We propose a Human-bot-Process interaction acts that are relevant to represent natural language conversations between the user and multi-step processes. In doing so, we enable human users to perform tasks by naturally interacting with processes.

Conversational agents are emerging as channels for a natural and accessible interaction with digital services. Their benefits span across a wide range of usage scenarios and address visual impairments and any situational impairments that may take advantage of voice-based interactions. A few works highlighted the potential and the feasibility of adopting conversational agents for making the Web truly accessible for everyone. Yet, there is still a lack of concrete guidance in designing conversational experiences for browsing the Web. This paper illustrates a human-centered process that involved 26 blind and visually impaired people to investigate their difficulties when using assistive technology for accessing the Web, and their attitudes and preferences on adopting conversational agents. In response to the identified challenges, the paper introduces patterns for conversational Web browsing. It also discusses design implications that can promote Conversational AI as a technology to enhance Web accessibility.

Systematic literature reviews (SLRs) are a central part of evidence-based research, which involves collecting and integrating empirical evidence on specific research questions. A key step in this process is building Boolean search queries, which are at the core of information retrieval systems that support literature search. This involves turning general research aims into specific search terms that can be combined into complex Boolean expressions. Researchers must build and refine search queries to ensure they have sufficient coverage and properly represent the literature. In this paper, we propose an adaptive query generation and refinement pipeline for SLR search that uses reinforcement learning to learn the optimal modifications to a query based on feedback from researchers about its performance. Empirical evaluations with 10 SLR datasets showed our approach achieves comparable performance to queries manually composed by SLR authors. We also investigate the impact of design decisions on the performance of the query generation and refinement pipeline. Specifically, we study the effects of the type of input seed, the use of general versus domain-specific word embedding models, the sampling strategy for relevance feedback, and number of iterations in the refinement process. Our results provide insights into the effects of these choices on the pipeline’s performance.