Keller Jolley Preece
Generating a response to a user query utilizing visual features of a video segment and a query-response-neural network
The present disclosure relates to systems, methods, and non-transitory computer-readable media for generating a response to a question received from a user during display or playback of a video segment by utilizing a query-response-neural network. The disclosed systems can extract a query vector from a question corresponding to the video segment using the query-response-neural network. The disclosed systems further generate context vectors representing both visual cues and transcript cues corresponding to the video segment using context encoders or other layers from the query-response-neural network. By utilizing additional layers from the query-response-neural network, the disclosed systems generate (i) a query-context vector based on the query vector and the context vectors, and (ii) candidate-response vectors representing candidate responses to the question from a domain-knowledge base or other source. To respond to a user's question, the disclosed systems further select a response from the candidate responses based on a comparison of the query-context vector and the candidate-response vectors.
Adobe Inc.
G G G G G G G G G G G G
G06F G06V G06V G06N G06V G06N G06V G06N G06V G06V G06N G06V
inventional inventional additional inventional inventional inventional inventional inventional inventional inventional inventional inventional
Seokhwan Ning Hailin Wentian
Kim Xu Jin Zhao