Python图像问答算法的聊天机器人软件的设计+源代码
毕业论文关键词：图像问答；聊天机器人；卷积神经网络；长短期记忆网络；
Design and Implementation of Chat Robot Software Based on Visual Question Answering Algorithm
Abstract: Chat Robot are one of the hot research fields in recent years .Now, most of the chat robots can only deal with the text questions. However, when it comes to something about images, they can not have any response. So we implement a visual question answering algorithm (VQA) which based on the paper --”Visual Question answering”. And then we design and implement a chat robot software. The visual question answering algorithm is based on the convolutional neural network(CNN) and the long-short memory(LSTM). The CNN is used to extract the image features. We translate the natural language question into word vectors. Then we put the word vectors of the question and the vector of image feature into LSTM, and the output is the answer to the question. To train the model,we use the pretrained COCO image features and GloVe which published by Stanford.The algorithm we implemented can answer some simple questions, such as "What is" "How many",etc. The accuracy of the algorithm in the dataset VQA (v1) can reach to 52.84%.However, when we use the model we should extract the image feature by ourslves.So we implement the VGG-16 network to extract image feature. Based on this image question answering algorithm and the Turing robot's web interface, we design and implement a chat robot. The chat robot has both the server and the android client. Where the server is implemented by using Python's Web.py framework. Also, We use the Baidu Translation to transfer the Chinese question to English question. Then we can use the VQA model to answer the question written by Chinese. The main role of the server is to receive messages sent by the client and make the appropriate response at the same time. The function of the client is to send messages to the server and receive messages. The android app can also take photos and see the system gallery. The chat robot has the image question answering, chatting, open field question answering, query the weather, tell jokes, tell stories and other functions.
Key words: Visual Question Answering; Chat Robot; CNN; LSTM;
目录
摘要 1
关键词 1
Abstract 1
Key words 2
1绪论 2
1.1研究背景及意义 2
1.2问答系统研究综述 2
1.2.1聊天机器人软件 3
1.2.2问答系统国内外研究现状 3
1.2.3图像问答综述 4
1.3存在的主要问题 5
1.4研究内容 5
1.5技术路线 5
1.6论文组织结构 6