基于Hadoop平台的文档检索技术研究与实现

摘要随着网络的迅猛发展，信息量呈爆炸式增长。越来越多的企业对于自身向顾客提供的产品或服务的检索都变得十分困难。对于小型企业更多的公司都采用数据库存储并使用数据库匹配读取来进行检索，但是当提供的产品或服务持续增加，不断进行数据库读取的操作显得十分缓慢。因此，为了提高对用户显示产品或服务信息的反馈速度，本文设计并实现了基于Lucene软件包的，适用于J2EE模型的简易产品检索的小型搜索系统。64774

本文通过学习通用搜索引擎基本原理，结合需求构建一个运行稳定、性能良好且具有可扩展性的小型搜索引擎系统。本文论述了系统开发的背景以及国内外搜索引擎技术发展现状，分析了小型搜索系统的需求，明确了系统开发过程中要解决的问题，提出了具体的设计方案。论文的主要工作如下：

1. 将整个工程建立在J2EE模型上，使用MyBatis持久层联系数据库与JavaBean。

2. 针对需求，完成对JavaBean建立索引并搜索。

3. 为了提高可扩展性，使用Java反射机制以及注解进行编程

4. 学习并设计大数据量情况下采用Hadoop平台进行分布式处理的模型

毕业论文关键词搜索 Lucene 可扩展性分布式

毕业设计说明书（论文）外文摘要

Title Research and Implementation of Hadoop-based Platform for Document Retrieval Technology

Abstract

With the rapid development of the network, the amount of information has exploded. More and more corporation find it difficult for customers to retrieve their products or services. Most of small companies are using database retrieve, but if the amount of product continues to increase, database reading is very slow. Therefore, in order to improve the speed of the feedback to customers, we designed and implemented a J2EE model small search system based on Lucene package.

This article studies the basic principles of search engines, combining demands to build a stable-operating, good-performing and scalable small search system. This paper discusses the search system development background, domestic and foreign search engine technology development status, analyses the requirements of the small search system, clears the problems to be solved in the process of developing the system, and then put forward the specific design. The main work is follows,

1. Build the entire project in the J2EE model, using MyBatis to connect the database with JavaBeans.

2. In response to demands, complete JavaBeans indexing and searching.

3. In order to improve scalability, using Java reflection mechanism and annotations for programming.

4. Study and design of large amount of data, using Hadoop platform for distributed processing.

Keywords search Lucene scalability distributed

1 引言 5

1.1 课题的研究背景 5

1.2 小型搜索引擎的发展 6

1.3 分布式搜索引擎现状 6

2 系统结构分析设计 8

2.1 系统需求分析 8

2.2 系统设计图 8

2.3 系统详细设计 10

3 搜索引擎的实现 16

3.1 开发平台 16

3.2 数据库模块的实现 16

3.3 索引模块的实现 17 基于Hadoop平台的文档检索技术研究与实现:http://www.youerw.com/jisuanji/lunwen_72094.html