Skip to content
codingfeng-fufuPublic

About

信息检索课程作业

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

3 Commits

Folders and files

Repository files navigation

# Scholar IR System A comprehensive academic paper information retrieval system built with Spring Boot and Apache Lucene. ## 馃幆 Overview Scholar IR System is a powerful information retrieval platform designed for processing and searching academic papers. It provides researchers, students, and academics with an efficient way to discover and explore scholarly literature through advanced search capabilities. ## 🚀 Quick Start **New to Scholar IR System?** Check out our guides: - **[⚡ 5-Minute Quick Start](QUICK_START.md)** - Get up and running immediately - **[📖 Detailed Setup Guide](DETAILED_SETUP_GUIDE.md)** - Complete installation and configuration ### Super Quick Start `�ash git clone https://github.com/codingfeng-fufu/IR_system.git cd IR_system mkdir -p data/pdfs data/index logs mvn spring-boot:run # Visit http://localhost:8080 ` ## 鉁?Features ### Core Functionality - **PDF Processing**: Automated metadata extraction using Grobid - **Full-Text Search**: Powered by Apache Lucene with relevance ranking - **Advanced Search**: Multi-field queries with boolean operators - **Smart Highlighting**: Automatic search term highlighting in results - **Similar Papers**: Content-based paper recommendations - **Responsive UI**: Modern web interface with Bootstrap 5 ### Search Capabilities - 鉁?Keyword search across all fields - 鉁?Field-specific search (title, authors, abstract, etc.) - 鉁?Boolean queries (AND, OR, NOT) - 鉁?Phrase search with quotes - 鉁?Wildcard search support - 鉁?Date range filtering - 鉁?Result sorting and pagination ### Technical Features - 鉁?Batch PDF processing (1000+ papers) - 鉁?Real-time indexing and search - 鉁?RESTful API endpoints - 鉁?Comprehensive error handling - 鉁?Performance monitoring - 鉁?Automated testing ## 馃彈锔?Architecture ``` 鈹屸攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹? 鈹屸攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹? 鈹屸攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹?鈹? PDF Files 鈹傗攢鈹€鈹€鈻垛攤 Grobid 鈹傗攢鈹€鈹€鈻垛攤 Paper 鈹?鈹? (Input) 鈹? 鈹? Parser 鈹? 鈹? Entities 鈹?鈹斺攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹? 鈹斺攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹? 鈹斺攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹? 鈹? 鈻?鈹屸攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹? 鈹屸攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹? 鈹屸攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹?鈹? Web UI 鈹傗梽鈹€鈹€鈹€鈹? Spring Boot 鈹傗梽鈹€鈹€鈹€鈹? Database 鈹?鈹? (Thymeleaf) 鈹? 鈹? Application 鈹? 鈹? (H2/MySQL) 鈹?鈹斺攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹? 鈹斺攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹? 鈹斺攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹? 鈹? 鈻? 鈹屸攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹? 鈹? Lucene 鈹? 鈹? Search Index 鈹? 鈹斺攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹?``` ## 馃洜锔?Technology Stack ### Backend - **Java 17** - Programming language - **Spring Boot 2.7** - Application framework - **Spring Data JPA** - Data persistence - **H2 Database** - Embedded database (development) - **Maven 3.9+** - Build tool ### Search Engine - **Apache Lucene 9.7** - Full-text search engine - **Standard Analyzer** - Text analysis - **Query Parser** - Query processing - **Highlighter** - Result highlighting ### PDF Processing - **Grobid 0.7.3** - PDF metadata extraction - **Apache PDFBox** - PDF manipulation - **XML Processing** - Metadata parsing ### Frontend - **Thymeleaf** - Server-side templating - **Bootstrap 5** - CSS framework - **jQuery** - JavaScript library - **Font Awesome** - Icons ## 馃殌 Quick Start ### Prerequisites - Java 17 or higher - Maven 3.9 or higher - Grobid server (optional, for PDF processing) ### Installation 1. **Clone the repository** ```bash git clone https://github.com/scholar-ir/system.git cd scholar-ir-system ``` 2. **Build the project** ```bash mvn clean compile ``` 3. **Run tests** ```bash mvn test ``` 4. **Start the application** ```bash mvn spring-boot:run ``` 5. **Access the system** - Open your browser and go to: `http://localhost:8080` - The application will start with an empty index ### Setting up Grobid (Optional) For PDF processing functionality: 1. **Download Grobid** ```bash wget https://github.com/kermitt2/grobid/archive/0.7.3.zip unzip 0.7.3.zip cd grobid-0.7.3 ``` 2. **Build and start Grobid** ```bash ./gradlew clean install ./gradlew run ``` 3. **Verify Grobid is running** - Grobid will be available at: `http://localhost:8070` ## 馃搧 Project Structure ``` src/ 鈹溾攢鈹€ main/ 鈹? 鈹溾攢鈹€ java/com/scholar/ir/ 鈹? 鈹? 鈹溾攢鈹€ controller/ # Web controllers 鈹? 鈹? 鈹溾攢鈹€ dto/ # Data transfer objects 鈹? 鈹? 鈹溾攢鈹€ model/ # Entity classes 鈹? 鈹? 鈹溾攢鈹€ repository/ # Data repositories 鈹? 鈹? 鈹溾攢鈹€ service/ # Business logic 鈹? 鈹? 鈹斺攢鈹€ ScholarIrApplication.java 鈹? 鈹斺攢鈹€ resources/ 鈹? 鈹溾攢鈹€ static/ # CSS, JS, images 鈹? 鈹溾攢鈹€ templates/ # Thymeleaf templates 鈹? 鈹斺攢鈹€ application.yml # Configuration 鈹斺攢鈹€ test/ # Unit tests ``` ## 馃敡 Configuration Key configuration options in `application.yml`: ```yaml scholar: ir: # Lucene index configuration index: path: ./data/index analyzer: standard # PDF processing configuration pdf: storage-path: ./data/pdfs batch-size: 10 # Grobid configuration grobid: server-url: http://localhost:8070 timeout: 60000 # Search configuration search: default-page-size: 10 max-page-size: 100 ``` ## 馃搳 API Endpoints ### Search API - `GET /api/search` - Search papers - `GET /api/paper/{id}` - Get paper details - `GET /api/suggestions` - Get search suggestions - `GET /api/paper/{id}/similar` - Get similar papers ### Admin API - `POST /api/admin/process-pdfs` - Process PDF files - `POST /api/admin/index/rebuild` - Rebuild search index - `GET /api/admin/stats` - Get system statistics - `GET /api/admin/health` - Health check ### Web Pages - `/` - Homepage with search interface - `/search` - Search results page - `/paper/{id}` - Paper details page - `/about` - About page - `/help` - Help and documentation ## 馃И Testing Run the test suite: ```bash # Run all tests mvn test # Run specific test class mvn test -Dtest=IndexServiceTest # Run tests with coverage mvn test jacoco:report ``` ## 馃搱 Performance ### Benchmarks - **Search Response Time**: < 2 seconds - **Concurrent Users**: 50+ - **Index Build Time**: < 30 minutes (1000 papers) - **Memory Usage**: < 512MB (typical) ### Optimization Tips - Use batch processing for large PDF collections - Enable index optimization for better search performance - Configure appropriate JVM heap size for large datasets - Use database connection pooling for production ## 馃殌 Deployment ### Development ```bash mvn spring-boot:run ``` ### Production ```bash # Build JAR mvn clean package # Run JAR java -jar target/ir-system-1.0.0-SNAPSHOT.jar # With custom configuration java -jar target/ir-system-1.0.0-SNAPSHOT.jar --spring.profiles.active=prod ``` ### Docker (Optional) ```dockerfile FROM openjdk:17-jdk-slim COPY target/ir-system-1.0.0-SNAPSHOT.jar app.jar EXPOSE 8080 ENTRYPOINT ["java", "-jar", "/app.jar"] ``` ## 馃 Contributing 1. Fork the repository 2. Create a feature branch (`git checkout -b feature/amazing-feature`) 3. Commit your changes (`git commit -m 'Add amazing feature'`) 4. Push to the branch (`git push origin feature/amazing-feature`) 5. Open a Pull Request ## 馃摑 License This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. ## 馃啒 Support - **Documentation**: Visit `/help` page in the application - **Issues**: Report bugs on GitHub Issues - **Email**: support@scholar-ir.com - **Wiki**: Check the project wiki for detailed guides ## 馃檹 Acknowledgments - **Apache Lucene** - For the powerful search engine - **Grobid** - For PDF metadata extraction - **Spring Boot** - For the excellent framework - **Bootstrap** - For the responsive UI components --- **Scholar IR System** - Making academic research more accessible through intelligent information retrieval. # IR-sytem # IR_system

About

信息检索课程作业

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages