Repository files navigation # Scholar IR System
A comprehensive academic paper information retrieval system built with Spring Boot and Apache Lucene.
## 馃幆 Overview
Scholar IR System is a powerful information retrieval platform designed for processing and searching academic papers. It provides researchers, students, and academics with an efficient way to discover and explore scholarly literature through advanced search capabilities.
## 🚀 Quick Start
**New to Scholar IR System?** Check out our guides:
- **[⚡ 5-Minute Quick Start](QUICK_START.md)** - Get up and running immediately
- **[📖 Detailed Setup Guide](DETAILED_SETUP_GUIDE.md)** - Complete installation and configuration
### Super Quick Start
`�ash
git clone https://github.com/codingfeng-fufu/IR_system.git
cd IR_system
mkdir -p data/pdfs data/index logs
mvn spring-boot:run
# Visit http://localhost:8080
`
## 鉁?Features
### Core Functionality
- **PDF Processing**: Automated metadata extraction using Grobid
- **Full-Text Search**: Powered by Apache Lucene with relevance ranking
- **Advanced Search**: Multi-field queries with boolean operators
- **Smart Highlighting**: Automatic search term highlighting in results
- **Similar Papers**: Content-based paper recommendations
- **Responsive UI**: Modern web interface with Bootstrap 5
### Search Capabilities
- 鉁?Keyword search across all fields
- 鉁?Field-specific search (title, authors, abstract, etc.)
- 鉁?Boolean queries (AND, OR, NOT)
- 鉁?Phrase search with quotes
- 鉁?Wildcard search support
- 鉁?Date range filtering
- 鉁?Result sorting and pagination
### Technical Features
- 鉁?Batch PDF processing (1000+ papers)
- 鉁?Real-time indexing and search
- 鉁?RESTful API endpoints
- 鉁?Comprehensive error handling
- 鉁?Performance monitoring
- 鉁?Automated testing
## 馃彈锔?Architecture
```
鈹屸攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹? 鈹屸攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹? 鈹屸攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹?鈹? PDF Files 鈹傗攢鈹€鈹€鈻垛攤 Grobid 鈹傗攢鈹€鈹€鈻垛攤 Paper 鈹?鈹? (Input) 鈹? 鈹? Parser 鈹? 鈹? Entities 鈹?鈹斺攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹? 鈹斺攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹? 鈹斺攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹? 鈹? 鈻?鈹屸攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹? 鈹屸攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹? 鈹屸攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹?鈹? Web UI 鈹傗梽鈹€鈹€鈹€鈹? Spring Boot 鈹傗梽鈹€鈹€鈹€鈹? Database 鈹?鈹? (Thymeleaf) 鈹? 鈹? Application 鈹? 鈹? (H2/MySQL) 鈹?鈹斺攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹? 鈹斺攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹? 鈹斺攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹? 鈹? 鈻? 鈹屸攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹? 鈹? Lucene 鈹? 鈹? Search Index 鈹? 鈹斺攢鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹€鈹?```
## 馃洜锔?Technology Stack
### Backend
- **Java 17** - Programming language
- **Spring Boot 2.7** - Application framework
- **Spring Data JPA** - Data persistence
- **H2 Database** - Embedded database (development)
- **Maven 3.9+** - Build tool
### Search Engine
- **Apache Lucene 9.7** - Full-text search engine
- **Standard Analyzer** - Text analysis
- **Query Parser** - Query processing
- **Highlighter** - Result highlighting
### PDF Processing
- **Grobid 0.7.3** - PDF metadata extraction
- **Apache PDFBox** - PDF manipulation
- **XML Processing** - Metadata parsing
### Frontend
- **Thymeleaf** - Server-side templating
- **Bootstrap 5** - CSS framework
- **jQuery** - JavaScript library
- **Font Awesome** - Icons
## 馃殌 Quick Start
### Prerequisites
- Java 17 or higher
- Maven 3.9 or higher
- Grobid server (optional, for PDF processing)
### Installation
1. **Clone the repository**
```bash
git clone https://github.com/scholar-ir/system.git
cd scholar-ir-system
```
2. **Build the project**
```bash
mvn clean compile
```
3. **Run tests**
```bash
mvn test
```
4. **Start the application**
```bash
mvn spring-boot:run
```
5. **Access the system**
- Open your browser and go to: `http://localhost:8080`
- The application will start with an empty index
### Setting up Grobid (Optional)
For PDF processing functionality:
1. **Download Grobid**
```bash
wget https://github.com/kermitt2/grobid/archive/0.7.3.zip
unzip 0.7.3.zip
cd grobid-0.7.3
```
2. **Build and start Grobid**
```bash
./gradlew clean install
./gradlew run
```
3. **Verify Grobid is running**
- Grobid will be available at: `http://localhost:8070`
## 馃搧 Project Structure
```
src/
鈹溾攢鈹€ main/
鈹? 鈹溾攢鈹€ java/com/scholar/ir/
鈹? 鈹? 鈹溾攢鈹€ controller/ # Web controllers
鈹? 鈹? 鈹溾攢鈹€ dto/ # Data transfer objects
鈹? 鈹? 鈹溾攢鈹€ model/ # Entity classes
鈹? 鈹? 鈹溾攢鈹€ repository/ # Data repositories
鈹? 鈹? 鈹溾攢鈹€ service/ # Business logic
鈹? 鈹? 鈹斺攢鈹€ ScholarIrApplication.java
鈹? 鈹斺攢鈹€ resources/
鈹? 鈹溾攢鈹€ static/ # CSS, JS, images
鈹? 鈹溾攢鈹€ templates/ # Thymeleaf templates
鈹? 鈹斺攢鈹€ application.yml # Configuration
鈹斺攢鈹€ test/ # Unit tests
```
## 馃敡 Configuration
Key configuration options in `application.yml`:
```yaml
scholar:
ir:
# Lucene index configuration
index:
path: ./data/index
analyzer: standard
# PDF processing configuration
pdf:
storage-path: ./data/pdfs
batch-size: 10
# Grobid configuration
grobid:
server-url: http://localhost:8070
timeout: 60000
# Search configuration
search:
default-page-size: 10
max-page-size: 100
```
## 馃搳 API Endpoints
### Search API
- `GET /api/search` - Search papers
- `GET /api/paper/{id}` - Get paper details
- `GET /api/suggestions` - Get search suggestions
- `GET /api/paper/{id}/similar` - Get similar papers
### Admin API
- `POST /api/admin/process-pdfs` - Process PDF files
- `POST /api/admin/index/rebuild` - Rebuild search index
- `GET /api/admin/stats` - Get system statistics
- `GET /api/admin/health` - Health check
### Web Pages
- `/` - Homepage with search interface
- `/search` - Search results page
- `/paper/{id}` - Paper details page
- `/about` - About page
- `/help` - Help and documentation
## 馃И Testing
Run the test suite:
```bash
# Run all tests
mvn test
# Run specific test class
mvn test -Dtest=IndexServiceTest
# Run tests with coverage
mvn test jacoco:report
```
## 馃搱 Performance
### Benchmarks
- **Search Response Time**: < 2 seconds
- **Concurrent Users**: 50+
- **Index Build Time**: < 30 minutes (1000 papers)
- **Memory Usage**: < 512MB (typical)
### Optimization Tips
- Use batch processing for large PDF collections
- Enable index optimization for better search performance
- Configure appropriate JVM heap size for large datasets
- Use database connection pooling for production
## 馃殌 Deployment
### Development
```bash
mvn spring-boot:run
```
### Production
```bash
# Build JAR
mvn clean package
# Run JAR
java -jar target/ir-system-1.0.0-SNAPSHOT.jar
# With custom configuration
java -jar target/ir-system-1.0.0-SNAPSHOT.jar --spring.profiles.active=prod
```
### Docker (Optional)
```dockerfile
FROM openjdk:17-jdk-slim
COPY target/ir-system-1.0.0-SNAPSHOT.jar app.jar
EXPOSE 8080
ENTRYPOINT ["java", "-jar", "/app.jar"]
```
## 馃 Contributing
1. Fork the repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Pull Request
## 馃摑 License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
## 馃啒 Support
- **Documentation**: Visit `/help` page in the application
- **Issues**: Report bugs on GitHub Issues
- **Email**: support@scholar-ir.com
- **Wiki**: Check the project wiki for detailed guides
## 馃檹 Acknowledgments
- **Apache Lucene** - For the powerful search engine
- **Grobid** - For PDF metadata extraction
- **Spring Boot** - For the excellent framework
- **Bootstrap** - For the responsive UI components
---
**Scholar IR System** - Making academic research more accessible through intelligent information retrieval.
# IR-sytem
# IR_system
You can’t perform that action at this time.