fix(nlp): add punkt_tab download, fix regex escape, and add QuestionEnhancer unit tests

[Rajratna-D]

Summary of Changes

1. NLTK 3.9+ Compatibility: Added nltk.download('punkt_tab', quiet=True) in backend/Generator/question_filters.py. In modern NLTK releases, tokenization expects punkt_tab, which previously resulted in a LookupError: Resource 'punkt_tab' not found.
2. Regex Raw String Fix: Added raw string prefix r"..." to re.findall(r".*?[.!\?]", text) in backend/Generator/main.py to resolve Python 3.12+ SyntaxWarning: invalid escape sequence '\?'.
3. Unit Tests: Added a dedicated test suite Testing/test_question_filters.py containing 11 tests covering:
   • Initializer validation
   • Standard question enhancement
   • Edge cases: empty string, None, single characters, non-string types
   • Case-insensitive question word alternatives
   • Punctuation spacing cleanup

Testing

Ran pytest locally:

pytest Testing/test_question_filters.py -v


<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

* **Improvements**
  * Question generation now has access to an additional language-processing resource during initialization, supporting sentence processing. No changes to the wording or behavior of generated questions are noted.
* **Tests**
  * Added coverage for question enhancement, including empty and invalid inputs, alternative question words, synonym exclusions, and punctuation spacing.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: c7c92369-f713-4f14-bc03-a9d3748c8a20

📥 Commits

Reviewing files that changed from the base of the PR and between 2038116 and e62299b.

📒 Files selected for processing (3)

• Testing/test_question_filters.py
• backend/Generator/main.py
• backend/Generator/question_filters.py

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 1 remain after this review.

---

📝 Walkthrough

Walkthrough

The changes add an NLTK resource download, update the notation for a sentence-splitting regex, and add tests for QuestionEnhancer initialization, inputs, alternatives, synonyms, and punctuation.

Changes

Question generation

Layer / File(s)	Summary
Question generation support and tests backend/Generator/question_filters.py, backend/Generator/main.py, Testing/test_question_filters.py	The filter module downloads punkt_tab. The sentence-splitting regex uses a raw string with the same pattern. Tests cover QuestionEnhancer initialization, standard and edge-case inputs, question-word alternatives, synonym exclusions, and punctuation spacing.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to e6229

No new merge-blocking behavior is established by these changes; the PR is ready for normal checks.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely identifies all three main changes: the punkt_tab download, the regex fix, and the QuestionEnhancer unit tests.
Docstring Coverage	✅ Passed	Docstring coverage is 92.31% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 13 functions across 3 files.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[gitcordapp]

Link your account with Gitcord

Thanks for opening this PR, @Rajratna-D!

To receive Discord notifications and contributor tracking for this organization:

1. Join Discord: https://discord.gg/hjUhu33uAn
2. In Discord, run /link Rajratna-D
3. Paste the verification code into your GitHub bio (or a public gist)
4. Click Verify in Discord (or run /verify-link Rajratna-D)

Once linked, Gitcord can notify you about reviews, merges, and more.

— Posted by Gitcord