Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
- Release publishing now validates the tag/version and runs formatting, lint, and tests before publishing the verified build.

### Fixed
- Recognize exact banned-word matches before stripping affixes, so listed words such as `فلسطين` are obfuscated while retaining existing affix-based similarity matching.
- Calculate `similarityScore` lengths and edit distance after lowercasing so Unicode case expansion cannot produce negative scores or incorrect similarity ratios.
- Prevent invalid levels from hanging `tashfeer`, `tashfeerBannedWords`, and `toOldArabicAndTashfeerBannedWords`: non-number values throw `TypeError`; negative, fractional, and non-finite numbers throw `RangeError`. Random-index generation handles empty input and caps the requested count at the word length ([#35](https://github.com/Seen-Arabic/Arabic-Services-JavaScript/issues/35), [PR #36](https://github.com/Seen-Arabic/Arabic-Services-JavaScript/pull/36)).
- Attempt browser bundle uploads after a successful build even if npm publishing fails ([commit 598035a](https://github.com/Seen-Arabic/Arabic-Services-JavaScript/commit/598035a)).
Expand Down
2 changes: 1 addition & 1 deletion src/scripts/tashfeer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,7 @@ function bannedSimilarityRatio(string: string): number {
*/
function checkIfBannedWord(string: string): boolean {
const std_ratio = 70;
return bannedSimilarityRatio(removeArabicAffixes(string)) >= std_ratio;
return BANNED_WORDS.includes(string) || bannedSimilarityRatio(removeArabicAffixes(string)) >= std_ratio;
}

/**
Expand Down
8 changes: 8 additions & 0 deletions tests/scripts.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -335,6 +335,14 @@ describe('#similarityScore', () => {
});

describe('#tashfeerBannedWords', () => {
it.each(['فلسطين', 'غزة', 'حرية'])('should obfuscate %s while preserving surrounding non-banned words', (word) => {
const sentence = `هذه ${word} جميلة`;
const obfuscated = ArabicServices.tashfeerBannedWords(sentence);
expect(obfuscated).not.toContain(word);
expect(obfuscated.startsWith('هذه ')).toBe(true);
expect(obfuscated.endsWith(' جميلة')).toBe(true);
});

it('should perform tashfeer obfuscation on banned words only', () => {
const sentence = 'جيش العدو يقتل الأطفال';
const result = ArabicServices.tashfeerBannedWords(sentence);
Expand Down
Loading