[arturobernalg]

JSPWIKI-1321, a follow-up of JSPWIKI-501 from the review of pull request 543.

A term of the name field with a * between other characters, such as blog*26, is now also looked for inside the words of the name, and its parts are looked for in order in different words. name:blog*26 and name:entry*26* find Weblog_blogentry_021126_1. A term that only ends in *, such as name:man*, still starts at the word. Only the name field changes.

The relevance of a result is scaled so that the best one is 100. Before, values above 100 appeared.

Tests: three new ones in SearchManagerTest fail on the code of pull request 543 and pass here. SearchManagerTest has 16 tests and all pass. Not run: the whole test suite, the cost of terms with an inner * on a large index, and the contents field.

https://issues.apache.org/jira/browse/JSPWIKI-1321

[arturobernalg]

@udittmer, this pull request covers what you asked for in the review of #543. With a page named Weblog_blogentry_021126_1, name:blog*26 and name:entry*26* now find it in my tests, and the relevance is scaled so that the best result is 100. A term that only ends in *, such as name:man* or name:entry*, still starts at the beginning of a word, so name:entry* does not find that page. I only tested it with automated tests and an in-memory index, not on a wiki with the Weblog plugin. Could you try it there and tell me whether it does what you expected?

[udittmer]

I'm not sure about this. It seems the search code is now becoming rather complex, and geared towards specific cases, while not being all that general any more. While it does now find the two cases mentioned above, and blogentry finds all blog pages, as does entry*26, entry or entry* find none, which is counterintuitive.

As to the scaling, it's now capped at 100, but the bars are not affected; see screenshots.

[arturobernalg]

@udittmer, thanks for trying it. You are right that the rule is not general: a term with an inner * is looked for inside the words, but entry and entry* still start at the beginning of a word, so they do not find Weblog_blogentry_021126_1.

Before I change anything, which rule would you expect for the name field? Should entry and entry* also find pages whose word only contains entry, so that every term of the name field matches inside the words? That would be one simple rule instead of special cases, but it would also return more pages for searches that work today, so I would measure how many before proposing it.

About the bars: the number is capped at 100 now, but the bars are drawn by GraphBar.js, which makes the smallest bar 40 px and the largest 440 px and everything else linear in between, so the cap does not change them. In your screenshot 100, 29 and 23 are about 880, 142 and 80 px wide, which is what that formula gives. Should the bar be proportional to the number (29 is 29% of the width), or is the current look fine and only the number mattered?

[udittmer]

Thinking about this some more, I actually think it's OK for "entry" not to find anything, given that it is in the middle of the word. In which case entry*26 also should not return any results.

I guess that point is debatable, and maybe just a design decision. When implementing Lucene solutions, I sometimes use porter stemming analyzers, so that searching for "entries" would also find "entry". But given the stemming operation, I wouldn't then expect to find any random string that occurs in the middle of a word.

So the question would be: should one expect to find strings in the middle of words, or just from the beginning? That is at least partly a matter of personal preference.

As to the bar chart, my expectation would be for the length of the bar to be strictly proportional to the value. So if the minimum was barely visible, that would reflect its low value.

[arturobernalg]

Thanks, that is a fair point. I measured a few searches with the code of #543 (what master has now) and with this pull request, on 11 pages: UnserBlog, four UnserBlog_blogentry pages, three Weblog_blogentry pages, Man_Page, Human_Page and Woman_Page.

On master, name:blog26 finds 0 pages but name:blog26* finds 3, and name:manpa finds 0 although name:man* finds Man_Page. With this pull request name:entry26 finds 3 pages, name:manpa* finds Man_Page, Human_Page and Woman_Page, and name:blog1 finds 6 pages instead of 5. name:entry and name:entry find nothing in both.

If matches should only start at the beginning of a word, I would drop the part of this pull request that looks inside the words, so that name:entry26 finds nothing, and make a * at the end optional, so that name:blog26 and name:blog26 both find UnserBlog_blogentry_261023_1, Weblog_blogentry_021126_1 and Weblog_blogentry_031226_1. Is that what you would expect, or should name:blog*26 find nothing?

About the bars, I will do that in a separate ticket, so that the bar is strictly proportional to the value.