new segments now support 1-hit encoding - #1337
Conversation
This commit alters the path for new segments, such that they too should now consider, and when appropriate, use the 1-hit encoding optimization.
|
First, this was the quick and dirty, minimal plausible change to get it working. It's quite possible this is wrong with a fuller understanding of what is going on. The question still remains why didn't @steveyen wire this up in the first place? Also, I put the DONOTMERGE label on this, as we may want to wait and merge this into a v2 or v3 of scorch (I don't see any backwards compat issues, but since we have new versions coming anyway, maybe ship this with that?) UPDATE: I forgot to mention that all bleve unit tests pass with this change, and the optimization test now does involve 1-hit encodings. Though this too is not a panacea, as now I suspect the test case ONLY actually tests 1-hit encodings... so we just changed which path was tested, not adding one. |
| locEncoder.Reset() | ||
| lastDocNum = 0 | ||
| lastFreq = 0 | ||
| lastNorm = 0 |
There was a problem hiding this comment.
actually these are either in the wrong place, or not necessary in this verison, as these variables go out of scope here...
1 participant
This commit alters the path for new segments, such that
they too should now consider, and when appropriate, use
the 1-hit encoding optimization.