[WIP] Revisiting JVector codec #15472
abernardi597
wants to merge
87 commits into
apache:main
from
abernardi597:jvector
Add jvector-4.0.0-rc.5 dependency
7577d1e0
[build-fails] Checkout opensearch jvector codec
7843b82f
Fix license headers
c65c4679
Run tidy
cf2aa852
Fix package declarations
3b98cd86
Remove logging
8f00f877
Remove KNNCounter stats
2e2f5640
Fix references to missing KNNConstants
a47ca910
Remove lombok.Value annotation from JVectorKnnCollector
c30a0702
Fix AcceptDocs param in JVectorKnnFloatVectorQuery
d9e5ba31
Fix AcceptDocs param in JVectorReader
6b178d8f
Fix static imports of SIMD_POOL*
9604146d
Remove Lombok.Getter from JVectorWriter
caba8d9f
Remove lombok annotations from VectorIndexFieldMetadata
0fcac22f
Fix illegal access to PerFieldKnnVectorsFormat.FieldReader
384cde85
Fix references to getCodec
4a563928
Fix references to TestUtils.generateRandomVectors
585e7772
Fix ThreadLeakFilters in test
8ddcddb1
Fix missing @Override
25cd540a
Remove unused members
ee5ed2d7
Fix unqualified javadoc
3c47063c
Suppress cases-omitted from switch expression
af5e0bed
Add basic javadocs for classes without
4b2beb89
Fix forbiddenApis error
f97bfc51
Rename KNNJVectorTests
17e211f5
Fix missing @Test annotations
c2c8082f
Use JVectorSearchStrategy to plumb search parameters to JVectorReader
d26e1f67
Use IntUnaryOperator for numberOfSubspacesPerVetorSupplier
7435bf9e
Fix missed call to KnnCollector.incVisitedCount
ddbae421
Skip search altogether when graph is empty
e5b7619a
Fix multiple fieldEntry lookups
41856a6c
Do not write empty graph
d5b6d4ff
Fix merging other formats
7726aa04
Remove incremental graph build
6cb01839
Fix leading readers indexing error
47080f5c
Fix PQ refinement
34f0685d
Dry out JVectorFloatVectorValues
31b973e4
Fix JVectorFloatVectorValues VectorScorers
10a42873
Fix merging empty graphs
65439575
Pull dimension from merged fieldInfo
5f069189
Fix imports
7cc84799
Use RemappedRandomAccessVectorValues to abstract graphNodeIdsToRavvOrds
9f4a5571
Fix GraphNodeIdToDocMap deserialization
a4d32c43
Remove type parameter from FieldWriter
765cfaef
Fix missing copy on FieldWriter.addValue
19c9c3c0
Move DocIndexIterator logic to GraphNodeIdToDocMap
8a9d5ab5
Replace OnDiskGraphIndex.View.liveBits() check with assertion
64aefe5f
Extract FieldWriter.randomAccessVectorValues to method
6defb82c
Use DocsWithFieldSet instead of List<Integer> in FieldWriter
ae5cb161
Add GraphNodeIdToDocMap constructor from DocsWithFieldSet
19817295
Fix sort sort-on-flush logic
3171addc
Fix JVectorFloatVectorValues.ordToDoc
38e0b47a
Pull out merging functionality into method
b2340dd2
Remove synchronized from RandomAccessMergedFloatVectorValues
b2e587b7
Add BaseKnnVectorsFormatTestCase to TestJVectorFormat
58776c31
Remove explicit SIMD pools
fa2908e2
Replace VectorSimilarityMapper with simple switch
3ad86d85
fixup! Remove explicit SIMD pools
b5255b37
Make RandomAccessVectorValuesOverVectorValues thread-safe
5e2a8a0f
Use OrdinalMapper for sorting index
d3aeeb5e
Don't write metadata for empty graphs
d5f8bb25
Fix JVector data slicing
d7b40bad
Fix JVectorRandomAccessReader imports
7e21ddfd
Remove pqCodebooksReaderSupplier
3ff7e4c1
Remove indexReaderSupplier
44847ec3
Remove useless offset/length fields
8aa2cb17
Use proper index slicing
af695713
Remove segmentName arg from getGraph
b03d201a
Improve primary index file handling on write
4267ec0e
Remove FieldInfos field
f516762c
Fixup improve missing graph
df18dc8c
Write all fields to the same file
bbdcee5d
fixup! Fix missing @Override
f90eacfb
Fix remove sorting
cecc473d
Avoid extra copies in RandomAccessMergedFloatVectorValues.getVector()
1d2b4e83
Move PQ encoding to FieldWriter.addValue instead of flush
cfbf4c2e
Use bulk read methods where possible (requires JVector byte-order pull
dc3dba60
Move BuildScoreProvider to FieldWriter
9ef6dd91
Use ImmutableGraphIndex for writeField
4b25b4bf
Build graph while adding docs
54959f50
Support maxDegrees per-layer
ab9e92a5
Start largestQuantizedReaderIndex at -1
528b6d9d
fixup! Move PQ encoding to FieldWriter.addValue instead of flush
fe3c7832
Don't re-use PQ codebooks
029a116f
fixup! fixup! Move PQ encoding to FieldWriter.addValue instead of flush
5976164a
Small re-organize PQ merge
d2738f6a
Merge branch 'apache:main' into jvector
1ddfe8f9
uschindler
commented
on 2025-12-13
Reviewers
Assignees
No one assigned
Labels
module:sandbox
module:build-infra
Milestone
No milestone
Login to write a write a comment.
Login via GitHub