Skip to content

SchemaObjectCache refactor #2307

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
Yuqi-Du wants to merge 19 commits into
base: main
Choose a base branch
from
Draft

SchemaObjectCache refactor #2307

Yuqi-Du wants to merge 19 commits into from

Conversation

@Yuqi-Du
Copy link
Contributor

@Yuqi-Du Yuqi-Du commented Dec 15, 2025 •
edited
Loading

What this PR does:

Major change

  • Change schemaCache to schemaObjectCache(two level cache -> one level cache), extended as TTL cache
  • identified by new SchemaObjectIdentifier as key.
  • Cache access change to async non-blocking style.

Notable points

  • delete files (TableBaseSchemaCache.java and SchemaCache.java), so the old two level cache files
  • need to break the circular dependency for both CqlSessionCacheSupplier.java and SchemaObjectCacheSupplier.java
  • null handling for possible keyspace and table, since schemaObjectIdentifier are used across DB/Keyspace/Table command.
  • update all tests, modify mocks for schemaObject and metadata, use schemaObjectIdentifier instead of names

Which issue(s) this PR fixes:
Fixes #1791

Checklist

  • Changes manually tested
  • Automated Tests added/updated
  • Documentation added/updated
  • CLA Signed: DataStax CLA

amorton and others added 4 commits December 3, 2025 08:46
@amorton
WIP - initial work to improve the tenant etc
3a716ed
@amorton
initial with changes based on tenant changes
fb95600
@Yuqi-Du
Merge remote-tracking branch 'origin/main' into ajm/fix-1791-schema-c…
63e447b 
…ache-refactor

# Conflicts:
#	src/main/java/io/stargate/sgv2/jsonapi/JsonApiStartUp.java
#	src/main/java/io/stargate/sgv2/jsonapi/api/request/EmbeddingCredentials.java
#	src/main/java/io/stargate/sgv2/jsonapi/api/request/EmbeddingCredentialsResolver.java
#	src/main/java/io/stargate/sgv2/jsonapi/api/request/HeaderBasedEmbeddingCredentialsResolver.java
#	src/main/java/io/stargate/sgv2/jsonapi/api/request/RequestContext.java
#	src/main/java/io/stargate/sgv2/jsonapi/api/request/RerankingCredentials.java
#	src/main/java/io/stargate/sgv2/jsonapi/api/request/UserAgent.java
#	src/main/java/io/stargate/sgv2/jsonapi/api/request/tenant/Tenant.java
#	src/main/java/io/stargate/sgv2/jsonapi/api/request/tenant/TenantFactory.java
#	src/main/java/io/stargate/sgv2/jsonapi/api/v1/CollectionResource.java
#	src/main/java/io/stargate/sgv2/jsonapi/api/v1/KeyspaceResource.java
#	src/main/java/io/stargate/sgv2/jsonapi/config/DatabaseType.java
#	src/main/java/io/stargate/sgv2/jsonapi/logging/LoggingMDCContext.java
#	src/main/java/io/stargate/sgv2/jsonapi/metrics/JsonProcessingMetricsReporter.java
#	src/main/java/io/stargate/sgv2/jsonapi/metrics/MetricsTenantDeactivationConsumer.java
#	src/main/java/io/stargate/sgv2/jsonapi/metrics/TenantRequestMetricsFilter.java
#	src/main/java/io/stargate/sgv2/jsonapi/service/cqldriver/CQLSessionCache.java
#	src/main/java/io/stargate/sgv2/jsonapi/service/cqldriver/CqlSessionFactory.java
#	src/main/java/io/stargate/sgv2/jsonapi/service/cqldriver/TenantAwareCqlSessionBuilder.java
#	src/main/java/io/stargate/sgv2/jsonapi/service/cqldriver/executor/CommandQueryExecutor.java
#	src/main/java/io/stargate/sgv2/jsonapi/service/cqldriver/executor/SchemaCache.java
#	src/main/java/io/stargate/sgv2/jsonapi/service/cqldriver/executor/TableBasedSchemaCache.java
#	src/main/java/io/stargate/sgv2/jsonapi/service/embedding/DataVectorizerService.java
#	src/main/java/io/stargate/sgv2/jsonapi/service/operation/reranking/RerankingMetrics.java
#	src/main/java/io/stargate/sgv2/jsonapi/service/processor/CommandLog.java
#	src/main/java/io/stargate/sgv2/jsonapi/service/processor/MeteredCommandProcessor.java
#	src/main/java/io/stargate/sgv2/jsonapi/service/provider/ModelUsage.java
#	src/main/java/io/stargate/sgv2/jsonapi/service/reranking/gateway/RerankingEGWClient.java
#	src/main/java/io/stargate/sgv2/jsonapi/service/resolver/CommandResolver.java
#	src/main/java/io/stargate/sgv2/jsonapi/service/resolver/CreateCollectionCommandResolver.java
#	src/main/java/io/stargate/sgv2/jsonapi/util/DynamicTTLCache.java
#	src/test/java/io/stargate/sgv2/jsonapi/api/request/UserAgentTests.java
#	src/test/java/io/stargate/sgv2/jsonapi/api/request/tenant/TenantFactoryTests.java
#	src/test/java/io/stargate/sgv2/jsonapi/api/request/tenant/TenantTests.java
#	src/test/java/io/stargate/sgv2/jsonapi/service/embedding/operation/EmbeddingGatewayClientTest.java
@Yuqi-Du
Merge from Main(Mostly latest Tenant refactoring work)
25d3521
@Yuqi-Du Yuqi-Du changed the title [workAjm/fix 1791 schema cache refactor Schema cache refactor Dec 15, 2025
Yuqi-Du added 4 commits January 7, 2026 09:14
@Yuqi-Du
Merge remote-tracking branch 'origin/main' into ajm/fix-1791-schema-c…
e75384e 
…ache-refactor

# Conflicts:
#	src/main/java/io/stargate/sgv2/jsonapi/api/model/command/builders/FilterClauseBuilder.java
#	src/main/java/io/stargate/sgv2/jsonapi/api/model/command/builders/SortClauseBuilder.java
@Yuqi-Du
clean up for collectionSchemaObject, schema type coverage UDT, INDEX …
4cfcce9 
…UNKNOWN_COLLECTION_OR_TABLE, convert all SchemaObjectName to identifier usage
@Yuqi-Du
remove DBRequestContext, use RequestContext
bada240
@Yuqi-Du
unit test compling fixes
bfb0328
@Yuqi-Du Yuqi-Du changed the title Schema cache refactor SchemaObjectCache refactor Jan 9, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Jan 13, 2026 •
edited
Loading

📉 Unit Test Coverage Delta vs Main Branch

Metric Value
Main Branch 50.61%
This PR 50.57%
Delta 🔴 -0.05%
⚠️ Coverage decreased

@github-actions
Copy link
Contributor

github-actions bot commented Jan 13, 2026 •
edited
Loading

Unit Test Coverage Report

Overall Project 50.57% -1.19% 🍏
Files changed 50.14% ❌

File Coverage
ClauseResolver.java 100% 🍏
JsonNamedValue.java 100% 🍏
DeleteCollectionCollectionOperation.java 100% 🍏
DeleteCollectionOperation.java 100% 🍏
SchemaObjectType.java 100% 🍏
TableUpdateSetResolver.java 100% 🍏
TableUpdateOperatorResolver.java 100% 🍏
TableUpdateUnsetResolver.java 100% 🍏
TableUpdatePullAllResolver.java 100% 🍏
TableUpdateAnalyzer.java 100% 🍏
UdtNamingRule.java 100% 🍏
IndexNamingRule.java 100% 🍏
KeyspaceNamingRule.java 100% 🍏
CollectionNamingRule.java 100% 🍏
SchemaObjectNamingRule.java 100% 🍏
TableNamingRule.java 100% 🍏
SchemaException.java 100% 🍏
SchemaObjectIdentifier.java 99.35% -0.65% 🍏
DefaultDriverExceptionHandler.java 97.65% 🍏
ReadAndUpdateCollectionOperation.java 96.87% 🍏
CreateNamespaceCommandResolver.java 95.92% 🍏
CqlNamedValueContainerFactory.java 95.83% 🍏
CountCollectionOperation.java 95.76% 🍏
CqlSessionCacheSupplier.java 95.4% -4.6% 🍏
InsertCollectionOperation.java 94.98% 🍏
WriteableTableRowBuilder.java 94.98% 🍏
WhereCQLClauseAnalyzer.java 94.35% 🍏
TableFilterClauseBuilder.java 93.62% 🍏
CQLSessionCache.java 90.68% 🍏
Task.java 86.73% 🍏
FilterResolver.java 86.49% 🍏
CommandContext.java 85.56% -5.45% 🍏
WriteableTableRow.java 83.75% 🍏
DeleteCollectionCommandResolver.java 83.33% 🍏
DynamicTTLCache.java 83.01% 🍏
TableUpdatePushResolver.java 82.87% 🍏
DropNamespaceCommandResolver.java 81.82% 🍏
FilterClauseBuilder.java 80.05% 🍏
ReadDBTask.java 79.9% 🍏
FindCollectionOperation.java 79.16% -2.51% 🍏
CqlIdentifierUtil.java 78.95% -5.26% 🍏
TableSchemaObject.java 78.33% 🍏
SchemaObjectCache.java 77.14% -22.86% 🍏
TableBasedSchemaObject.java 75.76% -24.24% 🍏
SortClauseBuilder.java 75.57% 🍏
NamedValue.java 74.8% 🍏
MapSetListTableFilter.java 72.41% 🍏
CreateCollectionOperation.java 67.98% -4.68% ❌
ErrorFormatters.java 65.36% -12.14% ❌
TableWhereCQLClause.java 63.96% 🍏
CqlNamedValue.java 59.33% 🍏
TableFilter.java 56% 🍏
NativeTypeTableFilter.java 55.27% 🍏
BaseTask.java 53.8% 🍏
JsonNamedValueContainerFactory.java 53.33% 🍏
FindEmbeddingProvidersCommandResolver.java 52.63% 🍏
DataVectorizer.java 51.37% -1.19% ❌
InsertOperationPage.java 50% 🍏
DBTask.java 48.75% -1.88% ❌
SchemaObject.java 47.62% -52.38% ❌
CommandResolver.java 42.35% -8.63% ❌
DatabaseSchemaObject.java 40% -60% ❌
TableFilterResolver.java 38.87% 🍏
TableUpdateResolver.java 37.05% 🍏
Recordable.java 31.96% 🍏
KeyspaceSchemaObject.java 30.77% -69.23% ❌
ApiVectorIndex.java 25.46% 🍏
FindCollectionsCollectionOperation.java 22.14% -22.14% ❌
TaskGroup.java 20.68% 🍏
InsertAttempt.java 18.31% 🍏
OperationAttempt.java 17.39% 🍏
TruncateCollectionOperation.java 16.39% -39.34% ❌
InTableFilter.java 13.33% 🍏
CollectionSchemaObject.java 8.68% -18.7% ❌
IndexFactoryFromIndexDesc.java 7.89% 🍏
ApiRegularIndex.java 7.09% 🍏
SchemaObjectFactory.java 6.3% -93.7% ❌
TableProjection.java 2.29% 🍏
ApiTextIndex.java 0% 🍏
ListIndexesCommandResolver.java 0% 🍏
TableReadDBOperationBuilder.java 0% 🍏
CreateTextIndexCommandResolver.java 0% 🍏
FindRerankingProvidersCommandResolver.java 0% 🍏
DropKeyspaceCommandResolver.java 0% 🍏
FindKeyspacesCommandResolver.java 0% 🍏
DropIndexCommandResolver.java 0% 🍏
FindCommandResolver.java 0% 🍏
DropTableCommandResolver.java 0% 🍏
CreateCollectionCommandResolver.java 0% 🍏
AlterTableCommandResolver.java 0% 🍏
DeleteOneCommandResolver.java 0% 🍏
DeleteManyCommandResolver.java 0% 🍏
CreateVectorIndexCommandResolver.java 0% 🍏
InsertOneCommandResolver.java 0% 🍏
CreateTableCommandResolver.java 0% 🍏
CreateTypeCommandResolver.java 0% 🍏
FindCollectionsCommandResolver.java 0% 🍏
CreateIndexCommandResolver.java 0% 🍏
ListTablesCommandResolver.java 0% 🍏
CreateKeyspaceCommandResolver.java 0% 🍏
DropTypeCommandResolver.java 0% 🍏
FindOneCommandResolver.java 0% 🍏
AlterTypeCommandResolver.java 0% 🍏
UpdateOneCommandResolver.java 0% 🍏
ListTypesCommandResolver.java 0% 🍏
InsertManyCommandResolver.java 0% 🍏
FindNamespacesCommandResolver.java 0% 🍏
CommandProcessor.java 0% -1.67% ❌
MeteredCommandProcessor.java 0% -6.6% ❌
DataVectorizerService.java 0% 🍏
CommandQueryExecutor.java 0% -12.6% ❌
GeneralResource.java 0% -75.56% ❌
CollectionResource.java 0% -32.14% ❌
KeyspaceResource.java 0% -76.6% ❌
EstimatedDocumentCountCollectionOperation.java 0% -25% ❌
RerankingTask.java 0% 🍏
RerankingMetrics.java 0% -11.21% ❌
RerankingTaskPage.java 0% 🍏
FindRerankingProvidersOperation.java 0% 🍏
IntermediateCollectionReadTask.java 0% -5.71% ❌
RerankingTaskBuilder.java 0% 🍏
SchemaObjectCacheSupplier.java 0% ❌
UnscopedSchemaObjectIdentifier.java 0% ❌
UpdateResolver.java 0% 🍏
CommandTarget.java 0% 🍏
CollectionSettingsV1Reader.java 0% -6.19% ❌
CollectionSettingsV0Reader.java 0% 🍏
TableSortClauseBuilder.java 0% 🍏
EmbeddingOperationFactory.java 0% 🍏
EmbeddingTask.java 0% 🍏
EmbeddingTaskGroupBuilder.java 0% 🍏
EmbeddingTaskBuilder.java 0% 🍏
TaskBuilder.java 0% 🍏
TaskOperation.java 0% 🍏
CompositeTaskInnerPage.java 0% 🍏
CompositeTaskOuterPage.java 0% 🍏
TaskAndDeferrables.java 0% 🍏
TaskAccumulator.java 0% 🍏
TaskPage.java 0% 🍏
CompositeTaskOperationBuilder.java 0% 🍏
DBTaskPage.java 0% 🍏
TaskGroupAndDeferrables.java 0% 🍏
CompositeTask.java 0% 🍏
AlterTableDBTaskBuilder.java 0% 🍏
CreateTypeDBTask.java 0% -6.45% ❌
DropTableExceptionHandler.java 0% 🍏
DropIndexExceptionHandler.java 0% 🍏
DropTypeExceptionHandler.java 0% 🍏
TableSimilarityFunction.java 0% 🍏
CreateIndexDBTaskBuilder.java 0% 🍏
CreateTableDBTaskBuilder.java 0% 🍏
AlterTypeDBTaskBuilder.java 0% 🍏
CreateIndexExceptionHandler.java 0% 🍏
TableDriverExceptionHandler.java 0% 🍏
CreateTableDBTask.java 0% -3.47% ❌
DropTableDBTaskBuilder.java 0% 🍏
DropIndexDBTask.java 0% -20% ❌
DropIndexDBTaskBuilder.java 0% 🍏
KeyspaceDriverExceptionHandler.java 0% 🍏
CreateTypeExceptionHandler.java 0% 🍏
TableInsertDBTaskBuilder.java 0% 🍏
DropTableDBTask.java 0% -14.63% ❌
TableInsertDBTask.java 0% 🍏
TableDeleteDBTaskBuilder.java 0% 🍏
DropTypeDBTaskBuilder.java 0% 🍏
CreateIndexDBTask.java 0% 🍏
CreateTypeDBTaskBuilder.java 0% 🍏
AlterTableExceptionHandler.java 0% 🍏
DropTypeDBTask.java 0% -14.63% ❌
TableMutationOperation.java 0% 🍏
CreateTableExceptionHandler.java 0% 🍏
TableReadDBTaskBuilder.java 0% 🍏
TableOperation.java 0% 🍏
AlterTypeExceptionHandler.java 0% 🍏
AlterTypeDBTask.java 0% -5.94% ❌
AlterTableDBTask.java 0% 🍏
TableInsertValuesCQLClause.java 0% 🍏
SchemaDBTask.java 0% 🍏
OperationAttemptContainer.java 0% 🍏
UpdateDBTask.java 0% 🍏
GenericOperation.java 0% -3.25% ❌
DeleteDBTask.java 0% 🍏
OperationAttemptAccumulator.java 0% 🍏
ListTypesDBTask.java 0% 🍏
DeleteDBTaskPage.java 0% 🍏
MetadataDBTaskPage.java 0% 🍏
ListIndexesDBTask.java 0% -5.94% ❌
OperationAttemptPage.java 0% 🍏
Operation.java 0% 🍏
TruncateDBTask.java 0% 🍏
TruncateDBTaskPage.java 0% 🍏
MetadataDBTask.java 0% -4.4% ❌
InsertDBTaskPage.java 0% 🍏
UpdateDBTaskBuilder.java 0% 🍏
ReadDBTaskPage.java 0% 🍏
UpdateDBTaskPage.java 0% 🍏
ListTablesDBTask.java 0% -15.94% ❌
SchemaDBTaskPage.java 0% 🍏
InsertDBTask.java 0% 🍏
OperationAttemptPageBuilder.java 0% 🍏
TableMemorySortClauseResolver.java 0% 🍏
TableCqlSortClauseResolver.java 0% 🍏
TableSortClauseResolver.java 0% 🍏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Refactor the NamespaceCache and SchemaCache classes

3 participants

@Yuqi-Du @amorton