Skip to content

Conversation

@kinyoklion
Copy link
Member

@kinyoklion kinyoklion commented Jan 13, 2026

BEGIN_COMMIT_OVERRIDE
chore: Add FDv2 data source interfaces.
chore: Add FDv2 polling data source.
chore: Scaffold FDv2 Data Source.
END_COMMIT_OVERRIDE


Note

Adds an experimental FDv2 data source pipeline with polling and change propagation.

  • Implements DefaultFDv2Requestor for FDv2 HTTP polling with version/state query params, ETag caching, header passthrough, and robust error handling
  • Introduces FDv2SourceResult, Initializer, Synchronizer, and SelectorSource interfaces and FDv2DataSource coordinator to run initializers then synchronizers
  • Provides PollingBase, PollingInitializerImpl, and PollingSynchronizerImpl (scheduled) to process events, map errors, and emit ChangeSet/status updates
  • Adds FDv2ChangeSetTranslator to convert FDv2 changes to DataStoreTypes.ChangeSet (flags/segments), with logging for unknown kinds
  • Adds IterableAsyncQueue utility for async result delivery
  • Extends StandardEndpoints with FDV2_POLLING_REQUEST_PATH and FDV2_STREAMING_REQUEST_PATH
  • Extensive unit tests for requestor, translator, initializer/synchronizer behaviors and queue

Written by Cursor Bugbot for commit 117b85c. This will update automatically on new commits. Configure here.

*/
public static class Status {
private final State state;
private final DataSourceStatusProvider.ErrorInfo errorInfo;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't particularly care for this type, but also we may not want what is effectively a duplicate.

* an FDv2 synchronizer produces a stream of results.
*/
public class FDv2SourceResult {
public enum State {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want running in this enum?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I shouldn't call it State. Its more like a notification of change, and it doesn't represents valid/running because that is handled by a changeset.

/**
* The data source has been instructed to disconnect and will not produce any further results.
*/
GOODBYE,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current description of GOODBYE and SHUTDOWN leaves room for me to think GOODBYE -> SHUTDOWN could be a thing.

I see the diagrams in the specific ones down below that shows that is not possible, but the enum details make it look like it could be?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GOODBYE is effectively a more specific shutdown. We didn't handle it in .Net because SDKs generally don't seem to have handled it yet, but it also is really hard to handle with data source updates.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically in .Net we had an overload broad OFF and it is hard to get the correct behavior because there were are really 2-3 different meanings. Overall we don't need to emit any data source status in most situations for something being shutdown or something being told to exit by flag delivery (but we may handle them different, I am still unsure how to handle goodbye).

SHUTDOWN means I asked the thing to shutdown.
GOODBYE means that FD or some external source asked it to disconnect. Having it different means we can decide to do something different. (I think just by entering this situation we handle most of the problems, because we can be like, cool we aren't running anymore, I don't care about this disconnect I just got from the event source.)
TERMINAL_ERROR means something has gone wrong.

@kinyoklion kinyoklion changed the title Rlamb/add fdv2 data source interfaces chore: Add FDv2 data source interfaces. Jan 14, 2026
@kinyoklion kinyoklion force-pushed the rlamb/add-fdv2-data-source-interfaces branch from dcbd9fd to 4b8313b Compare January 14, 2026 19:00
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very early scaffold that gives an idea how this stuff would connect together.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Convenience class that allows for easy mapping of producer/consumer to an iterator.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Encapsulates all the poll-specific logic which is then used by the polling synchronizer/initializer.

}
runSynchronizers();
// TODO: Handle. We have ran out of sources or we are shutting down.
});
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing startFuture completion when sources exhausted

High Severity

When all data sources are exhausted without receiving data (all initializers and synchronizers fail or lists are empty), startFuture is never completed. The TODO comment at line 99 acknowledges this needs handling but nothing is implemented. Any caller waiting on start().get() without a timeout would block forever since startFuture.complete() is only called in close() or upon successful data receipt.

Fix in Cursor Fix in Web

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep. Not implemented in this PR.

@kinyoklion kinyoklion force-pushed the rlamb/add-fdv2-data-source-interfaces branch from d7385d9 to 117b85c Compare January 15, 2026 23:16
public static FDv2SourceResult goodbye(String reason) {
// TODO: Goodbye reason.
return new FDv2SourceResult(null, new Status(State.GOODBYE, null), ResultType.STATUS);
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Method parameter reason is silently discarded

Low Severity

The goodbye(String reason) method accepts a reason parameter but completely ignores it, passing null to the Status constructor instead. When PollingBase calls FDv2SourceResult.goodbye() with the reason extracted from the protocol handler, that diagnostic information is silently discarded. This makes debugging connection termination issues harder since the server's provided reason (e.g., "service-unavailable") is lost.

Additional Locations (1)

Fix in Cursor Fix in Web

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will be handled in a future update.

@kinyoklion kinyoklion merged commit bef2500 into main Jan 15, 2026
21 checks passed
@kinyoklion kinyoklion deleted the rlamb/add-fdv2-data-source-interfaces branch January 15, 2026 23:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants