Repository navigation
Add support for streaming #100
Description
See #8: A crucial piece of functionality still missing in our SDK is response streaming - a quite complex piece of functionality to get right, especially with the incomplete documentation of how different providers implement it. But we need it, that's for sure.
What we'll need to do is:
- Implement the primitives needed for streaming (likely some kind of stream response class, potentially a new stream method on
HttpTransporter). - Implement actual streaming for the three providers (which can fortunately be centralized in the OpenAI API compatible implementation for now).
Putting here some references for inspiration in how I implemented streaming (still with some limitations) in the AI Services plugin for WordPress:
- https://github.com/felixarntz/ai-services/blob/main/includes/Services/HTTP/Contracts/Stream_Request_Handler.php
- https://github.com/felixarntz/ai-services/blob/main/includes/Services/HTTP/Stream_Response.php
- https://github.com/felixarntz/ai-services/blob/main/includes/Services/Traits/Generative_AI_API_Client_Trait.php#L182
- https://github.com/felixarntz/ai-services/blob/main/includes/Services/Base/OpenAI_Compatible_AI_Text_Generation_Model.php#L118
There's definitely things to improve there which we should do here, but it's a decent starting point to review this and come up with effectively a v2 of that implementation.
Activity
thelovekesh commentedon Jun 18, 2026
I'm picking this up. Before implementing this, here's what I'm planning:
RequestOptionsto request a streamed response.ResponseDTO to expose the body as a stream.HttpTransporterto support streaming when sending the request and building theResponse.EventStreamParserInterface, so providers can plug in their own parser or fall back to the default.GenerativeAiResultChunkDTO and aStreamingTextGenerationModelInterfaceso models can expose streaming text generation and yield result chunks (which can be aggregated back into a finalGenerativeAiResult).PromptBuilder(and theAiClientfacade) with streaming entry points.I think this is sufficient within the scope of this SDK; individual providers can extend it based on their own SSE specifics.