Skip to content

Fix AI service model selection#322

Open
graemerocher wants to merge 3 commits into
2.0.xfrom
issue-233-ai-service-model-selection
Open

Fix AI service model selection#322
graemerocher wants to merge 3 commits into
2.0.xfrom
issue-233-ai-service-model-selection

Conversation

@graemerocher

Copy link
Copy Markdown
Collaborator

Summary

  • configure only the chat model for non-streaming AI services and only the streaming chat model for streaming AI services
  • retain support for mixed service interfaces by configuring both models when a bean exposes both return styles
  • add regression coverage for chat-only and streaming-only AI service interfaces

Verification

  • ./gradlew :micronaut-langchain4j-core:test --tests io.micronaut.langchain4j.aiservices.AiServiceFactoryModelSelectionTest

Resolves #233

Co-Authored-By: Codex with GPT-5 <codex@openai.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates AiServiceFactory to avoid configuring both ChatModel and StreamingChatModel for AI services that only need one model type, aligning Micronaut’s wiring with LangChain4j’s expectations and preventing builder errors.

Changes:

  • Add model-selection logic that inspects an @AiService bean’s executable methods to decide whether to configure ChatModel, StreamingChatModel, or both.
  • Use LangChain4j TokenStreamAdapter SPI to classify streaming return types beyond TokenStream.
  • Add regression tests for chat-only and streaming-only AI service interfaces.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
micronaut-langchain4j-core/src/main/java/io/micronaut/langchain4j/aiservices/AiServiceFactory.java Introduces method-based model selection and conditional wiring of chat vs streaming chat models.
micronaut-langchain4j-core/src/test/java/io/micronaut/langchain4j/aiservices/AiServiceFactoryModelSelectionTest.java Adds tests ensuring only the needed model is configured for chat-only vs streaming-only services.

@graemerocher graemerocher moved this to Ready for review in 5.0.0 Release Apr 30, 2026
@graemerocher

Copy link
Copy Markdown
Collaborator Author

@copilot apply changes based on the comments in this thread

Copilot AI commented Apr 30, 2026

Copy link
Copy Markdown
Contributor

@copilot apply changes based on the comments in this thread

Applied all three review changes in f4567ca:

  1. isAssignableFrom fixisStreamingReturnType now uses TokenStream.class.isAssignableFrom(returnType.getType()) to correctly handle subtype returns.
  2. Deterministic error handling – the streaming test now captures errors in an AtomicReference<Throwable> and counts down the latch in both success and error paths, then re-throws after await().
  3. BOTH test – added configuresBothModelsForMixedServices covering a MixedAssistant interface with both String chat(String) and TokenStream streamChat(String) methods, asserting ModelSelection.BOTH and validating both call styles.

All 3 tests pass.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • ge.micronaut.io
    • Triggering command: /usr/lib/jvm/temurin-25-jdk-amd64/bin/java /usr/lib/jvm/temurin-25-jdk-amd64/bin/java --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.prefs/java.util.prefs=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED --add-opens=java.base/java.nio.charset=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.xml/javax.xml.namespace=ALL-UNNAMED --add-opens=java.base/java.time=ALL-UNNAMED --enable-native-access=ALL-UNNAMED -Xmx1g -Dfile.encoding=UTF-8 -Duser.country -Duser.language=en -Duser.variant -cp (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

@sdelamo sdelamo removed this from 5.0.0 Release May 7, 2026
@github-project-automation github-project-automation Bot moved this to Backlog in 5.1.0 Release May 7, 2026
@graemerocher graemerocher moved this from Backlog to In review in 5.1.0 Release May 27, 2026
@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: bug Something isn't working

Projects

Status: In review

Development

Successfully merging this pull request may close these issues.

AI Service interceptor configures both ChatModel and StreamingChatModel

4 participants