Skip to content
January 11, 2025

Structured Output in Spring AI

· 4 min read · GenAI, Spring AI
  • Spring provides a way to convert outputs from Large Language Models (LLMs) into structured formats, such as JSON, for easier integration with applications.
  • StructuredOutputConverter is the key component in Spring AI for managing the transformation of LLM responses into structured objects. It converts raw text responses into user-defined formats like List, Map, or custom objects.
  • The converter helps with deserializing LLM responses, making it easy to transform complex textual output into structured data types that can be used in your Spring application.
  • The converter supports various output types, including single objects, lists, and maps. You can define your own type references to control how the output is mapped to your Java types.
  • Developers can define custom formats by specifying custom data classes, allowing the LLM’s unstructured text to be mapped directly to domain-specific objects
  • By using ParameterizedTypeReference, Spring AI allows for more flexible and generic type handling, such as deserializing JSON arrays into List objects or JSON objects into a Map.
  • The converter automatically handles the extraction of relevant data from the raw response, removing the need for manual parsing or mapping
  • Spring AI currently does not support returning objects using the stream() method.
  • You must manually collect and convert the raw stream data into structured objects using the StructuredOutputConverter
  • Spring simplifies working with LLMs by automatically structuring the output in a way that can be directly used within Java applications, promoting both flexibility and ease of integration
  • Let us understand this by these code snippets
  • This code snippet gets Book details from LLM without putting it in any format . This will simply return paragraph or text
// Endpoint to generate a simple text response with book details as a string
    @GetMapping("/text")
    public String getBook() {
        // Calls the chat client to generate a list of 5 books in a plain text format
        return chatClient.prompt("Generate  5 book name with title, author,publicationYear and ISBN.")
                .call() // Make the API call
                .content(); // Extract the content (the response as a string)
    }
  • Now to get a response as a Book object with title, author , publicationYear and ISBN field, lets create a Book record
public record Book(String title, String author, int publicationYear, String ISBN) {}
  • This code snippet, will query the LLM , get the response and convert it as a Book type. The entity() method provides this functionality.
  @GetMapping("/bookOP")
    public Book getBookObject() {
        // Calls the chat client to generate one book's details (title, author, publication year, ISBN)
        // Returns the book as a Book entity
        return chatClient.prompt("Generate  1 book name with title, author, publicationYear and ISBN.")
                .call() // Make the API call
                .entity(Book.class); // Deserialize the response into a Book object
    }
  • To return a list of Book objects use ParameterizedTypeReference as given below
  // Endpoint to generate a list of Book objects (5 books)
    @GetMapping("/bookList")
    public List<Book> getBookList() {
        // Calls the chat client to generate a list of 5 books with details
        // The response is deserialized into a list of Book objects using a ParameterizedTypeReference
        return chatClient.prompt("Generate  5 book name with title, author, publicationYear and ISBN.")
                .call() // Make the API call
                .entity(new ParameterizedTypeReference<List<Book>>() {
                }); // Deserialize the response into a List<Book>
    }
  • To return a map of Book objects with a key
@GetMapping("/bookMap")
    public Map<String, Book> getBookMap() {
        // Calls the chat client to generate a map of books (keyed by "numbers") with details
        // The response is deserialized into a Map<String, Book> using a ParameterizedTypeReference
        return chatClient.prompt("Generate 5 book detail with title, author, publicationYear and ISBN, under the key name 'numbers'")
                .call() // Make the API call
                .entity(new ParameterizedTypeReference<Map<String, Book>>() {
                }); // Deserialize the response into a Map<String, Book>
    }
  • Current implementation does not provide a way to stream data , but until then you should manually convert the raw output (such as a string or unstructured data returned by the LLM) into structured Java objects. This is done using the StructuredOutputConverter, which helps you process and map the raw response to a more usable Java type.
@GetMapping("/bookListGen")
    public Flux<Book> storytWithStream(@RequestParam(defaultValue = "Generate  15 book name with title, author,publicationYear and ISBN") String message) {
        // Creates a BeanOutputConverter to convert the response into a List<Book> from the API
        var converter = new BeanOutputConverter<>(new ParameterizedTypeReference<List<Book>>() {
        });

        // Sends a prompt to the chat client, passing a dynamic message for the book details
        // The message can be customized via a query parameter
        Flux<String> flux = this.chatClient.prompt()
                .user(u -> u.text("""
                                  %s
                                  {format}
                                """.formatted(message)) // Insert the dynamic message here
                        .param("format", converter.getFormat())) // Pass any additional parameters, such as format
                .stream() // Stream the response content
                .content(); // Get the content (raw text response)

        // Collect the streamed content into a single string
        String content = flux.collectList().block().stream().collect(Collectors.joining());

        // Convert the content (raw string) into a List<Book> using the converter
        List<Book> actorFilms = converter.convert(content);

        // Return the list of books as a Flux<Book> to stream them asynchronously
        return Flux.fromIterable(actorFilms); // Convert the List<Book> into a Flux stream
    }

Source code – https://github.com/rajeshsgr/structured-op-demo-spring-ai/blob/main/README.md

Tags: , , ,

Leave a Reply