Spring Boot AI + Azure OpenAI + Function Callback Example

In previous tutorial we had implemented Spring Boot + Azure OpenAI Hello World Example.

Also in another previous tutorial we had implemented Spring Boot + Azure OpenAI + RAG to provide reference text using vector database to openai.
We use RAG (Retrieval-Augmented Generation) for -

• Augmenting the LLM's knowledge by retrieving relevant external information before generating a response
• Helping expand the model's contextual knowledge beyond its training data
• Typically used to provide up-to-date or domain-specific information
• Focuses on improving the content and accuracy of the generated response

However it may happen that openai may need access to some real time data. Like if some question is asked about the stock market then openai should be able to access realtime stock information. This is where callback comes into picture.
We use callbacks for -

• Provide a way to intercept and modify the LLM's generation process
• Allow for real-time monitoring, logging, or modification of the model's output
• Enable additional processing or side effects during generation

For this tutorial we will be making use of cryptocurrency example. Suppose a user asks some question about bitcoin, then openai will need to know about current bitcoin price during real time. Providing this realtime info can achieved using function callback.

Implementation

Download the source code we had implemented for Spring Boot + Azure OpenAI Hello World Example. If we start this example and ask it the question - How is Bitcoin performing today? we do not get proper response as we can see that openai does not have currect data about bitcoin.

We need to provide openai with realtime information about bitcoin price to better answer this question. So we will be making use of Coin Market Cap which will provide real time bitcoin and other crypto data. We will write a function callback which will make a call to Coin Market Cap to get real time cryto data and provide this function callback to openai. So whenever OpenAi will need real time crypto data it will make use of this function callback.
For this first we will need to create an account for Coin Market Cap. Once account is created log in and get the api key.

We can now use coinmarketcap to get crypto real time data as follows-

Use this in the properties file as follows-

azure.openai.api.key=66T7YqY4CaqXB9fdfdfD6q1RTz9p45goTXeVAJPpWKERdclRxRLgJQQJ99BBACYeBjFXJ3w3AAABACOGmCSx
azure.openai.endpoint=https://javainuse-service.openai.azure.com/
azure.openai.deployment.model.id=gpt-4o
azure.openai.embedding.model.id=text-embedding-3-small
coinmarketcap.api.key=5a5537c70-t7ac-4b40-a8da-1fb7dsdsdbc3458

Let us modify the prompt controller to create this function callback and provide it to openai.

• This is a Spring Boot REST controller for handling cryptocurrency-related queries using Azure OpenAI's function calling capability.
• The controller integrates two external services:
  • Azure OpenAI API for AI-powered responses
  • CoinMarketCap API for retrieving cryptocurrency data
• Configuration is handled through Spring's @Value annotations to inject:
  • Azure OpenAI API key, endpoint, and deployment model ID
  • CoinMarketCap API key
• The main endpoint "/answer" accepts POST requests with a JSON body containing a question.
• The AI function calling process follows these steps:
  • Initializes an OpenAI client with Azure credentials
  • Sets up a system prompt that instructs the AI to help with cryptocurrency information
  • Defines a "getCryptoPrice" function with a schema specifying a cryptocurrency symbol parameter
  • Sends the user's question to Azure OpenAI
• When Azure OpenAI decides to call the getCryptoPrice function:
  • The controller extracts the cryptocurrency symbol from the function arguments
  • Makes an API call to CoinMarketCap to retrieve real-time data
  • Processes the response to extract key metrics (price, volume, market cap, etc.)
  • Returns the data back to Azure OpenAI as a function response
• Azure OpenAI then generates a final response incorporating the cryptocurrency data.
• Error handling is implemented throughout the process to catch and report any issues.
• The approach demonstrates a practical implementation of AI function calling, allowing the AI model to request specific external data when needed to answer user queries.

package com.azure.ai.openai.usage;

import com.azure.ai.openai.OpenAIClient;
import com.azure.ai.openai.OpenAIClientBuilder;
import com.azure.ai.openai.models.*;
import com.azure.core.credential.AzureKeyCredential;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpMethod;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.client.RestTemplate;

import java.util.*;

@RestController
public class PromptController {
    @Value("${azure.openai.api.key}")
    private String azureOpenaiKey;

    @Value("${azure.openai.endpoint}")
    private String endpoint;

    @Value("${azure.openai.deployment.model.id}")
    private String deploymentOrModelId;
    
    @Value("${coinmarketcap.api.key}")
    private String coinMarketCapApiKey;
    
    private final String CRYPTO_BASE_URL = "https://pro-api.coinmarketcap.com/v1/cryptocurrency/quotes/latest";
    
    private final RestTemplate restTemplate = new RestTemplate();
    private final ObjectMapper objectMapper = new ObjectMapper();

    @PostMapping("/answer")
    public List<String> getMethodName(@RequestBody PromptQuestion promptQuestion) {
        List<String> responseList = new ArrayList<>();
        try {
            OpenAIClient client = new OpenAIClientBuilder()
                    .endpoint(endpoint)
                    .credential(new AzureKeyCredential(azureOpenaiKey))
                    .buildClient();
            
            List<ChatMessage> messages = new ArrayList<>();
            messages.add(new ChatMessage(ChatRole.SYSTEM, 
                "You are an AI assistant that helps people find information about cryptocurrencies. " +
                "You can use the getCryptoPrice function to get latest information about cryptocurrencies."));
            messages.add(new ChatMessage(ChatRole.USER, getPrompt(promptQuestion)));

            // Define function for crypto price lookup
            List<FunctionDefinition> functions = new ArrayList<>();
            
            // Parameters for getCryptoPrice function
            Map<String, Object> symbolParams = new HashMap<>();
            symbolParams.put("type", "string");
            symbolParams.put("description", "The cryptocurrency symbol (e.g., BTC, ETH, XRP)");
            
            Map<String, Object> functionProperties = new HashMap<>();
            functionProperties.put("symbol", symbolParams);
            
            Map<String, Object> functionSchema = new HashMap<>();
            functionSchema.put("type", "object");
            functionSchema.put("required", List.of("symbol"));
            functionSchema.put("properties", functionProperties);
            
            functions.add(new FunctionDefinition("getCryptoPrice")
                    .setDescription("Get the latest price and information about a cryptocurrency")
                    .setParameters(functionSchema));
            
            ChatCompletionsOptions options = new ChatCompletionsOptions(messages)
                    .setTemperature(0.7)
                    .setTopP(0.95)
                    .setMaxTokens(800)
                    .setFunctions(functions);
            
            ChatCompletions completions = client.getChatCompletions(deploymentOrModelId, options);
            
            for (ChatChoice choice : completions.getChoices()) {
                ChatMessage responseMessage = choice.getMessage();
                System.out.println(responseMessage.getContent());
                if (responseMessage.getFunctionCall() != null) {
                    FunctionCall functionCall = responseMessage.getFunctionCall();
                    
                    System.out.println("Function Called: " + functionCall.getName());
                    System.out.println("Function Arguments: " + functionCall.getArguments());
                    
                    if ("getCryptoPrice".equals(functionCall.getName())) {
                        try {
                            JsonNode args = objectMapper.readTree(functionCall.getArguments());
                            String symbol = args.get("symbol").asText();
                            
                            String cryptoData = getCryptoPriceData(symbol);
                            
                            messages.add(responseMessage);
                            
                            messages.add(new ChatMessage(ChatRole.FUNCTION, cryptoData)
                                    .setName("getCryptoPrice"));
                            
                            ChatCompletionsOptions finalOptions = new ChatCompletionsOptions(messages)
                                    .setTemperature(0.7)
                                    .setMaxTokens(800);
                            
                            ChatCompletions finalCompletions = client.getChatCompletions(deploymentOrModelId, finalOptions);
                            
                            for (ChatChoice finalChoice : finalCompletions.getChoices()) {
                                responseList.add(finalChoice.getMessage().getContent().trim());
                            }
                        } catch (Exception e) {
                            responseList.add("Error processing cryptocurrency request: " + e.getMessage());
                        }
                    }
                } else {
                    // No function call, just return the content
                    responseList.add(responseMessage.getContent().trim());
                }
            }
        } catch (Exception ex) {
            ex.printStackTrace();
            responseList.add("Exception Occurred: " + ex.getMessage());
        }
        return responseList;
    }
    
    private String getCryptoPriceData(String symbol) {
        try {
            HttpHeaders headers = new HttpHeaders();
            headers.set("X-CMC_PRO_API_KEY", coinMarketCapApiKey);
            headers.set("Accept", "application/json");
            
            String url = CRYPTO_BASE_URL + "?symbol=" + symbol.toUpperCase();
            
            ResponseEntity<String> response = restTemplate.exchange(
                url, 
                HttpMethod.GET, 
                new HttpEntity<>(headers),
                String.class
            );
            
            JsonNode rootNode = objectMapper.readTree(response.getBody());
            JsonNode dataNode = rootNode.path("data").path(symbol.toUpperCase());
            
            if (dataNode.isMissingNode()) {
                return "{\"error\": \"Cryptocurrency not found\"}";
            }
            
            Map<String, Object> resultMap = new HashMap<>();
            resultMap.put("name", dataNode.path("name").asText());
            resultMap.put("symbol", dataNode.path("symbol").asText());
            resultMap.put("price", dataNode.path("quote").path("USD").path("price").asDouble());
            resultMap.put("volume_24h", dataNode.path("quote").path("USD").path("volume_24h").asDouble());
            resultMap.put("percent_change_1h", dataNode.path("quote").path("USD").path("percent_change_1h").asDouble());
            resultMap.put("percent_change_24h", dataNode.path("quote").path("USD").path("percent_change_24h").asDouble());
            resultMap.put("percent_change_7d", dataNode.path("quote").path("USD").path("percent_change_7d").asDouble());
            resultMap.put("market_cap", dataNode.path("quote").path("USD").path("market_cap").asDouble());
            resultMap.put("last_updated", dataNode.path("quote").path("USD").path("last_updated").asText());
            
            return objectMapper.writeValueAsString(resultMap);
            
        } catch (Exception e) {
            e.printStackTrace();
            return "{\"error\": \"" + e.getMessage() + "\"}";
        }
    }

    private String getPrompt(PromptQuestion promptQuestion) {
        String input = promptQuestion.getQuestion().trim();        
        return input;
    }
}

For above the equivalent json schema for getCryptoPrice which is passed to openai will be -

{
  "name": "getCryptoPrice",
  "description": "Get the latest price and information about a cryptocurrency",
  "parameters": {
    "type": "object",
    "required": ["symbol"],
    "properties": {
      "symbol": {
        "type": "string",
        "description": "The cryptocurrency symbol (e.g., BTC, ETH, XRP)"
      }
    }
  }
}

If we now again ask the same question - How is Bitcoin performing today? we get proper response from OpenAI.

Download Source Code

Download it -
Spring Boot AI + Azure OpenAI + Function Callback Example