OpenAI.InteractiveMedia

OpenAI.InteractiveMedia is a .NET library for interacting with OpenAI and Azure OpenAI. The library uses the OpenAI Responses API for chat interactions, including streaming, structured outputs, multimodal inputs, and tool calling.

Installation

dotnet add package OpenAI.InteractiveMedia

Supported Frameworks

• .NET 10

OpenAI Registration

To register the service in your application for OpenAI, use the following code snippet:

using OpenAI.InteractiveMedia.OpenAIMedia;

builder.Services.AddOpenAIMedia("your-api-key"); // Replace with your actual API key

Azure OpenAI Registration

To register the service in your application for Azure OpenAI, use the following code snippet:

using OpenAI.InteractiveMedia.OpenAIMedia;

builder.Services.AddOpenAIMedia(
    new OpenAIMediaConfiguration
    {
        UseAzureOpenAI = true,
        ApiKey = builder.Configuration["AzureOpenAI:ApiKey"]!,        
        AzureOpenAIEndpoint = builder.Configuration["AzureOpenAI:Endpoint"]!,
        AzureOpenAIDeploymentName = builder.Configuration["AzureOpenAI:DeploymentName"]!
    });

Runtime Configuration

Runtime provider configuration can be changed dynamically using the Configure method.

This is useful for:

• multi-tenant applications
• customer-specific OpenAI accounts
• per-request Azure OpenAI deployments

Example:

openAIMediaService.Configure(new OpenAIMediaConfiguration
{
    ApiKey = "customer-api-key",
    UseAzureOpenAI = true,
    AzureOpenAIEndpoint = "https://my-resource.openai.azure.com/",
    AzureOpenAIDeploymentName = "gpt-5-mini-production"
});

OpenAIMediaService is registered as a scoped service, ensuring runtime configuration isolation between requests/scopes.

Azure OpenAI Notes

When using Azure OpenAI:

• UseAzureOpenAI must be enabled.
• AzureOpenAIEndpoint must point to the Azure OpenAI resource.
• AzureOpenAIDeploymentName must contain the Azure deployment name.
• The Azure deployment name is used as the request model identifier.
• The Azure deployment name does not have to match the underlying model name.

Default Configuration

OpenAI Base URL

• Property: OpenAIBaseUrl
• Default value: https://api.openai.com/
• Description: Base URL of the OpenAI API.

Use Azure OpenAI

• Property: UseAzureOpenAI
• Default value: false
• Description: Determines whether Azure OpenAI-specific behavior should be used.

Azure OpenAI Endpoint

• Property: AzureOpenAIEndpoint
• Default value: null
• Description: Endpoint of the Azure OpenAI resource (for example, https://my-resource.openai.azure.com/).

Azure OpenAI Deployment Name

• Property: AzureOpenAIDeploymentName
• Default value: null
• Description: Deployment name used when Azure OpenAI is enabled.

API Version

• Property: Version
• Default value: v1
• Description: The specific version of the OpenAI API you wish to use. This allows for compatibility with future changes in the API.

Responses Endpoint

• Property: ResponsesEndpoint
• Default value: responses
• Description: Primary endpoint used by chat features in this package.

Image Generations Endpoint

• Property: ImageGenerationsEndpoint
• Default value: images/generations
• Description: Endpoint used for image generation requests.

HTTP Client Timeout

• Property: HttpClientTimeout
• Default value: 00:03:00 (3 minutes)
• Description: Maximum duration to wait for an API response.

Models Pricing

• Property: ModelsPricing
• Default value: Empty list
• Description: Pricing configuration for token cost calculation.
• Note: If using pricing, ensure to set the prices according to OpenAI Pricing.

Configuration Example

You can customize the configuration by modifying the properties in the OpenAIMediaOptions class.
To utilize the OpenAIMediaOptions with custom settings, use the following code snippet:

builder.Services.AddOpenAIMedia("your-api-key", options =>
{
    options.Version = "v1";
    options.ResponsesEndpoint = "responses";
    options.ImageGenerationsEndpoint = "images/generations";
    options.HttpClientTimeout = TimeSpan.FromMinutes(5);

    options.ModelsPricing = new List<ModelPricing>
    {
        new ModelPricing
        {
            ModelId = Gpt4oMini.Id,
            InputTokensPrice_USD_1M = 0.15m,
            CachedInputTokensPrice_USD_1M = 0.075m,
            OutputTokensPrice_USD_1M = 0.60m
        }
    };
});

In your application, you can bind configuration values from the appsettings.json file directly to the OpenAIMediaOptions class using the following:

builder.Services.AddOpenAIMedia("your-api-key", options =>
{
    builder.Configuration.GetSection("OpenAIMediaOptions").Bind(options);
});

Then, configure these settings in the appsettings.json file:

{
  "OpenAIMediaOptions": {
    "Version": "v1",
    "ResponsesEndpoint": "responses",
    "ImageGenerationsEndpoint": "images/generations",
    "HttpClientTimeout": "00:05:00",
    "ModelsPricing": [
      {
        "ModelId": "gpt-4o-mini",
        "InputTokensPrice_USD_1M": 0.15,
        "CachedInputTokensPrice_USD_1M": 0.075,
        "OutputTokensPrice_USD_1M": 0.60
      }
    ]
  }
}

Available Models

The following models are available as predefined classes for chat and image generation.

Class	Model ID	Usage
Gpt4oMini	gpt-4o-mini	Chat
Gpt4o	gpt-4o	Chat
Gpt5Mini	gpt-5-mini	Chat
Gpt54	gpt-5.4	Chat
Gpt54Mini	gpt-5.4-mini	Chat
Gpt54Nano	gpt-5.4-nano	Chat
O3Mini	o3-mini	Chat
GptImage1	gpt-image-1	Image Generation
GptImage1Mini	gpt-image-1-mini	Image Generation

Chat Model Capability Matrix

The following matrix summarizes chat-model capabilities as validated by this library (and aligned with current OpenAI model/reasoning documentation where available):

Model ID	Temperature	ReasoningEffort	Structured Outputs	Tools	Notes
gpt-4o	Supported range 0–2	No	Yes	Yes	If temperature is not set, the default is 1.
gpt-4o-mini	Supported range 0–2	No	Yes	Yes	If temperature is not set, the default is 1.
gpt-5-mini	Only default (null or 1)	Yes	Yes	Yes	temperature is fixed.
gpt-5.4	Only default (null or 1)	Yes	Yes	Yes	General-purpose flagship chat model in this package; temperature is fixed.
gpt-5.4-mini	Only default (null or 1)	Yes	Yes	Yes	temperature is fixed.
gpt-5.4-nano	Only default (null or 1)	Yes	Yes	Yes	temperature is fixed.
o3-mini	Not supported	Yes	Yes	Yes	Any non-null Temperature is rejected.

Notes (applies to all chat models in this package):

• ResponseFormat must be a structured object; string values are rejected.
• For structured outputs, both nested (type = "json_schema" with json_schema: { ... }) and flattened Responses format objects are accepted. The nested shape is normalized internally.
• Tools are configured using the Tools collection and ToolChoice request options.
• Supported tools currently include WebSearchTool and FunctionTool.

Documentation references:

• OpenAI Models overview: https://platform.openai.com/docs/models
• OpenAI Reasoning guide: https://platform.openai.com/docs/guides/reasoning

This setup allows you to manage configuration directly through appsettings.json.

Usage Example

using OpenAI.InteractiveMedia.OpenAIMedia;
using OpenAI.InteractiveMedia.Chat.AiModels;
using OpenAI.InteractiveMedia.Chat.Enums;
using OpenAI.InteractiveMedia.Chat.Models.Requests;
using OpenAI.InteractiveMedia.Chat.Models.Responses;
using OpenAI.InteractiveMedia.Images.AiModels;
using OpenAI.InteractiveMedia.Images.Models;
using OpenAI.InteractiveMedia.Exceptions;
using OpenAI.InteractiveMedia.Chat.Models.Tools;
using System.Text.Json.Nodes;

internal sealed class ExampleClass
{   
    // OpenAI media facade for interacting with chat and image API clients.
    private readonly OpenAIMediaService _openAIMedia; 

    // Constructor to inject the OpenAIMediaService dependency.
    public ExampleClass(OpenAIMediaService openAIMedia)
    {
        _openAIMedia = openAIMedia;
    }    

    // ---------------------------------------------------------------------------------------
    // Method to stream a response from chat using the WebSearchTool
    // ---------------------------------------------------------------------------------------

    public async Task StreamWebSearchResponseAsync()
    {
        var input = new List<Input>
        {
            new(Role.User,
                [new TextContent("Search the web and summarize what firstis.eu does. Include the website URL in the answer.")])
        };

        var request = new ChatRequest(input, Gpt54Mini.Instance)
        {
            Tools =
            [
                new WebSearchTool()
            ],

            ToolChoice = ToolChoiceMode.Auto,

            ReasoningEffort = ReasoningEffortLevel.Medium
        };

        await foreach (var stream in _openAIMedia.ChatClient.StreamResponseEnumerableAsync(request))
        {
            Console.Write(stream.Details.OutputText);
        }
    }

    // ---------------------------------------------------------------------------------------
    // Method to get a reply from chat
    // ---------------------------------------------------------------------------------------

    public async Task GetChatResponseWithTextAndImageAsync()
    {
        // System message content, typically used to define the assistant's behavior or provide specific instructions.
        var systemContents = new List<Content>
        {      
            new TextContent("You are an AI assistant that provides helpful and concise answers."),        
        };

        // User message content, including both text and image inputs.
        var userContents = new List<Content>
        {        
            new TextContent("What are the main differences between these two images?"),
            new ImageContentUrl("https://example.com/image1.jpg"),              // URL of an image
            new ImageContentBase64("base64_image", ImageMimeType.Png)           // Base64-encoded image
        };  

        // List of messages sent to the OpenAI chat model, including system and user roles.
        var input = new()
        {
            new Input(Role.System, systemContents),
            new Input(Role.User, userContents)
        };               

        var request = new ChatRequest(input, Gpt4oMini.Instance)
        {
            Temperature = 0.2f
        };

        try
        {
            var chatResponse = await _openAIMedia.ChatClient.GetResponseAsync(request, cancellationToken: default);
        
            Console.WriteLine($"Response Content: {chatResponse.GetResponseText()}\n" +
                              $"Model Used: {chatResponse.Details.Model}\n" +
                              $"Created At (unix): {chatResponse.Details.CreatedAt}\n" +
                              $"Usage Details: {chatResponse.TokensUsage}");
        }
        catch (OpenAiException ex)
        {
            // Handle exception            
        }
    }

    // ---------------------------------------------------------------------------------------
    // Method to get a reply from chat in the specified format (JSON Schema)
    // ---------------------------------------------------------------------------------------

    public async Task GetChatResponseWithStructuredOutputAsync()
    {    
        // The response will be returned in the following format defined by the 'responseFormat' object:
        var responseFormat = new
        {
            type = "json_schema",
            json_schema = new
            {
                name = "city_data",
                schema = new
                {
                    type = "object",
                    properties = new
                    {
                        cities = new
                        {
                            type = "array",
                            items = new
                            {
                                type = "object",
                                properties = new
                                {
                                    name = new { type = "string" },
                                    country = new { type = "string" },
                                    population = new { type = "integer" }
                                },
                                required = new[] { "name", "country", "population" },
                                additionalProperties = false
                            }
                        }
                    },
                    required = new[] { "cities" },
                    additionalProperties = false
                },
                strict = true
            }
        };

        var input = new List<Input>
        {       
            new(Role.User,
                [new TextContent("Write down the ten largest cities.")])
        };

        var request = new ChatRequest(input, Gpt4oMini.Instance)
        {
            Temperature = 0.2f,
            ResponseFormat = responseFormat
        };

        try
        {
            var chatResponse = await _openAIMedia.ChatClient.GetResponseAsync(request, cancellationToken: default);
        
            Console.WriteLine($"Response Content: {chatResponse.GetResponseText()}\n" +
                              $"Model Used: {chatResponse.Details.Model}\n" +
                              $"Created At (unix): {chatResponse.Details.CreatedAt}\n" +
                              $"Usage Details: {chatResponse.TokensUsage}");
        }
        catch (OpenAiException ex)
        {
            // Handle exception            
        }
    }

    // ---------------------------------------------------------------------------------------
    // Method to get a reply from chat using PDF file input (Responses API input_file)
    // ---------------------------------------------------------------------------------------

    public async Task GetChatResponseFromPdfAsync()
    {
        // Base64-encoded PDF content (without data: prefix)
        string pdfBase64 = "JVBERi0xLjQK...";

        var input = new List<Input>
        {
            new(Role.User,
            [
                new TextContent("Extract invoice number, supplier name, date, and total amount from this PDF."),
                new FileContentBase64(pdfBase64, FileMimeType.Pdf, "invoice-2026-01.pdf")
            ])
        };

        var request = new ChatRequest(input, Gpt54Mini.Instance)
        {
            ResponseFormat = new
            {
                type = "json_schema",
                json_schema = new
                {
                    name = "invoice_extraction",
                    schema = new
                    {
                        type = "object",
                        properties = new
                        {
                            invoiceNumber = new { type = "string" },
                            supplierName = new { type = "string" },
                            issueDate = new { type = "string" },
                            totalAmount = new { type = "number" }
                        },
                        required = new[] { "invoiceNumber", "supplierName", "issueDate", "totalAmount" },
                        additionalProperties = false
                    },
                    strict = true
                }
            }
        };

        var chatResponse = await _openAIMedia.ChatClient.GetResponseAsync(request);
        Console.WriteLine(chatResponse.GetResponseText());
    }

    // ---------------------------------------------------------------------------------------
    // Method to stream a response from chat
    // ---------------------------------------------------------------------------------------

    public async Task StreamResponseAsync()
    {
        var input = new List<Input>
        {
            new(Role.User,
                [new TextContent("Summarize the SOLID principles.")])
        };

        var request = new ChatRequest(input, Gpt5Mini.Instance);

        try
        {
            await foreach (var stream in _openAIMedia.ChatClient.StreamResponseEnumerableAsync(request))
            {
                // Output each piece of streaming content as it is received
                Console.Write(stream.GetResponseText());

                if (stream.TokensUsage is not null)
                {                
                    Console.WriteLine($"\nModel Used: {stream.Details.Model}\n" +
                                      $"Created At (unix): {stream.Details.CreatedAt}\n" +
                                      $"Usage Details: {stream.TokensUsage}");
                }
            }  
        }
        catch (OpenAiException ex)
        {
            // Handle exception            
        }      
    }

    // For non-streamed responses, use GetResponseText() directly.
    // It first uses Details.OutputText and then falls back to output[].content[] where type == "output_text".

    // ---------------------------------------------------------------------------------------
    // Method to generate an image
    // ---------------------------------------------------------------------------------------

    public async Task GenerateImageAsync()
    {         
        var imageRequest = new ImageRequest("Clean Architecture schema", DallE3.Instance, DallE3.Quality.Standard, DallE3.Size._1024x1024)
        {
            ResponseFormat = DallE3.ResponseFormat.Url
        };
        
        try
        {
            var imageResponse = await _openAIMedia.ImageClient.GenerateAsync(imageRequest, cancellationToken: default);
        
            Console.WriteLine($"Image URL: {imageResponse.Url}\n" +
                              $"Revised Prompt: {imageResponse.RevisedPrompt}\n" +
                              $"Created At: {imageResponse.CreatedAt}");
        }
        catch (OpenAiException ex)
        {
            // Handle exception            
        }
    }    
}

Function Tools Example

// ---------------------------------------------------------------------------------------
// Method demonstrating OpenAI Responses API function tools integration
// ---------------------------------------------------------------------------------------

public async Task FunctionToolsAsync()
{
    var weatherTool = new FunctionTool(
        "get_weather",
        "Gets the current weather for a city.",
        new JsonObject
        {
            ["type"] = "object",
            ["properties"] = new JsonObject
            {
                ["city"] = new JsonObject
                {
                    ["type"] = "string"
                }
            },
            ["required"] = new JsonArray("city"),
            ["additionalProperties"] = false
        });

    var input = new List<Input>
    {
        new(Role.User,
        [
            new TextContent("What is the weather in Prague?")
        ])
    };

    var request = new ChatRequest(input, Gpt54Mini.Instance)
    {
        Tools = [weatherTool],
        ToolChoice = ToolChoiceMode.Auto,
        Store = true
    };

    var response = await _openAIMedia.ChatClient.GetResponseAsync(request);

    foreach (var output in response.Details.Output)
    {
        if (output.Type != "function_call")
            continue;

        Console.WriteLine($"Tool Name: {output.Name}");
        Console.WriteLine($"Arguments: {output.Arguments}");

        // Execute your application logic here.
        // Example:
        // var result = weatherService.GetWeather(...);

        string toolResult = output.Name switch
        {
            "get_weather" => """{"temperature":25}""",
            _ => throw new InvalidOperationException($"Unknown tool '{output.Name}'.")
        };

        var continuationRequest = new ChatRequest(Gpt54Mini.Instance)
        {
            PreviousResponseId = response.Details.Id,

            FunctionCallResults =
            [
                new FunctionCallResult(output.CallId!, toolResult)
            ]
        };

        var finalResponse = await _openAIMedia.ChatClient.GetResponseAsync(continuationRequest);

        Console.WriteLine(finalResponse.GetResponseText());
    }
}

Tool Execution

Function tools are not executed automatically by the library.

When the model returns a function_call output item:

1. Your application must deserialize the tool arguments.
2. Execute the corresponding business logic or external API call.
3. Send the result back using FunctionCallOutputContent.
4. Continue the conversation with another Responses API request.

This design keeps tool execution fully controlled by the host application.

Tool Choice

ToolChoice controls how the model uses tools:

• Auto → model decides whether to use tools
• Required → model must use a tool
• None → disables tool usage for the request

Validation and Error Handling

• Requests are validated locally before calling OpenAI.
• Invalid configuration values throw ValidationException.
• Unsupported parameter/model combinations throw ValidationException.
• OpenAI API errors throw OpenAiException.
• Invalid/unexpected API response payloads throw InvalidDataException.
• Unsupported tool/model combinations throw ValidationException.

Notes

Ensure to replace the placeholders in the code snippets with actual values applicable to your application.