AssemblyAI.SemanticKernel 1.1.0

<img src="https://github.com/AssemblyAI/assemblyai-python-sdk/blob/master/assemblyai.png?raw=true" width="500" alt="AssemblyAI logo"/>

AssemblyAI integration for Semantic Kernel

Transcribe audio using AssemblyAI with Semantic Kernel plugins.

Get started

Add the AssemblyAI.SemanticKernel NuGet package to your project.

dotnet add package AssemblyAI.SemanticKernel

Next, register the AssemblyAI plugin into your kernel:

using AssemblyAI.SemanticKernel;
using Microsoft.SemanticKernel;

// Build your kernel
var kernel = Kernel.CreateBuilder();

// Get AssemblyAI API key from env variables, or much better, from .NET configuration
string apiKey = Environment.GetEnvironmentVariable("ASSEMBLYAI_API_KEY")
  ?? throw new Exception("ASSEMBLYAI_API_KEY env variable not configured.");

kernel.ImportPluginFromObject(
    new TranscriptPlugin(apiKey: apiKey)
    TranscriptPlugin.PluginName
);

Usage

Get the Transcribe function from the transcript plugin and invoke it with the context variables.

var result = await kernel.InvokeAsync(
    nameof(AssemblyAIPlugin), 
    AssemblyAIPlugin.TranscribeFunctionName, 
    new KernelArguments
    {
        ["INPUT"] = "https://storage.googleapis.com/aai-docs-samples/espn.m4a"
    }
);
Console.WriteLine(result.GetValue<string>());

You can get the transcript using result.GetValue<string>().

You can also upload local audio and video file. To do this:

• Set the AssemblyAI:Plugin:AllowFileSystemAccess configuration to true.
• Configure the INPUT variable with a local file path.

kernel.ImportPluginFromObject(
    new TranscriptPlugin(apiKey: apiKey)
    {
        AllowFileSystemAccess = true
    }
);
var result = await kernel.InvokeAsync(
    nameof(AssemblyAIPlugin), 
    AssemblyAIPlugin.TranscribeFunctionName, 
    new KernelArguments
    {
        ["INPUT"] = "https://storage.googleapis.com/aai-docs-samples/espn.m4a"
    }
);
Console.WriteLine(result.GetValue<string>());

You can also invoke the function from within a semantic function like this.

const string prompt = """
                      Here is a transcript:
                      {{AssemblyAIPlugin.Transcribe "https://storage.googleapis.com/aai-docs-samples/espn.m4a"}}
                      ---
                      Summarize the transcript.
                      """;
var result = await kernel.InvokePromptAsync(prompt);
Console.WriteLine(result.GetValue<string>());

All the code above explicitly invokes the transcript plugin, but it can also be invoked as part of a plan. Check out the Sample project) which uses a plan to transcribe an audio file in addition to explicit invocation.

Notes

• The AssemblyAI integration only supports Semantic Kernel with .NET at this moment. If there's demand, we will extend support to other platforms, so let us know!
• Feel free to file an issue in case of bugs or feature requests.