Skip to main content

Groq AI Provider (@voltagent/groq-ai)

The Groq AI Provider connects VoltAgent to the Groq API, enabling the use of language models hosted on Groq's high-performance infrastructure. It utilizes the groq-sdk Node.js library.

Key Characteristics:

  • High-Speed Inference: Leverages Groq's LPU™ Inference Engine for fast response times.
  • API Key Authentication: Uses Groq API keys for authentication.
  • OpenAI-Compatible API: Primarily interacts with Groq's chat completions endpoint, which shares similarities with the OpenAI API structure.

Installation​

npm install @voltagent/core @voltagent/groq-ai zod

Note: groq-sdk is a peer dependency. zod is required if using generateObject.

Configuration​

The GroqProvider requires your Groq API key.

You can provide the API key directly during instantiation or set the GROQ_API_KEY environment variable.

import { GroqProvider } from "@voltagent/groq-ai";

// Option 1: Direct API Key
const groqProviderDirect = new GroqProvider({
  apiKey: "YOUR_GROQ_API_KEY",
});

// Option 2: Environment Variable (GROQ_API_KEY)
// Ensure process.env.GROQ_API_KEY is set
const groqProviderEnv = new GroqProvider();

Get your API key from the GroqCloud Console.

Full Runnable Example​

For a complete, runnable example demonstrating the use of this provider, please see:

Usage​

Instantiate your Agent with the configured GroqProvider:

import { Agent } from '@voltagent/core';
import { GroqProvider } from '@voltagent/groq-ai';

// Using environment variable configuration from above
const groqProvider = new GroqProvider();

const agent = new Agent({
  name: 'Groq Speed Agent',
  description: 'An agent powered by Groq's fast inference',
  llm: groqProvider,
  // Specify the desired Groq model ID (e.g., Llama3, Mixtral)
  // Find available models: https://console.groq.com/docs/models
  model: 'llama3-8b-8192',
});

// Example call
async function run() {
  const response = await agent.generateText("Explain what makes Groq's inference engine fast.");
  console.log(response.text);
}

run();

Supported Methods​

  • generateText: Supported. Calls Groq SDK's chat.completions.create.
  • streamText: Supported. Calls Groq SDK's chat.completions.create with stream: true.
  • generateObject: Supported. Calls Groq SDK's chat.completions.create with response_format: { type: 'json_object' }. Requires a z.ZodObject schema and a model that supports JSON mode.
  • streamObject: ❌ Not Supported. This provider currently does not implement object streaming.

Multi-modal Support​

❌ Not Supported.

The provider currently only handles string content within BaseMessage objects. Multi-modal inputs (like images) are not processed.

Model Selection & Options​

The specific Groq model ID (e.g., 'llama3-70b-8192', 'mixtral-8x7b-32768') is set via the model property during Agent instantiation. Refer to the Groq Models Documentation for available model IDs.

You can override or provide additional Groq-specific generation parameters (like temperature, max_tokens, top_p, stop, etc.) per-request using the provider key within the options object of generateText, streamText, or generateObject.

// Example: Overriding temperature and max_tokens for a specific call
const response = await agent.generateText("Write a short story about a futuristic city.", {
  provider: {
    temperature: 0.8,
    maxTokens: 500,
    // Other Groq-specific parameters (max_tokens, top_p, etc.)
  },
});

Refer to the groq-sdk documentation or the Groq API Reference for the full list of available parameters.

Code Examples​

Text Generation (generateText)​

import { Agent } from "@voltagent/core";
import { GroqProvider } from "@voltagent/groq-ai";

async function main() {
  // Assumes GROQ_API_KEY is set in environment variables
  const groqProvider = new GroqProvider();

  const agent = new Agent({
    name: "Groq Text Agent",
    description: "Generates text using Groq AI",
    llm: groqProvider,
    model: "mixtral-8x7b-32768", // Example model
  });

  const prompt = "What is the capital of France?";

  try {
    const response = await agent.generateText(prompt);
    console.log(`Agent response to "${prompt}":`);
    console.log(response.text);
    console.log("Usage:", response.usage);
    console.log("Finish Reason:", response.finishReason);
  } catch (error) {
    console.error("Error generating text:", error);
  }
}

main();

Streaming Text (streamText)​

import { Agent } from "@voltagent/core";
import { GroqProvider } from "@voltagent/groq-ai";

async function main() {
  const groqProvider = new GroqProvider({
    // Or provide apiKey directly
    // apiKey: 'YOUR_GROQ_API_KEY'
  });

  const agent = new Agent({
    name: "Groq Streaming Agent",
    description: "Streams text using Groq AI",
    llm: groqProvider,
    model: "llama3-8b-8192", // Example model
  });

  const prompt = "Write a haiku about speed.";

  try {
    const streamResponse = await agent.streamText(prompt);

    console.log(`Streaming agent response to "${prompt}":`);
    let fullText = "";
    for await (const chunk of streamResponse.textStream) {
      fullText += chunk;
      process.stdout.write(chunk);
    }
    console.log("\n--- Stream Finished ---");
    // Note: Usage info might be available via callbacks (onFinish/onStepFinish)
    // or attached to the stream result in the provider implementation.
    // Check GroqProvider source for specifics if needed.
  } catch (error) {
    console.error("Error streaming text:", error);
  }
}

main();

Generating Structured Objects (generateObject)​

Requires a model that supports JSON mode (e.g., Llama3 models on Groq). Find compatible models on the Groq models page.

import { Agent } from "@voltagent/core";
import { GroqProvider } from "@voltagent/groq-ai";
import { z } from "zod"; // Import Zod

// Define a Zod schema for the desired object structure
const citySchema = z.object({
  name: z.string().describe("The city name"),
  country: z.string().describe("The country the city is in"),
  population: z.number().int().positive().describe("Estimated population"),
  landmarks: z.array(z.string()).describe("List of famous landmarks"),
});

async function main() {
  const groqProvider = new GroqProvider();

  // Use a model supporting JSON mode
  const agent = new Agent({
    name: "Groq Object Agent",
    description: "Generates structured data using Groq AI",
    llm: groqProvider,
    model: "llama3-70b-8192",
  });

  const prompt = "Generate details for Paris, France.";

  try {
    // Call generateObject with the prompt and Zod schema
    const response = await agent.generateObject(prompt, citySchema, {
      provider: {
        temperature: 0.1, // Low temperature for JSON mode
      },
    });

    console.log("Generated Object:");
    console.log(response.object);
    console.log("Usage:", response.usage);
    console.log("Finish Reason:", response.finishReason);
  } catch (error) {
    console.error("Error generating object:", error);
  }
}

main();

Table of Contents