Skip to content

Integrating LiteLLM with EggAI

This example demonstrates integrating LiteLLM into the eggai SDK. It shows a system with two AI agents—SupportAgent and EscalationAgent—to handle customer inquiries efficiently and escalate complex issues when necessary.

Key features:

  • LiteLLM integration
  • Tool usage
  • Collaborative between two agents

The code for this example is available here.

Prerequisites

Ensure you have the following dependencies installed:

  • Python 3.10+
  • Docker and Docker Compose

Ensure you have a valid OpenAI API key set in your environment:

export OPENAI_API_KEY="your-api-key"

Setup Instructions

Clone the EggAI repository:

git clone git@github.com:eggai-tech/EggAI.git

Move into the examples/litellm_agent folder:

cd examples/litellm_agent

Create and activate a virtual environment:

python -m venv .venv
source .venv/bin/activate  # For Windows: venv\Scripts\activate

Install the required dependencies:

pip install -r requirements.txt

Start Redpanda using Docker Compose:

docker compose up -d

Run the Example

python main.py

Expected output:

Handling customer inquiry: What is your return policy?
Agent is running. Press Ctrl+C to stop.
Querying knowledge base for: return_policy
Failed to decode response from SupportAgent, message was:  {
    "response": "Our return policy is 30 days from the date of purchase. Items must be in their original condition."
}
{
    "response": "Our return policy is 30 days from the date of purchase. Items must be in their original condition."
}
Handling customer inquiry: I have a billing issue that isn't resolved yet.
Querying knowledge base for: billing_issue
Response from SupportAgent: {'response': 'escalate'}
Escalating issue to EscalationAgent...
Creating support ticket for issue: I have a billing issue that isn't resolved yet.
Ticket created for escalated issue:  {'ticket_id': 'TCKT7707', 'department': 'Billing Department'}
^CTask was cancelled. Cleaning up...

What happens:

  • Customers submit inquiries like "What is your return policy?" via the humans channel.
  • SupportAgent: Processes general inquiries using the GetKnowledge tool.
  • Responds directly if the inquiry is simple.
  • Escalates complex issues to the EscalationAgent.
  • EscalationAgent:
  • Creates a support ticket for escalated issues using the TicketingTool.
  • Publishes ticket details to the agents channel and informs the customer.

Clean Up

Stop and clean up the Docker containers:

docker compose down -v

Next Steps

Ready to explore further? Check out:

  • Advanced Examples: Discover more complex use cases in the examples folder.
  • Contribution Guidelines: Get involved and help improve EggAI!
  • GitHub Issues: Submit a bug or feature request.
  • Documentation: Refer to the official docs for deeper insights.