llm-integration

Cache OpenAI chat completion calls with memoized client utilities.

9|Updated Aug 8, 2023

Search Tags:#caching #python #llm #transformation #openai #memoization #chat-format

Ready to run

npx skills add https://github.com/anhvth/speedy_utils --skill llm-integration

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: llm-integration
Source: https://github.com/anhvth/speedy_utils/tree/main/.github/skills/llm-integration
Command: npx skills add https://github.com/anhvth/speedy_utils --skill llm-integration

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Memoized OpenAI client utilities optimize chat completion calls by caching responses to reduce API usage.

Core Features & Use Cases

Memoized OpenAI clients (MOpenAI, MAsyncOpenAI) that cache post requests to save costs and latency.
Chat format transformation utilities (transform_messages) to convert between ChatML, ShareGPT, text, and simulated transcripts.
Prompts preparation for local LLM inference and offline workflows.

Quick Start

Install speedy_utils and instantiate MOpenAI to begin memoizing chat completions.

Frequently Asked Questions about llm-integration

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I cache OpenAI chat completion calls in Python to reduce API costs?▼

To cache OpenAI chat completion calls in Python and reduce API costs, instantiate the MOpenAI client from this Skill to automatically memoize post requests and save latency.

Can I transform chat messages between ChatML and ShareGPT formats?▼

Yes, you can transform chat messages between ChatML and ShareGPT formats using the transform_messages utility, which also supports text and simulated transcripts for varied workflows.

What dependencies do I need to memoize OpenAI API calls?▼

To memoize OpenAI API calls, you need the speedy_utils library for caching utilities and the official openai package for API access, ensuring proper client configuration.

Does this memoization approach support asynchronous OpenAI clients?▼

Yes, this memoization approach supports asynchronous OpenAI clients through the MAsyncOpenAI class, allowing you to cache API responses in non-blocking workflows.

How do I prepare prompts for local LLM inference from chat formats?▼

To prepare prompts for local LLM inference from chat formats, use the transform_messages utility to convert ChatML or ShareGPT histories into plain text or simulated transcripts.