Your own Azure deployment

IMPORTANT - this option is only available if you purchase one of the Caching Only plans from the Microsoft Azure Marketplace and not available if you purchase CogCache from our website. Also note you will not benefit the discounts that CogCache provides for the tokens.

1️⃣ Create a CogCache account

If you don't have a CogCache account you can create a CogCache account through the Microsoft Azure store. You can find the CogCache listing here.

2️⃣ Generate a CogCache API key

To authenticate the CogCache Proxy API, you need a CogCache API key. You can easily generate an API key during onboarding. Alternatively, you can go to the Keys page to generate it there.

3️⃣ Integrate CogCache with your setup

There are multiple options to integrate with CogCache based on the language of choice.

✅
Option A - cURL

Integration steps:

Add theCogcache-Auth header and set the CogCache API key as value.
As you are using your own Azure OpenAI deployment use your own Api-Key value generated while setting up the OpenAI deployment on Azure.

curl --request POST \
    --url https://proxy-api.cogcache.com/azure/openai/deployments/YOUR_AZURE_LLM_MODEL/chat/completions?api-version=AZURE_API_VERSION \
    --header 'Api-Key: YOUR_AZURE_OPENAI_API_KEY' \
    --header 'Cogcache-Auth: Bearer COGCACHE_API_KEY' \
    --header 'Cogcache-OpenAI-Api-Base: https://YOUR_AZURE_RESOURCE_NAME.openai.azure.com' \
    --header 'Content-Type: application/json' \
    --data '{
      "messages": [
          {
              "role": "system",
              "content": "Assistant is a large language model trained by OpenAI."
          },
          {
              "role":"user",
              "content":"Write a blog post about Generative AI"
          }
      ],
      "stream":true
  }'

✅
Option B - Python

Integration steps:

Modify the azure_endpointand set a value of https://proxy-api.cogcache.com/azure
Add theCogcache-Auth header and set the CogCache API key as value.
Add theCogcache-OpenAI-Api-Base header and set as value your Azure resource name.
As you are using your own Azure OpenAI deployment use your own api_key value generated while setting up the OpenAI deployment on Azure.

from openai import AzureOpenAI

COGCACHE_API_KEY = '' # the generated CogCache API key

AZURE_API_VERSION = '' # the AZURE_API_VERSION of your choice
YOUR_AZURE_OPENAI_API_KEY = '' # your Azure OpenAI API Key
YOUR_AZURE_RESOURCE_NAME = '' # your Azure OpenAI resource name
YOUR_AZURE_LLM_MODEL = '' # your Azure OpenAI LLM model deployment name


client = AzureOpenAI(
  azure_endpoint = 'https://proxy-api.cogcache.com/azure', 
  api_key=YOUR_AZURE_OPENAI_API_KEY,
  api_version=AZURE_API_VERSION,
  default_headers={
    "Cogcache-Auth": f"Bearer {COGCACHE_API_KEY}",
    "Cogcache-OpenAI-Api-Base": f"https://{YOUR_AZURE_RESOURCE_NAME}.openai.azure.com"
  }
)

response = client.chat.completions.create(
    model=YOUR_AZURE_LLM_MODEL,
    stream=True,
    messages=[
        {"role": "system", "content": "Assistant is a large language model trained by OpenAI."},
        {"role": "user", "content": "Write a blog post about Generative AI"},
    ]
)

for chunk in response:
    print(chunk)

✅
Option C - Node.js

Integration steps:

When initialising the OpenAIClient use this value for the endpoint:https://proxy-api.cogcache.com/azure
When making the client.streamChatCompletions call make sure to add theCogcache-Auth header and set the CogCache API key as value.
Also for theclient.streamChatCompletions call, add theCogcache-OpenAI-Api-Base header and set as value your Azure resource name.
As you are using your own Azure OpenAI deployment use your own api_key value generated while setting up the OpenAI deployment on Azure.

const { OpenAIClient, AzureKeyCredential } = require("@azure/openai");

const COGCACHE_API_KEY = ''; // the generated CogCache API key

const AZURE_API_VERSION = ''; // the Azure OpenAI version of your choice
const YOUR_AZURE_OPENAI_API_KEY = ''; // your Azure OpenAI API Key
const YOUR_AZURE_RESOURCE_NAME = ''; // your Azure OpenAI resource name
const YOUR_AZURE_LLM_MODEL = ''; // your Azure OpenAI LLM model deployment name

const client = new OpenAIClient(
  'https://proxy-api.cogcache.com/azure',
  new AzureKeyCredential(YOUR_AZURE_OPENAI_API_KEY),
  {
    apiVersion:AZURE_API_VERSION
  }
);

const messages = [
  {
    role: 'system',
    content: 'Assistant is a large language model trained by OpenAI.',
  },
  {
    role: 'user',
    content: 'Write a blog post about metaverse',
  },
];

const main = async () => {
  const events = await client.streamChatCompletions(
    YOUR_AZURE_LLM_MODEL,
    messages,
    {
      requestOptions: {
        headers: {
          'Cogcache-Auth': `Bearer ${COGCACHE_API_KEY}`,
          'Cogcache-OpenAI-Api-Base': `https://${YOUR_AZURE_RESOURCE_NAME}.openai.azure.com`,
        },
      },
    }
  );

  for await (const event of events) {
    for (const choice of event.choices) {
      const content = choice.delta?.content
      if (content !== undefined) {
        console.log(content)
      }
    }
  }
};

main().catch((err) => {
  console.error('Something went wrong:', err);
});

✅
Option D - LangChain

Integration steps:

Modify the azure_endpointand set a value of https://proxy-api.cogcache.com/azure
Add theCogcache-Auth header and set the CogCache API key as value.
Add theCogcache-OpenAI-Api-Base header and set as value your Azure resource name.
As you are using your own Azure OpenAI deployment use your own openai_api_key value generated while setting up the OpenAI deployment on Azure.

from langchain_openai import AzureChatOpenAI
from langchain.schema import HumanMessage, SystemMessage
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler

COGCACHE_API_KEY = '' # the generated CogCache API key

AZURE_API_VERSION = '' # the AZURE_API_VERSION of your choice
YOUR_AZURE_OPENAI_API_KEY = '' # your Azure OpenAI API Key
YOUR_AZURE_RESOURCE_NAME = '' # your Azure OpenAI resource name
YOUR_AZURE_LLM_MODEL = '' # your Azure OpenAI LLM model deployment name

model = AzureChatOpenAI(
    streaming=True,
    callbacks=[StreamingStdOutCallbackHandler()],
    azure_endpoint="https://proxy-api.cogcache.com/azure",
    openai_api_version=AZURE_API_VERSION,
    deployment_name=YOUR_AZURE_LLM_MODEL,
    openai_api_key=YOUR_AZURE_OPENAI_API_KEY,
    openai_api_type="azure",
    default_headers={
        "Cogcache-Auth": f"Bearer {COGCACHE_API_KEY}",
        "Cogcache-OpenAI-Api-Base": f"https://{YOUR_AZURE_RESOURCE_NAME}.openai.azure.com"
    },
)

model.invoke(
    [
        SystemMessage(content="Assistant is a large language model trained by OpenAI."),
        HumanMessage(content="Write a blog post about Generative AI"),
    ]
)

✅
Option E - LangChain JS

Integration steps:

Modify the azureOpenAIBasePathand set a value of https://proxy-api.cogcache.com/azure
Add theCogcache-Auth header and set the CogCache API key as value.
Add theCogcache-OpenAI-Api-Base header and set as value your Azure resource name.
As you are using your own Azure OpenAI deployment use your own azureOpenAIApiKey value generated while setting up the OpenAI deployment on Azure.

import { ChatOpenAI } from 'langchain/chat_models/openai';
import { ChatMessage } from 'langchain/schema';


const COGCACHE_API_KEY = ''; // the generated CogCache API key

const AZURE_API_VERSION = ''; // the Azure OpenAI version of your choice
const YOUR_AZURE_OPENAI_API_KEY = ''; // your Azure OpenAI API Key
const YOUR_AZURE_RESOURCE_NAME = ''; // your Azure OpenAI resource name
const YOUR_AZURE_LLM_MODEL = ''; // your Azure OpenAI LLM model deployment name

const model = new ChatOpenAI({
  streaming: true,
  azureOpenAIApiKey: YOUR_AZURE_OPENAI_API_KEY,
  azureOpenAIApiVersion: AZURE_API_VERSION,
  azureOpenAIApiDeploymentName: YOUR_AZURE_LLM_MODEL,
  azureOpenAIBasePath:
    'https://proxy-api.cogcache.com/azure',
  configuration: {
    baseOptions: {
      headers: {
        'Cogcache-Auth': `Bearer ${COGCACHE_API_KEY}`,
        'Cogcache-OpenAI-Api-Base': `https://${YOUR_AZURE_RESOURCE_NAME}.openai.azure.com`,
      },
    },
  },
});


(async () => {
  const response = await model.invoke(
    [
      new ChatMessage(
        'Assistant is a large language model trained by OpenAI.',
        'system'
      ),
      new ChatMessage('Write a blog post about Generative AI', 'user'),
    ],
    {
      callbacks: [
        {
          handleLLMNewToken: (token, ...resp) => {
            console.log('handleLLMNewToken', token);
          },
          handleLLMEnd: async (output) => {
            console.log('handleLLMEnd', output);
          },
          handleChatModelStart: (...chat) => {
            console.log('handleChatModelStart', chat);
          },
        },
      ],
    }
  );
})().catch((err) => {
  console.error('Something went wrong:', err);
});

Your own Azure deployment

1️⃣ Create a CogCache account

2️⃣ Generate a CogCache API key

3️⃣ Integrate CogCache with your setup

✅
Option A - cURL

✅
Option B - Python

✅
Option C - Node.js

✅
Option D - LangChain

✅
Option E - LangChain JS

For any common integration problems consult our FAQ here.

1️⃣ Create a CogCache account

2️⃣ Generate a CogCache API key

3️⃣ Integrate CogCache with your setup

✅Option A - cURL

✅Option B - Python

✅Option C - Node.js

✅Option D - LangChain

✅Option E - LangChain JS

For any common integration problems consult our FAQ here.

✅
Option A - cURL

✅
Option B - Python

✅
Option C - Node.js

✅
Option D - LangChain

✅
Option E - LangChain JS