LangChain JS

Follow this guide on how to integrate CogCache using LangChain JS.

1️⃣ Create a CogCache account

If you don't have a CogCache account you can create a CogCache account through the Microsoft Azure store. You can find the CogCache listing here.

2️⃣ Generate a CogCache API key

To authenticate the CogCache Proxy API, you need a CogCache API key. You can easily generate an API key during onboarding. Alternatively, you can go to the Keys page to generate it there.

3️⃣ Integrate CogCache with your setup

Integration steps:

1. Set the baseURLand use a value of https://proxy-api.cogcache.com/v1/.
2. Add the CogCache authorization header and set the CogCache API key as value.
3. Since you're not using your own LLM also set the value of the openai_api_key to the CogCache API key.
4. Choose the right model for you from this table by setting the COGCACHE_LLM_MODEL value.

Node

import { ChatOpenAI } from "@langchain/openai";

const COGCACHE_LLM_MODEL = ""; // the model of choice
const COGCACHE_API_KEY = ""; // the generated CogCache API key

const model = new ChatOpenAI({
  configuration: {
    baseURL: "https://proxy-api.cogcache.com/v1/",
    baseOptions: {
      headers: {
        "Authorization": `Bearer ${COGCACHE_API_KEY}`,
      },
    },
  },
  apiKey: COGCACHE_API_KEY,
  model: COGCACHE_LLM_MODEL,
  streaming: true,
});

(async () => {
  const response = await model.invoke(
    [
       ["system", "Assistant is a large language model trained by OpenAI."],
       ["user", "Write a blog post about Generative AI"] ,
    ],
    {
      callbacks: [
        {
          handleLLMNewToken: async (token) => {
            console.log("handleLLMNewToken", token);
          },
          handleLLMEnd: async (output) => {
            console.log("handleLLMEnd", output);
          },
          handleChatModelStart: (...chat) => {
            console.log("handleChatModelStart", chat);
          },
        },
      ],
    }
  );
})().catch((err) => {
  console.error("Something went wrong:", err);
});

For any common integration problems consult our FAQ here.