Request headers

Cache-Control

This header affects the cache behaviour as follows:

no-store - disables storing the answer from the LLM in the cache
no-cache - disables retrieval of answer from cache, goes directly to the LLM
only-if-cached - retrieves answer only from the cache and if not found it returns 504 error code
public - normal behaviour, it returns from cache if answer found there, otherwise retrieves answer from LLM and stores it in the cache

CogCache-Auth

This header is used to authenticate and authorise the CogCache Proxy API. It helps CogCache identify the organization and application.

The format is like this: Bearer COGCACHE_APIKEY

CogCache-OpenAI-Api-Base

This is the URL to your Azure OpenAI deployment, in this format https://YOUR_RESOURCE_NAME.openai.azure.com

Response headers

X-Cache

This header tells if the response was provided from the cache or not. However, the header will not be available if Cache-Control has a value of no-store or no-cache.

Possibles values:

hit - the answer was retrieved from cache
miss - the answer was retrieved directly from the LLM

CogCache-Cache-Entry-ID

This response header holds the cache entry unique ID when the response is retrieved from cache. When the response is retrieved from the LLM this header will not be available.