Environments
The endpoints managing environment resources are located under the /environments
path.
Environment Object
The Environment object represent an environment in which language models run.
We currently support the following environment:
azure_openai
- Azure OpenAI Servicebedrock
- Amazon Bedrockgroq
- Groqhuggingface_ie
- HuggingFace's Inference Endpointmistralai
- Mistral AI's La Platformeopenai
- OpenAIreplicate
- Replicatetogetherai
- TogetherAIvertexai
- Google's Vertex AIwastsonx
- IBM's watsonx.ai
In addition to the core inference providers above, we have created virtual environment, to assemble models and platform into a virtual, synthetic LLM, and offer several balancing / execution strategies:
virtual_lb
- a synthetic environment that allows load balancing between multiple modelsvirtual_mediator
- a synthetic environment that allows multi-head execution and LLM-mediated selection
- Name
name
- Type
- string
- Description
Name of the environment, e.g., "Synthetic LLM".
- Name
endpoint_url
- Type
- string
- Description
The URL for the environment's endpoint.
- Name
apiKey
- Type
- string
- Description
API key for accessing the environment, it is always masked in the response, to prevent accidental exposure.
- Name
provider
- Type
- string
- Description
Provider of the environment, like 'virtual
or
openai`.
- Name
project
- Type
- string
- Description
Identifier of the associated project.
- Name
enabled_models
- Type
- AIModel[]
- Description
Array of enabled models with their details like ID, name, provider, tags, and owner.
- Name
config
- Type
- object
- Description
Configuration details of the environment, including model balancing and formatting. It is only used for
virtual
environments, to host the list of models and their weights.
- Name
id
- Type
- string
- Description
Unique identifier of the environment.
- Name
created_at
- Type
- Date
- Description
Date and time when the environment was created.
- Name
updated_at
- Type
- Date
- Description
Date and time when the environment was last updated.
Properties
Environment contains properties to configure endpoints, API keys, models, and other related settings for language model interactions.
{
"name": "Synthetic LLM",
"endpoint_url": "",
"apiKey": "",
"provider": "virtual",
"project": "654df9de09676ad3b8631dc3",
"enabled_models": [...],
"created_at": "2023-11-18T09:51:13.392Z",
"updated_at": "2023-11-18T09:51:53.362Z",
"config": {...},
"id": "65588911a963a422f6a96d8a"
}
List environmentsGET /environments
Get the list of environments in the current project. \
Parameters
No parameters.
Returns
A list of environment references. An environment reference is a subset of the environment object.
Request
curl https://api.composableprompts.com/api/v1/environments \
-H "Accept: application/json"
-H "Authorization: Bearer {token}"
Response
[
{
"name": "Replicate",
"endpoint_url": "",
"apiKey": "r8_8****",
"provider": "replicate",
"project": "654df9de09676ad3b8631dc3",
"enabled_models": [
{
"id": "02e509c789964a7ea8736978a43525956ef40397be9033abf9fd2badfe68c9e3",
"name": "llama-2-70b-chat/0.8.1@~02e509",
"provider": "replicate",
"description": "A 70 billion parameter language model from Meta, fine tuned for chat completions",
"tags": [],
"owner": "meta"
},
...
],
"created_at": "2023-11-13T05:30:45.747Z",
"updated_at": "2023-11-13T05:31:50.361Z",
"id": "6551b485d09905deacdbf814"
},
{
"name": "OpenAI",
"endpoint_url": "",
"apiKey": "sk-7****",
"provider": "openai",
"project": "654df9de09676ad3b8631dc3",
"enabled_models": [
{
"id": "gpt-4",
"name": "gpt-4",
"provider": "openai",
"type": "text",
"tags": [],
"owner": "openai"
},
...
],
"created_at": "2023-11-10T09:38:57.104Z",
"updated_at": "2023-11-10T09:39:07.212Z",
"id": "654dfa3109676ad3b8631e24"
},
...
]
Retrieve an environmentGET /environments/:envId
Get an environment object given its ID.
Parameters
No parameters.
Returns
An environment object.
Request
curl https://api.composableprompts.com/api/v1/environments/654dfa3109676ad3b8631e24 \
-H "Accept: application/json"
-H "Authorization: Bearer {token}"
Response
{
"name": "OpenAI",
"endpoint_url": "",
"apiKey": "sk-7ZNRv*******************************************",
"provider": "openai",
"project": "654df9de09676ad3b8631dc3",
"enabled_models": [
{
"id": "gpt-4",
"name": "gpt-4",
"provider": "openai",
"type": "text",
"tags": [],
"owner": "openai"
},
...
],
"created_at": "2023-11-10T09:38:57.104Z",
"updated_at": "2023-11-10T09:39:07.212Z",
"id": "654dfa3109676ad3b8631e24"
}