Execution Runs
The endpoints managing execution runs resources are located under the /runs
path.
Excution Run Object
An Execution Run represents the execution of an interaction: both the request and the response.
Properties
- Name
id
- Type
- string
- Description
Unique identifier for the execution run.
- Name
result
- Type
- any
- Description
The result of the execution run.
- Name
parameters
- Type
- any
- Description
Input data passed to the prompt templates to generate the final prompt.
- Name
interaction
- Type
- Interaction|string
- Description
Definition of the interaction for the execution run.
- Name
environment
- Type
- ExecutionEnvironmentRef
- Description
Reference to the execution environment used for the execution run.
- Name
modelId
- Type
- string
- Description
Identifier of the model used for the execution run.
- Name
ttl
- Type
- number
- Description
Time to live for the execution run.
- Name
status
- Type
- ExecutionRunStatus
- Description
Current status of the execution run. Can be one of:
created
- the execution was created but not yet sent to the target environment to be executed.processing
- the execution is being processed by the target environment.completed
- the execution was completed successfully.failed
- the execution failed.
- Name
token_use
- Type
- ExecutionTokenUsageInfo
- Description
Information about the token usage for the execution run.
- Name
execution_time
- Type
- number
- Description
Time taken for the execution run in milliseconds.
- Name
created_at
- Type
- Date
- Description
Date and time when the execution run was created.
- Name
updated_at
- Type
- Date
- Description
Date and time when the execution run was last updated.
- Name
account
- Type
- AccountRef
- Description
Reference to the account associated with the execution run.
- Name
project
- Type
- ProjectRef
- Description
Reference to the project associated with the execution run.
- Name
config
- Type
- InteractionExecutionConfiguration
- Description
Configuration for the execution of the interaction.
- Name
error
- Type
- object
- Description
Error information for the execution run, if any.
- Name
draft
- Type
- string
- Description
Status indicating the interaction is in draft mode.
- Name
published
- Type
- string
- Description
Status indicating the interaction is published.
- Name
archived
- Type
- string
- Description
Status indicating the interaction is archived.
- Name
prompt
- Type
- number
- Description
Number of tokens used for the prompt in the execution run.
- Name
result
- Type
- number
- Description
Number of tokens used for the result in the execution run.
- Name
total
- Type
- number
- Description
Total number of tokens used in the execution run.
- Name
environment
- Type
- string
- Description
Identifier of the execution environment for the interaction.
- Name
temperature
- Type
- number
- Description
Sampling temperature to control randomness of the model's output.
- Name
max_tokens
- Type
- number
- Description
Maximum number of tokens (words/characters) for the model to generate in the output.
- Name
cache_policy
- Type
- CachePolicy
- Description
Policy determining how the results are cached.
ExecutionRun
{
"parameters": {
"object": "snow"
},
"tags": [
"cli",
"virtual"
],
"environment": {
"name": "Synthetic",
"id": "655660c9f19a49a47106075a"
},
"interaction": {
"name": "WhatColor",
"status": "draft",
"prompts": [
{
"type": "template",
"template": {
"role": "user",
"name": "SimplePrompt",
"content_type": "jst",
"id": "6554cfb57eae1c28ef5f3d4b"
},
"id": "6554cfd47eae1c28ef5f3d53"
}
],
"updated_at": "2023-11-30T18:28:26.808Z",
"id": "6554cf617eae1c28ef5f3d40"
},
"model": "cohere.command-text-v14",
"status": "completed",
"ttl": 3600,
"source": {
"type": "api",
"label": "Key Created On 2023-10-24 16:31:28 via API",
"principal_type": "apikey",
"principal_id": "653772d5d73c5ac92045df97",
"client_ip": "::1"
},
"account": "652d77895674c387e105948c",
"project": "654df9de09676ad3b8631dc3",
"config": {
"formatter": "genericTextLLM",
"id": "656766aa862862fbc8d5a2d5"
},
"created_at": "2023-11-29T16:28:26.676Z",
"updated_at": "2023-11-29T16:28:28.490Z",
"execution_time": 1508,
"prompt": "User: What color is the snow?\n\nSystem: You must answer using the following JSONSchema:\n ---\n {\"type\":\"object\",\"properties\":{\"color\":{\"type\":\"string\"}},\"required\":[\"color\"]}\n ---",
"result": {
"color": "white"
},
"token_use": {
"result": 25,
"prompt": 198,
"total": 223
},
"modelId": "cohere.command-text-v14",
"id": "656766aa862862fbc8d5a2d4"
}
List RunsGET /runs
Get the list of the runs in the current project. \
Parameters
- Name
limit
- Type
- number
- Modifier
- optional
- Description
The number of items to return in the response. Used for pagination. Default is 100.
- Name
offset
- Type
- number
- Modifier
- optional
- Description
The number of items to skip before starting to collect the result set. Used for pagination. Default is 0.
- Name
interaction
- Type
- string
- Modifier
- optional
- Description
Filter by a an interaxction ID. Can be a comma separated list of IDs.
- Name
model
- Type
- string
- Modifier
- optional
- Description
Filter by a model name. Can be a comma separated list of models.
- Name
environment
- Type
- string
- Modifier
- optional
- Description
Filter by an enviroment ID. Can be a comma separated list of IDs.
- Name
status
- Type
- string
- Modifier
- optional
- Description
Filter by the run status. Can be one of:
created
processing
completed
failed
- Name
tag
- Type
- string
- Modifier
- optional
- Description
Filter by tag. Can be a comma separated list of tags.
- Name
anchorDate
- Type
- ISO date (string)
- Modifier
- optional
- Description
A date to use as an anchor for the
updated_at
filter. The date must be in ISO format. Theupdated_at
filter will return all runs updated after or before the anchor date as specified by theanchorDateDirection
field.
- Name
anchorDateDirection
- Type
- string
- Modifier
- optional
- Description
The direction to use with anchorDate. Can be one of:
before
after
. The default isafter
.
- Name
parent
- Type
- string|false
- Modifier
- optional
- Description
Can be either the ID of a parent run or
false
. Iffalse
only top level runs are returned. If a run ID is specified only runs that are children of the specified run are returned.
Returns
A list of run references. A run reference is a subset of the Execution Run object.
Request
curl https://api.composableprompts.com/api/v1/runs?limit=50&interaction=6554cf617eae1c28ef5f3d40&tag=cli \
-H "Accept: application/json"
-H "Authorization: Bearer {token}"
Response
[
{
parameters: {
object: "flower"
},
tags: [ "cli", "virtual" ],
environment: {
name: "Synthetic",
id: "655660c9f19a49a47106075a"
},
modelId: "anthropic.claude-v2",
status: "completed",
ttl: 3600,
source: {
type: "api",
label: "Key Created On 2023-10-24 16:31:28 via API",
principal_type: "apikey",
principal_id: "653772d5d73c5ac92045df97",
client_ip: "::1"
},
account: {
name: "Dengen Labs",
id: "652d77895674c387e105948c"
},
project: {
name: "Experiments",
id: "654df9de09676ad3b8631dc3"
},
config: {
formatter: "claude",
id: "655cada12a8da9904a723b8b"
},
created_at: "2023-11-21T13:16:17.165Z",
updated_at: "2023-11-21T13:16:20.651Z",
prompt: "\n\n\nHuman: What color is the flower?\n\n\nHuman: You must answer using the following JSONSchema:\n ---\n {\"type\":\"object\",\"properties\":{\"color\":{\"type\":\"string\"}},\"required\":[\"color\"]}\n ---\n\nAssistant:",
execution_time: 3324,
result: {
color: "yellow"
},
token_use: {
result: 4,
prompt: 0,
total: 4
},
interaction: {
name: "WhatName",
status: "draft",
prompts: [
[Object ...]
],
updated_at: "2023-11-24T10:52:51.067Z",
id: "6554cf617eae1c28ef5f3d40"
},
id: "655cada12a8da9904a723b8a"
},
...
]
Retrieve an execution runGET /runs/:runId
Get an execution run object given its ID.
Parameters
No parameters.
Returns
An execution run object.
Request
curl https://api.composableprompts.com/api/v1/runs/655cada12a8da9904a723b8a \
-H "Accept: application/json"
-H "Authorization: Bearer {token}"
Response
{
parameters: {
object: "flower"
},
tags: [ "cli", "testing_session2", "virtual" ],
environment: {
name: "Synthetic",
id: "655660c9f19a49a47106075a"
},
modelId: "anthropic.claude-v2",
status: "completed",
ttl: 3600,
source: {
type: "api",
label: "Key Created On 2023-10-24 16:31:28 via API",
principal_type: "apikey",
principal_id: "653772d5d73c5ac92045df97",
client_ip: "::1"
},
account: "652d77895674c387e105948c",
project: "654df9de09676ad3b8631dc3",
config: {
formatter: "claude",
id: "655cada12a8da9904a723b8b"
},
created_at: "2023-11-21T13:16:17.165Z",
updated_at: "2023-11-21T13:16:20.651Z",
prompt: "\n\n\nHuman: What color is the flower?\n\n\nHuman: You must answer using the following JSONSchema:\n ---\n {\"type\":\"object\",\"properties\":{\"color\":{\"type\":\"string\"}},\"required\":[\"color\"]}\n ---\n\nAssistant:",
execution_time: 3324,
result: {
color: "yellow"
},
token_use: {
result: 4,
prompt: 0,
total: 4
},
interaction: {
name: "WhatName",
status: "draft",
prompts: [
[Object ...]
],
updated_at: "2023-11-24T10:52:51.067Z",
id: "6554cf617eae1c28ef5f3d40"
},
id: "655cada12a8da9904a723b8a"
}
Search execution runsPOST /runs/search
Search execution runs
Parameters
- Name
limit
- Type
- number
- Modifier
- optional
- Description
The number of items to return in the response. Used for pagination. Defaults to 100.
- Name
offset
- Type
- number
- Modifier
- optional
- Description
The number of items to skip before starting to collect the result set. Used for pagination. Default is 0.
- Name
onlyMeta
- Type
- boolean
- Modifier
- optional
- Description
Whether to return only the facet meta data witout the search results. Defaults to false. It only make sense to use this parameter when the
facets
parameter is specified.
- Name
query
- Type
- string
- Modifier
- false
- Description
A Lucene query to search the input data object.
- Name
defaultQueryPath
- Type
- string
- Modifier
- false
- Description
Default path for the property to match if no property path is not specified in the query
- Name
facets
- Type
- object
- Modifier
- false
- Description
A map of facet name to facet configuration. If specified the facets are used to generate the facet meta data. that will be returned along with the search results.
This is usefull when you want need faceted search.There are 3 types of facets that can be used:
- String facet - used to facet on string properties.
- Numeric facet - used to facet on numeric properties.
- Date facet - used to facet on date properties.
String facets has the following properties:
{ type: "string", path: string, numBuckets?: number, // defaults to 10 }
Numeric and Date facets are similar.
Here is the shape of a numeric facet
{ type: "number", path: string, boundaries: number[], default?: string }
And here is the shape of a date facet:
{ type: "date", path: string, boundaries: string[], // dates in ISO 8601 format default?: string }
All facet configurations define must define a
path
property which is the path to the property to facet on. The following properties are available to facet on:- modelId - the model used to execute the interaction (string facet).
- status - the status of the execution run (string facet).
- tags - the tags used on the execution run (string facet).
- updated_at - the date the execution run was last updated (date facet).
- execution_time - the execution time (numeric facet).
The
numBuckets
property of the string facet is used to control the number of buckets to use for the facet. It defualts to 10.
Theboundaries
is a list of numeric or dates (in ISO 8601 format), in ascending order, that specify the boundaries for each bucket. You must specify at least two boundaries. Each adjacent pair of values acts as the inclusive lower bound and the exclusive upper bound for the bucket.
Thedefault
property is used to specify the bucket name to use for values not matching any of the boundaries.Example
{ facets: { model: { type: "string", path: "modelId", numBuckets: 10, }, date: { type: "date", path: "updated_at", boundaries: [ "2023-11-27T12:08:23.925Z", "2023-10-27T12:08:23.925Z", "2023-09-27T12:08:23.925Z" ], default: 'other' } } }
- Name
filters
- Type
- object
- Modifier
- false
- Description
A map of property keys to property values to use for filtering.
The following execution run properties are available to filter:
- Name
interaction
- Type
- string
- Modifier
- optional
- Description
Filter by a an interaction ID.
- Name
environemnt
- Type
- string
- Modifier
- optional
- Description
Filter by a an environment ID.
- Name
parent
- Type
- string|false
- Modifier
- optional
- Description
Can be either the ID of a parent run or
false
. Iffalse
only top level runs are returned. If a run ID is specified only runs that are children of the specified run are returned.
- Name
status
- Type
- string
- Modifier
- optional
- Description
Filter by a run status. Can be one of:
created
processing
completed
failed
- Name
tags
- Type
- string[]
- Modifier
- optional
- Description
Filter by tags. An array of tags.
- Name
modelId
- Type
- string
- Modifier
- optional
- Description
Filter by tags. A comma separated list of tags.
- Name
updated_at
- Type
- object
- Modifier
- optional
- Description
Filter by a range of dates. This is a object containg one or two pairs of the following properties:
- lt: a iso 8601 date string to match the upper bound of the range (exclusive)
- lte: a iso 8601 date string to match the upper bound of the range (inclusive)
- gt: a iso 8601 date string to match the lower bound of the range (exclusive)
- gte: a iso 8601 date string to match the lower bound of the range (inclusive)
The range may have only a lower bound, only an upper bound, or both.
Returns
A list of execution runs which match the search criteria.
Request
curl https://api.composableprompts.com/api/v1/runs/search \
-H "Accept: application/json"
-H "Authorization: Bearer {token}"
-X POST
-d {...}
Response
Same as list execution runs (i.e. /runs) endpoint
Execute an InteractionPOST /runs
This endpoint is creating an execution run by starting the execution of an interaction. There are 2 modes of execution:
- blocking: the execution is started and the response is returned when the execution is completed.
- streaming: the execution is started and the response is returned immediately as an execution run in
created
state.
The client must then start the execution by requesting an event strean as specified by the Server-Sent-Events.
In the browser you can do this by using theEventSource
API. InNode.js
you can use a sevrer side implementation of theEventSource
API like eventsource.
The event stream will sent the following events:
message
- a new chunk of the result data is available. The eventdata
property is the chunk as a string.close
- the execution is completed. The eventdata
property is the final execution reuslt object which can either has a status ofcompleted
orfailed
.
To start streaming the execution you should perform a GET request on /runs/:runId/stream
.
Parameters
- Name
interaction
- Type
- string
- Modifier
- required
- Description
The interaxction ID to execute.
- Name
stream
- Type
- boolean
- Modifier
- optional
- Description
Wheter or not to stream the execution. Default is
false
.
- Name
data
- Type
- object
- Modifier
- optional
- Description
The input data to use top generate the prompt.
- Name
tags
- Type
- string
- Modifier
- optional
- Description
The tags to add to the execution run.
- Name
config
- Type
- object
- Modifier
- optional
- Description
A configuration object to customize the execution.
- Name
environment
- Type
- string
- Modifier
- optional
- Description
An execution evnvironemnt ID to be used to generate the test data. If not specified the default interaction environment will be used.
- Name
model
- Type
- string
- Modifier
- optional
- Description
The model to be used to execute the interaction. If not specified the default model of the environment is used.
- Name
temperature
- Type
- number
- Modifier
- optional
- Description
The temperature to be used to execute the data. Defaults to 0.7.
- Name
max_tokens
- Type
- number
- Modifier
- optional
- Description
The maximum number of tokens to be used when generating the result.
- Name
formatter
- Type
- number
- Modifier
- optional
- Description
A custom formatter to use.
Returns
An execution run object.
Request
curl https://api.composableprompts.com/api/v1/runs \
-H "Accept: application/json"
-H "Authorization: Bearer {token}"
-X POST
-d {...}
Response
{
result: {
color: "blue"
},
parameters: {
object: "car"
},
tags: [],
environment: "654dfa3109676ad3b8631e24",
interaction: "6554cf617eae1c28ef5f3d40",
modelId: "gpt-4",
status: "completed",
ttl: 3600,
source: {
type: "api",
label: "Key Created On 2023-10-24 16:31:28 via API",
principal_type: "apikey",
principal_id: "653772d5d73c5ac92045df97",
client_ip: "::1"
},
account: "652d77895674c387e105948c",
project: "654df9de09676ad3b8631dc3",
config: {
environment: "654dfa3109676ad3b8631e24",
model: "gpt-4",
id: "656486b6958d9210bae70291"
},
input_embedding: [],
result_embedding: [],
created_at: "2023-11-27T12:08:22.691Z",
updated_at: "2023-11-27T12:08:23.925Z",
execution_time: 1100,
token_use: {
prompt: 51,
result: 9,
total: 60
},
prompt: [
{
content: "\n What color is the car?\n",
role: "user"
}
],
id: "656486b6958d9210bae70290"
}
Stream an execution runGET /runs/:runId/stream
Used to start streaming an execution run. The execution run must be in created
state.
Must be used after creating a run with stream
set to true
. \
See [#execute-an-interaction](Execute an Interaction) for more details.
Parameters
No parameters.
Returns
A stream of events as specified by the Server-Sent-Events protocol.
Request
curl -N https://api.composableprompts.com/api/v1/runs/:runId/stream