Composable Prompts

Execution Runs

The endpoints managing execution runs resources are located under the /runs path.

Excution Run Object

An Execution Run represent the execution of an interaction: both the request and the response.

Properties

  • Name
    id
    Type
    string
    Description

    Unique identifier for the execution run.

  • Name
    result
    Type
    any
    Description

    The result of the execution run.

  • Name
    parameters
    Type
    any
    Description

    Input data passed to the prompt templates to generate the final prompt.

  • Name
    interaction
    Type
    Interaction|string
    Description

    Definition of the interaction for the execution run.

  • Name
    environment
    Type
    ExecutionEnvironmentRef
    Description

    Reference to the execution environment used for the execution run.

  • Name
    modelId
    Type
    string
    Description

    Identifier of the model used for the execution run.

  • Name
    ttl
    Type
    number
    Description

    Time to live for the execution run.

  • Name
    status
    Type
    ExecutionRunStatus
    Description

    Current status of the execution run. Can be one of:

    • created - the execution was created but not yet sent to the target environment to be executed.
    • processing - the execution is being processed by the target environment.
    • completed - the execution was completed successfully.
    • failed - the execution failed.
  • Name
    token_use
    Type
    ExecutionTokenUsageInfo
    Description

    Information about the token usage for the execution run.

  • Name
    execution_time
    Type
    number
    Description

    Time taken for the execution run in milliseconds.

  • Name
    created_at
    Type
    Date
    Description

    Date and time when the execution run was created.

  • Name
    updated_at
    Type
    Date
    Description

    Date and time when the execution run was last updated.

  • Name
    account
    Type
    AccountRef
    Description

    Reference to the account associated with the execution run.

  • Name
    project
    Type
    ProjectRef
    Description

    Reference to the project associated with the execution run.

  • Name
    config
    Type
    InteractionExecutionConfiguration
    Description

    Configuration for the execution of the interaction.

  • Name
    error
    Type
    object
    Description

    Error information for the execution run, if any.

  • Name
    draft
    Type
    string
    Description

    Status indicating the interaction is in draft mode.

  • Name
    published
    Type
    string
    Description

    Status indicating the interaction is published.

  • Name
    archived
    Type
    string
    Description

    Status indicating the interaction is archived.

  • Name
    prompt
    Type
    number
    Description

    Number of tokens used for the prompt in the execution run.

  • Name
    result
    Type
    number
    Description

    Number of tokens used for the result in the execution run.

  • Name
    total
    Type
    number
    Description

    Total number of tokens used in the execution run.

  • Name
    environment
    Type
    string
    Description

    Identifier of the execution environment for the interaction.

  • Name
    temperature
    Type
    number
    Description

    Sampling temperature to control randomness of the model's output.

  • Name
    max_tokens
    Type
    number
    Description

    Maximum number of tokens (words/characters) for the model to generate in the output.

  • Name
    cache_policy
    Type
    CachePolicy
    Description

    Policy determining how the results are cached.

ExecutionRun

{
    "parameters": {
        "object": "snow"
    },
    "tags": [
        "cli",
        "virtual"
    ],
    "environment": {
        "name": "Synthetic",
        "id": "655660c9f19a49a47106075a"
    },
    "interaction": {
        "name": "WhatColor",
        "status": "draft",
        "prompts": [
            {
                "type": "template",
                "template": {
                    "role": "user",
                    "name": "SimplePrompt",
                    "content_type": "jst",
                    "id": "6554cfb57eae1c28ef5f3d4b"
                },
                "id": "6554cfd47eae1c28ef5f3d53"
            }
        ],
        "updated_at": "2023-11-30T18:28:26.808Z",
        "id": "6554cf617eae1c28ef5f3d40"
    },
    "model": "cohere.command-text-v14",
    "status": "completed",
    "ttl": 3600,
    "source": {
        "type": "api",
        "label": "Key Created On 2023-10-24 16:31:28 via API",
        "principal_type": "apikey",
        "principal_id": "653772d5d73c5ac92045df97",
        "client_ip": "::1"
    },
    "account": "652d77895674c387e105948c",
    "project": "654df9de09676ad3b8631dc3",
    "config": {
        "formatter": "genericTextLLM",
        "id": "656766aa862862fbc8d5a2d5"
    },
    "created_at": "2023-11-29T16:28:26.676Z",
    "updated_at": "2023-11-29T16:28:28.490Z",
    "execution_time": 1508,
    "prompt": "User: What color is the snow?\n\nSystem: You must answer using the following JSONSchema:\n        ---\n        {\"type\":\"object\",\"properties\":{\"color\":{\"type\":\"string\"}},\"required\":[\"color\"]}\n        ---",
    "result": {
        "color": "white"
    },
    "token_use": {
        "result": 25,
        "prompt": 198,
        "total": 223
    },
    "modelId": "cohere.command-text-v14",
    "id": "656766aa862862fbc8d5a2d4"
}

List Runs
GET /runs

Get the list of the runs in the current project. \

Parameters

  • Name
    limit
    Type
    number
    Modifier
    optional
    Description

    The number of items to return in the response. Used for pagination. Default is 100.

  • Name
    offset
    Type
    number
    Modifier
    optional
    Description

    The number of items to skip before starting to collect the result set. Used for pagination. Default is 0.

  • Name
    interaction
    Type
    string
    Modifier
    optional
    Description

    Filter by a an interaxction ID. Can be a comma separated list of IDs.

  • Name
    model
    Type
    string
    Modifier
    optional
    Description

    Filter by a model name. Can be a comma separated list of models.

  • Name
    environment
    Type
    string
    Modifier
    optional
    Description

    Filter by an enviroment ID. Can be a comma separated list of IDs.

  • Name
    status
    Type
    string
    Modifier
    optional
    Description

    Filter by the run status. Can be one of: created processing completed failed

  • Name
    tag
    Type
    string
    Modifier
    optional
    Description

    Filter by tag. Can be a comma separated list of tags.

  • Name
    anchorDate
    Type
    ISO date (string)
    Modifier
    optional
    Description

    A date to use as an anchor for the updated_at filter. The date must be in ISO format. The updated_at filter will return all runs updated after or before the anchor date as specified by the anchorDateDirection field.

  • Name
    anchorDateDirection
    Type
    string
    Modifier
    optional
    Description

    The direction to use with anchorDate. Can be one of: before after. The default is after.

  • Name
    parent
    Type
    string|false
    Modifier
    optional
    Description

    Can be either the ID of a parent run or false. If false only top level runs are returned. If a run ID is specified only runs that are children of the specified run are returned.

Returns

A list of run references. A run reference is a subset of the Execution Run object.

Request

curl https://api.composableprompts.com/api/v1/runs?limit=50&interaction=6554cf617eae1c28ef5f3d40&tag=cli \
  -H "Accept: application/json"
  -H "Authorization: Bearer {token}"

Response

[
{
  parameters: {
    object: "flower"
  },
  tags: [ "cli", "virtual" ],
  environment: {
    name: "Synthetic",
    id: "655660c9f19a49a47106075a"
  },
  modelId: "anthropic.claude-v2",
  status: "completed",
  ttl: 3600,
  source: {
    type: "api",
    label: "Key Created On 2023-10-24 16:31:28 via API",
    principal_type: "apikey",
    principal_id: "653772d5d73c5ac92045df97",
    client_ip: "::1"
  },
  account: {
    name: "Dengen Labs",
    id: "652d77895674c387e105948c"
  },
  project: {
    name: "Experiments",
    id: "654df9de09676ad3b8631dc3"
  },
  config: {
    formatter: "claude",
    id: "655cada12a8da9904a723b8b"
  },
  created_at: "2023-11-21T13:16:17.165Z",
  updated_at: "2023-11-21T13:16:20.651Z",
  prompt: "\n\n\nHuman: What color is the flower?\n\n\nHuman: You must answer using the following JSONSchema:\n        ---\n        {\"type\":\"object\",\"properties\":{\"color\":{\"type\":\"string\"}},\"required\":[\"color\"]}\n        ---\n\nAssistant:",
  execution_time: 3324,
  result: {
    color: "yellow"
  },
  token_use: {
    result: 4,
    prompt: 0,
    total: 4
  },
  interaction: {
    name: "WhatName",
    status: "draft",
    prompts: [
      [Object ...]
    ],
    updated_at: "2023-11-24T10:52:51.067Z",
    id: "6554cf617eae1c28ef5f3d40"
  },
  id: "655cada12a8da9904a723b8a"
},
  ...
]

Retrieve an execution run
GET /runs/:runId

Get an execution run object given its ID.

Parameters

No parameters.

Returns

An execution run object.

Request

curl https://api.composableprompts.com/api/v1/runs/655cada12a8da9904a723b8a \
  -H "Accept: application/json"
  -H "Authorization: Bearer {token}"

Response

{
  parameters: {
    object: "flower"
  },
  tags: [ "cli", "testing_session2", "virtual" ],
  environment: {
    name: "Synthetic",
    id: "655660c9f19a49a47106075a"
  },
  modelId: "anthropic.claude-v2",
  status: "completed",
  ttl: 3600,
  source: {
    type: "api",
    label: "Key Created On 2023-10-24 16:31:28 via API",
    principal_type: "apikey",
    principal_id: "653772d5d73c5ac92045df97",
    client_ip: "::1"
  },
  account: "652d77895674c387e105948c",
  project: "654df9de09676ad3b8631dc3",
  config: {
    formatter: "claude",
    id: "655cada12a8da9904a723b8b"
  },
  created_at: "2023-11-21T13:16:17.165Z",
  updated_at: "2023-11-21T13:16:20.651Z",
  prompt: "\n\n\nHuman: What color is the flower?\n\n\nHuman: You must answer using the following JSONSchema:\n        ---\n        {\"type\":\"object\",\"properties\":{\"color\":{\"type\":\"string\"}},\"required\":[\"color\"]}\n        ---\n\nAssistant:",
  execution_time: 3324,
  result: {
    color: "yellow"
  },
  token_use: {
    result: 4,
    prompt: 0,
    total: 4
  },
  interaction: {
    name: "WhatName",
    status: "draft",
    prompts: [
      [Object ...]
    ],
    updated_at: "2023-11-24T10:52:51.067Z",
    id: "6554cf617eae1c28ef5f3d40"
  },
  id: "655cada12a8da9904a723b8a"
}

Search execution runs
POST /runs/search

Search execution runs

Parameters

  • Name
    limit
    Type
    number
    Modifier
    optional
    Description

    The number of items to return in the response. Used for pagination. Defaults to 100.

  • Name
    offset
    Type
    number
    Modifier
    optional
    Description

    The number of items to skip before starting to collect the result set. Used for pagination. Default is 0.

  • Name
    onlyMeta
    Type
    boolean
    Modifier
    optional
    Description

    Whether to return only the facet meta data witout the search results. Defaults to false. It only make sense to use this parameter when the facets parameter is specified.

  • Name
    query
    Type
    string
    Modifier
    false
    Description

    A Lucene query to search the input data object.

  • Name
    defaultQueryPath
    Type
    string
    Modifier
    false
    Description

    Default path for the property to match if no property path is not specified in the query

  • Name
    facets
    Type
    object
    Modifier
    false
    Description

    A map of facet name to facet configuration. If specified the facets are used to generate the facet meta data. that will be returned along with the search results.
    This is usefull when you want need faceted search.

    There are 3 types of facets that can be used:

    • String facet - used to facet on string properties.
    • Numeric facet - used to facet on numeric properties.
    • Date facet - used to facet on date properties.

    String facets has the following properties:

    {
  type: "string",
  path: string,
  numBuckets?: number, // defaults to 10
}

    Numeric and Date facets are similar.

    Here is the shape of a numeric facet

    {
  type: "number",
  path: string,
  boundaries: number[],
  default?: string
}

    And here is the shape of a date facet:

    {
  type: "date",
  path: string,
  boundaries: string[], // dates in  ISO 8601 format
  default?: string
}

    All facet configurations define must define a path property which is the path to the property to facet on. The following properties are available to facet on:

    • modelId - the model used to execute the interaction (string facet).
    • status - the status of the execution run (string facet).
    • tags - the tags used on the execution run (string facet).
    • updated_at - the date the execution run was last updated (date facet).
    • execution_time - the execution time (numeric facet).

    The numBuckets property of the string facet is used to control the number of buckets to use for the facet. It defualts to 10.
    The boundaries is a list of numeric or dates (in ISO 8601 format), in ascending order, that specify the boundaries for each bucket. You must specify at least two boundaries. Each adjacent pair of values acts as the inclusive lower bound and the exclusive upper bound for the bucket.
    The default property is used to specify the bucket name to use for values not matching any of the boundaries.

    Example

    {
  facets: {
    model: {
      type: "string",
      path: "modelId",
      numBuckets: 10,
    },
    date: {
      type: "date",
      path: "updated_at",
      boundaries: [
        "2023-11-27T12:08:23.925Z",
        "2023-10-27T12:08:23.925Z",
        "2023-09-27T12:08:23.925Z"
      ],
      default: 'other'
    }
  }
}
  • Name
    filters
    Type
    object
    Modifier
    false
    Description

    A map of property keys to property values to use for filtering.

    The following execution run properties are available to filter:

    • Name
      interaction
      Type
      string
      Modifier
      optional
      Description

      Filter by a an interaction ID.

    • Name
      environemnt
      Type
      string
      Modifier
      optional
      Description

      Filter by a an environment ID.

    • Name
      parent
      Type
      string|false
      Modifier
      optional
      Description

      Can be either the ID of a parent run or false. If false only top level runs are returned. If a run ID is specified only runs that are children of the specified run are returned.

    • Name
      status
      Type
      string
      Modifier
      optional
      Description

      Filter by a run status. Can be one of: created processing completed failed

    • Name
      tags
      Type
      string[]
      Modifier
      optional
      Description

      Filter by tags. An array of tags.

    • Name
      modelId
      Type
      string
      Modifier
      optional
      Description

      Filter by tags. A comma separated list of tags.

    • Name
      updated_at
      Type
      object
      Modifier
      optional
      Description

      Filter by a range of dates. This is a object containg one or two pairs of the following properties:

      • lt: a iso 8601 date string to match the upper bound of the range (exclusive)
      • lte: a iso 8601 date string to match the upper bound of the range (inclusive)
      • gt: a iso 8601 date string to match the lower bound of the range (exclusive)
      • gte: a iso 8601 date string to match the lower bound of the range (inclusive)

      The range may have only a lower bound, only an upper bound, or both.

Returns

A list of execution runs which match the search criteria.

Request

curl https://api.composableprompts.com/api/v1/runs/search \
  -H "Accept: application/json"
  -H "Authorization: Bearer {token}"
  -X POST
  -d {...}

Response


Same as list execution runs (i.e. /runs) endpoint

Execute an Interaction
POST /runs

This endpoint is creating an execution run by starting the execution of an interaction. There are 2 modes of execution:

  1. blocking: the execution is started and the response is returned when the execution is completed.
  2. streaming: the execution is started and the response is returned immediately as an execution run in created state.
    The client must then start the execution by requesting an event strean as specified by the Server-Sent-Events.
    In the browser you can do this by using the EventSource API. In Node.js you can use a sevrer side implementation of the EventSource API like eventsource.
    The event stream will sent the following events:
  • message - a new chunk of the result data is available. The event data property is the chunk as a string.
  • close - the execution is completed. The event data property is the final execution reuslt object which can either has a status of completed or failed.

To start streaming the execution you should perform a GET request on /runs/:runId/stream.

Parameters

  • Name
    interaction
    Type
    string
    Modifier
    required
    Description

    The interaxction ID to execute.

  • Name
    stream
    Type
    boolean
    Modifier
    optional
    Description

    Wheter or not to stream the execution. Default is false.

  • Name
    data
    Type
    object
    Modifier
    optional
    Description

    The input data to use top generate the prompt.

  • Name
    tags
    Type
    string
    Modifier
    optional
    Description

    The tags to add to the execution run.

  • Name
    config
    Type
    object
    Modifier
    optional
    Description

    A configuration object to customize the execution.

    • Name
      environment
      Type
      string
      Modifier
      optional
      Description

      An execution evnvironemnt ID to be used to generate the test data. If not specified the default interaction environment will be used.

    • Name
      model
      Type
      string
      Modifier
      optional
      Description

      The model to be used to execute the interaction. If not specified the default model of the environment is used.

    • Name
      temperature
      Type
      number
      Modifier
      optional
      Description

      The temperature to be used to execute the data. Defaults to 0.7.

    • Name
      max_tokens
      Type
      number
      Modifier
      optional
      Description

      The maximum number of tokens to be used when generating the result.

    • Name
      formatter
      Type
      number
      Modifier
      optional
      Description

      A custom formatter to use.

Returns

An execution run object.

Request

curl https://api.composableprompts.com/api/v1/runs \
  -H "Accept: application/json"
  -H "Authorization: Bearer {token}"
  -X POST
  -d {...}

Response

{
  result: {
    color: "blue"
  },
  parameters: {
    object: "car"
  },
  tags: [],
  environment: "654dfa3109676ad3b8631e24",
  interaction: "6554cf617eae1c28ef5f3d40",
  modelId: "gpt-4",
  status: "completed",
  ttl: 3600,
  source: {
    type: "api",
    label: "Key Created On 2023-10-24 16:31:28 via API",
    principal_type: "apikey",
    principal_id: "653772d5d73c5ac92045df97",
    client_ip: "::1"
  },
  account: "652d77895674c387e105948c",
  project: "654df9de09676ad3b8631dc3",
  config: {
    environment: "654dfa3109676ad3b8631e24",
    model: "gpt-4",
    id: "656486b6958d9210bae70291"
  },
  input_embedding: [],
  result_embedding: [],
  created_at: "2023-11-27T12:08:22.691Z",
  updated_at: "2023-11-27T12:08:23.925Z",
  execution_time: 1100,
  token_use: {
    prompt: 51,
    result: 9,
    total: 60
  },
  prompt: [
    {
      content: "\n What color is the car?\n",
      role: "user"
    }
  ],
  id: "656486b6958d9210bae70290"
}

Stream an execution run
GET /runs/:runId/stream

Used to start streaming an execution run. The execution run must be in created state.
Must be used after creating a run with stream set to true. \
See [#execute-an-interaction](Execute an Interaction) for more details.

Parameters

No parameters.

Returns

A stream of events as specified by the Server-Sent-Events protocol.

Request

curl -N https://api.composableprompts.com/api/v1/runs/:runId/stream

Was this page helpful?