Operations: a/b tests

Create, start, stop and read A/B tests. 6 operations. How to call them, and what the answers mean: Operations reference.

Generated from the operation catalog: src/lib/ops/public-catalog.json in publicsummerengine at 0d85f61e2b and the summer-engine npm package 4.0.0 for engine and local operations. If something here is wrong, the operation’s definition is wrong.

Operation	What it does	Asks you first
`experiments.create`	Make a draft store page test with 2 to 4 versions (A first, usually the published page; others change name, tagline, description, language or art).	Not by default
`experiments.discard`	Discard a draft A/B test that never ran.	Yes, by default
`experiments.list`	The store page A/B tests of one game: each test’s id, state (draft, running, stopped), versions and what each changes, countries and winner.	Never
`experiments.results`	Visitors, purchases, conversion and revenue for each version of one test (default: the running or newest one).	Never
`experiments.start`	Start a draft A/B test, so players see its versions; one test runs per game at a time.	Yes, for now
`experiments.stop`	Stop a running A/B test, with or without a winning version; a winning price becomes the price.	Yes, for now

create

experiments.create: Draft an A/B test Make a draft store page test with 2 to 4 versions (A first, usually the published page; others change name, tagline, description, language or art). Players see nothing until the owner starts it. Next: experiments.start.


MCP	`summer_call` with `"op": "experiments.create"`
CLI	`summer experiments create`
Runs	On Summer’s servers: the hosted MCP, or the npm MCP after `summer login`. Needs your Summer sign-in.
Effect	Changes your own drafts, files or settings. Nobody else sees it yet.
Asks you first	No: Make and build is on by default in Agent permissions. When it is off, your agent asks you first.
3.x names	`summer_grow_ab_tests#create`

Inputs:

Input	Type	Required	Description
`gameId`	string	Yes	From games.list.
`name`	string	Yes	The test name. Length 1 to 80.
`countries`	string[]	No	Only players in these countries (ISO codes). Empty: everyone. Items 0 to 50.
`variants`	object[]	Yes	Version A first: {} keeps the published store page. Items 2 to 4.
`variants[].weight`	integer	No	Whole-percent share of visitors; all add up to 100. Left out everywhere: an even split. Range 1 to 99.
`variants[].name`	string	No	Length 0 to 80.
`variants[].subtitle`	string	No	Length 0 to 160.
`variants[].description`	string	No	Length 0 to 4000.
`variants[].locale`	string	No	Language of this version’s text, e.g. de.
`variants[].art`	object	No	Art ids from the game’s published media.
`variants[].art.keyArt`	string	No
`variants[].art.capsuleWide`	string	No
`variants[].art.capsuleTall`	string	No
`dryRun`	boolean	No	Check and describe the call without doing it: nothing changes, nothing is spent.
`idempotencyKey`	string	No	Retry key. Reuse it only to retry the identical request, so it never happens twice. Length 1 to 128.

Input JSON schema

{
  "type": "object",
  "properties": {
    "gameId": {
      "description": "From games.list.",
      "type": "string",
      "pattern": "^[A-Za-z0-9_-]{1,128}$"
    },
    "name": {
      "description": "The test name.",
      "type": "string",
      "minLength": 1,
      "maxLength": 80
    },
    "countries": {
      "description": "Only players in these countries (ISO codes). Empty: everyone.",
      "maxItems": 50,
      "type": "array",
      "items": {
        "type": "string",
        "pattern": "^[A-Za-z]{2}$"
      }
    },
    "variants": {
      "description": "Version A first: {} keeps the published store page.",
      "minItems": 2,
      "maxItems": 4,
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "weight": {
            "description": "Whole-percent share of visitors; all add up to 100. Left out everywhere: an even split.",
            "type": "integer",
            "minimum": 1,
            "maximum": 99
          },
          "name": {
            "type": "string",
            "maxLength": 80
          },
          "subtitle": {
            "type": "string",
            "maxLength": 160
          },
          "description": {
            "type": "string",
            "maxLength": 4000
          },
          "locale": {
            "description": "Language of this version's text, e.g. de.",
            "type": "string",
            "pattern": "^[a-z]{2,3}(-[A-Z]{2})?$"
          },
          "art": {
            "description": "Art ids from the game's published media.",
            "type": "object",
            "properties": {
              "keyArt": {
                "type": "string",
                "pattern": "^[A-Za-z0-9_-]{1,128}$"
              },
              "capsuleWide": {
                "type": "string",
                "pattern": "^[A-Za-z0-9_-]{1,128}$"
              },
              "capsuleTall": {
                "type": "string",
                "pattern": "^[A-Za-z0-9_-]{1,128}$"
              }
            }
          }
        }
      }
    },
    "dryRun": {
      "description": "Check and describe the call without doing it: nothing changes, nothing is spent.",
      "type": "boolean"
    },
    "idempotencyKey": {
      "description": "Retry key. Reuse it only to retry the identical request, so it never happens twice.",
      "type": "string",
      "minLength": 1,
      "maxLength": 128
    }
  },
  "required": [
    "gameId",
    "name",
    "variants"
  ]
}

Result JSON schema

{
  "type": "object",
  "properties": {
    "game": {
      "type": "object",
      "properties": {
        "id": {
          "type": "string"
        },
        "name": {
          "type": "string"
        }
      },
      "required": [
        "id",
        "name"
      ],
      "additionalProperties": false
    },
    "test": {
      "type": "object",
      "properties": {
        "testId": {
          "type": "string"
        },
        "name": {
          "type": "string"
        },
        "state": {
          "type": "string"
        },
        "versions": {
          "type": "array",
          "items": {
            "type": "object",
            "additionalProperties": {}
          }
        }
      },
      "required": [
        "testId",
        "name",
        "state",
        "versions"
      ],
      "additionalProperties": {}
    },
    "next": {
      "type": "string"
    },
    "studioUrl": {
      "type": "string"
    }
  },
  "required": [
    "game",
    "test",
    "next",
    "studioUrl"
  ],
  "additionalProperties": false
}

Example: Test a new tagline against the published one.

{
  "name": "summer_call",
  "arguments": {
    "op": "experiments.create",
    "input": {
      "gameId": "g_123",
      "name": "Darker tagline",
      "variants": [
        {},
        {
          "subtitle": "Survive until dawn."
        }
      ]
    }
  }
}

summer experiments create --game-id g_123 --name 'Darker tagline' --input '{"variants":[{},{"subtitle":"Survive until dawn."}]}'

discard

experiments.discard: Discard a draft A/B test Discard a draft A/B test that never ran. Players never saw it, and it cannot start again. Runs when the person allows Delete for good; otherwise answers needs_approval with a link. A running test: experiments.stop.


MCP	`summer_call` with `"op": "experiments.discard"`
CLI	`summer experiments discard`
Runs	On Summer’s servers: the hosted MCP, or the npm MCP after `summer login`. Needs your Summer sign-in.
Effect	Removes or stops something for good.
Asks you first	Yes: Delete for good is off by default in Agent permissions.

Inputs:

Input	Type	Required	Description
`gameId`	string	Yes	From games.list.
`testId`	string	Yes	From experiments.list. Length 1 to 128.
`dryRun`	boolean	No	Check and describe the call without doing it: nothing changes, nothing is spent.
`idempotencyKey`	string	No	Retry key. Reuse it only to retry the identical request, so it never happens twice. Length 1 to 128.

Input JSON schema

{
  "type": "object",
  "properties": {
    "gameId": {
      "description": "From games.list.",
      "type": "string",
      "pattern": "^[A-Za-z0-9_-]{1,128}$"
    },
    "testId": {
      "description": "From experiments.list.",
      "type": "string",
      "minLength": 1,
      "maxLength": 128
    },
    "dryRun": {
      "description": "Check and describe the call without doing it: nothing changes, nothing is spent.",
      "type": "boolean"
    },
    "idempotencyKey": {
      "description": "Retry key. Reuse it only to retry the identical request, so it never happens twice.",
      "type": "string",
      "minLength": 1,
      "maxLength": 128
    }
  },
  "required": [
    "gameId",
    "testId"
  ]
}

Result JSON schema

{
  "type": "object",
  "properties": {
    "game": {
      "type": "object",
      "properties": {
        "id": {
          "type": "string"
        },
        "name": {
          "type": "string"
        }
      },
      "required": [
        "id",
        "name"
      ],
      "additionalProperties": false
    },
    "test": {
      "type": "object",
      "properties": {
        "testId": {
          "type": "string"
        },
        "name": {
          "type": "string"
        },
        "state": {
          "type": "string"
        },
        "versions": {
          "type": "array",
          "items": {
            "type": "object",
            "additionalProperties": {}
          }
        }
      },
      "required": [
        "testId",
        "name",
        "state",
        "versions"
      ],
      "additionalProperties": {}
    },
    "next": {
      "type": "string"
    },
    "studioUrl": {
      "type": "string"
    }
  },
  "required": [
    "game",
    "test",
    "studioUrl"
  ],
  "additionalProperties": false
}

Example: Discard the draft test nobody started.

{
  "name": "summer_call",
  "arguments": {
    "op": "experiments.discard",
    "input": {
      "gameId": "g_123",
      "testId": "exp_43"
    }
  }
}

summer experiments discard --game-id g_123 --test-id exp_43

list

experiments.list: List a game’s A/B tests The store page A/B tests of one game: each test’s id, state (draft, running, stopped), versions and what each changes, countries and winner. Next: experiments.results or experiments.start.


MCP	`summer_call` with `"op": "experiments.list"`
CLI	`summer experiments list`
Runs	On Summer’s servers: the hosted MCP, or the npm MCP after `summer login`. Needs your Summer sign-in.
Effect	Reads only. Changes nothing.
Asks you first	No. It changes nothing.
3.x names	`summer_grow_ab_tests`

Inputs:

Input	Type	Required	Description
`gameId`	string	Yes	From games.list.

Input JSON schema

{
  "type": "object",
  "properties": {
    "gameId": {
      "description": "From games.list.",
      "type": "string",
      "pattern": "^[A-Za-z0-9_-]{1,128}$"
    }
  },
  "required": [
    "gameId"
  ]
}

Result JSON schema

{
  "type": "object",
  "properties": {
    "game": {
      "type": "object",
      "properties": {
        "id": {
          "type": "string"
        },
        "name": {
          "type": "string"
        }
      },
      "required": [
        "id",
        "name"
      ],
      "additionalProperties": false
    },
    "tests": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "testId": {
            "type": "string"
          },
          "name": {
            "type": "string"
          },
          "state": {
            "type": "string"
          },
          "versions": {
            "type": "array",
            "items": {
              "type": "object",
              "additionalProperties": {}
            }
          }
        },
        "required": [
          "testId",
          "name",
          "state",
          "versions"
        ],
        "additionalProperties": {}
      }
    },
    "note": {
      "type": "string"
    },
    "studioUrl": {
      "type": "string"
    }
  },
  "required": [
    "game",
    "tests",
    "note",
    "studioUrl"
  ],
  "additionalProperties": false
}

Example: Which tests does this game have?

{
  "name": "summer_call",
  "arguments": {
    "op": "experiments.list",
    "input": {
      "gameId": "g_123"
    }
  }
}

summer experiments list --game-id g_123

results

experiments.results: Read A/B test results Visitors, purchases, conversion and revenue for each version of one test (default: the running or newest one). Next: experiments.stop with a winner when the owner picks one.


MCP	`summer_call` with `"op": "experiments.results"`
CLI	`summer experiments results`
Runs	On Summer’s servers: the hosted MCP, or the npm MCP after `summer login`. Needs your Summer sign-in.
Effect	Reads only. Changes nothing.
Asks you first	No. It changes nothing.
3.x names	`summer_grow_ab_tests#results`

Inputs:

Input	Type	Required	Description
`gameId`	string	Yes	From games.list.
`testId`	string	No	From experiments.list. Length 1 to 128.

Input JSON schema

{
  "type": "object",
  "properties": {
    "gameId": {
      "description": "From games.list.",
      "type": "string",
      "pattern": "^[A-Za-z0-9_-]{1,128}$"
    },
    "testId": {
      "description": "From experiments.list.",
      "type": "string",
      "minLength": 1,
      "maxLength": 128
    }
  },
  "required": [
    "gameId"
  ]
}

Result JSON schema

{
  "type": "object",
  "properties": {
    "game": {
      "type": "object",
      "properties": {
        "id": {
          "type": "string"
        },
        "name": {
          "type": "string"
        }
      },
      "required": [
        "id",
        "name"
      ],
      "additionalProperties": false
    },
    "test": {
      "type": "object",
      "properties": {
        "testId": {
          "type": "string"
        },
        "name": {
          "type": "string"
        },
        "state": {
          "type": "string"
        },
        "versions": {
          "type": "array",
          "items": {
            "type": "object",
            "additionalProperties": {}
          }
        }
      },
      "required": [
        "testId",
        "name",
        "state",
        "versions"
      ],
      "additionalProperties": {}
    },
    "results": {
      "type": "array",
      "items": {
        "type": "object",
        "additionalProperties": {}
      }
    },
    "note": {
      "type": "string"
    },
    "studioUrl": {
      "type": "string"
    }
  },
  "required": [
    "game",
    "test",
    "results",
    "note",
    "studioUrl"
  ],
  "additionalProperties": false
}

Example: How is the running test doing?

{
  "name": "summer_call",
  "arguments": {
    "op": "experiments.results",
    "input": {
      "gameId": "g_123"
    }
  }
}

summer experiments results --game-id g_123

start

experiments.start: Start an A/B test Start a draft A/B test, so players see its versions; one test runs per game at a time. Runs when the person allows Prices and live changes; otherwise answers needs_approval with a link. Next: experiments.results.


MCP	`summer_call` with `"op": "experiments.start"`
CLI	`summer experiments start`
Runs	On Summer’s servers: the hosted MCP, or the npm MCP after `summer login`. Needs your Summer sign-in.
Effect	Changes what players see or pay, or who can act on a game.
Asks you first	Yes, for now: Summer Games still asks you first. Soon it follows Prices and live changes in Agent permissions (on by default).
3.x names	`summer_grow_ab_tests#start`

Inputs:

Input	Type	Required	Description
`gameId`	string	Yes	From games.list.
`testId`	string	Yes	From experiments.list. Length 1 to 128.
`dryRun`	boolean	No	Check and describe the call without doing it: nothing changes, nothing is spent.
`idempotencyKey`	string	No	Retry key. Reuse it only to retry the identical request, so it never happens twice. Length 1 to 128.

Input JSON schema

{
  "type": "object",
  "properties": {
    "gameId": {
      "description": "From games.list.",
      "type": "string",
      "pattern": "^[A-Za-z0-9_-]{1,128}$"
    },
    "testId": {
      "description": "From experiments.list.",
      "type": "string",
      "minLength": 1,
      "maxLength": 128
    },
    "dryRun": {
      "description": "Check and describe the call without doing it: nothing changes, nothing is spent.",
      "type": "boolean"
    },
    "idempotencyKey": {
      "description": "Retry key. Reuse it only to retry the identical request, so it never happens twice.",
      "type": "string",
      "minLength": 1,
      "maxLength": 128
    }
  },
  "required": [
    "gameId",
    "testId"
  ]
}

Result JSON schema

{
  "type": "object",
  "properties": {
    "game": {
      "type": "object",
      "properties": {
        "id": {
          "type": "string"
        },
        "name": {
          "type": "string"
        }
      },
      "required": [
        "id",
        "name"
      ],
      "additionalProperties": false
    },
    "test": {
      "type": "object",
      "properties": {
        "testId": {
          "type": "string"
        },
        "name": {
          "type": "string"
        },
        "state": {
          "type": "string"
        },
        "versions": {
          "type": "array",
          "items": {
            "type": "object",
            "additionalProperties": {}
          }
        }
      },
      "required": [
        "testId",
        "name",
        "state",
        "versions"
      ],
      "additionalProperties": {}
    },
    "next": {
      "type": "string"
    },
    "studioUrl": {
      "type": "string"
    }
  },
  "required": [
    "game",
    "test",
    "studioUrl"
  ],
  "additionalProperties": false
}

Example: Start the drafted tagline test.

{
  "name": "summer_call",
  "arguments": {
    "op": "experiments.start",
    "input": {
      "gameId": "g_123",
      "testId": "exp_42"
    }
  }
}

summer experiments start --game-id g_123 --test-id exp_42

stop

experiments.stop: Stop an A/B test Stop a running A/B test, with or without a winning version; a winning price becomes the price. A draft: experiments.discard. Runs when the person allows Prices and live changes; otherwise answers needs_approval with a link.


MCP	`summer_call` with `"op": "experiments.stop"`
CLI	`summer experiments stop`
Runs	On Summer’s servers: the hosted MCP, or the npm MCP after `summer login`. Needs your Summer sign-in.
Effect	Changes what players see or pay, or who can act on a game.
Asks you first	Yes, for now: Summer Games still asks you first. Soon it follows Prices and live changes in Agent permissions (on by default).
3.x names	`summer_grow_ab_tests#stop`

Inputs:

Input	Type	Required	Description
`gameId`	string	Yes	From games.list.
`testId`	string	Yes	From experiments.list. Length 1 to 128.
`winner`	”A” \| “B” \| “C” \| “D”	No	The winning version, for a running test.
`dryRun`	boolean	No	Check and describe the call without doing it: nothing changes, nothing is spent.
`idempotencyKey`	string	No	Retry key. Reuse it only to retry the identical request, so it never happens twice. Length 1 to 128.

Input JSON schema

{
  "type": "object",
  "properties": {
    "gameId": {
      "description": "From games.list.",
      "type": "string",
      "pattern": "^[A-Za-z0-9_-]{1,128}$"
    },
    "testId": {
      "description": "From experiments.list.",
      "type": "string",
      "minLength": 1,
      "maxLength": 128
    },
    "winner": {
      "description": "The winning version, for a running test.",
      "type": "string",
      "enum": [
        "A",
        "B",
        "C",
        "D"
      ]
    },
    "dryRun": {
      "description": "Check and describe the call without doing it: nothing changes, nothing is spent.",
      "type": "boolean"
    },
    "idempotencyKey": {
      "description": "Retry key. Reuse it only to retry the identical request, so it never happens twice.",
      "type": "string",
      "minLength": 1,
      "maxLength": 128
    }
  },
  "required": [
    "gameId",
    "testId"
  ]
}

Result JSON schema

{
  "type": "object",
  "properties": {
    "game": {
      "type": "object",
      "properties": {
        "id": {
          "type": "string"
        },
        "name": {
          "type": "string"
        }
      },
      "required": [
        "id",
        "name"
      ],
      "additionalProperties": false
    },
    "test": {
      "type": "object",
      "properties": {
        "testId": {
          "type": "string"
        },
        "name": {
          "type": "string"
        },
        "state": {
          "type": "string"
        },
        "versions": {
          "type": "array",
          "items": {
            "type": "object",
            "additionalProperties": {}
          }
        }
      },
      "required": [
        "testId",
        "name",
        "state",
        "versions"
      ],
      "additionalProperties": {}
    },
    "next": {
      "type": "string"
    },
    "studioUrl": {
      "type": "string"
    }
  },
  "required": [
    "game",
    "test",
    "studioUrl"
  ],
  "additionalProperties": false
}

Example: Stop the test; version B won.

{
  "name": "summer_call",
  "arguments": {
    "op": "experiments.stop",
    "input": {
      "gameId": "g_123",
      "testId": "exp_42",
      "winner": "B"
    }
  }
}

summer experiments stop --game-id g_123 --test-id exp_42 --winner B

Overview

Operations

HTTP API

GDScript API

Migration and extensions

MCP

create

discard

list

results

start

stop

​create

​discard

​list

​results

​start

​stop

create

discard

list

results

start

stop