2 weeks ago
Choose which AI models your tracks can use, and cap their tokens
You can now choose which AI models a track can call and set a token limit for each model. This keeps the AI cost of every play predictable. It works with Amazon Bedrock on AWS accounts and Vertex AI on Google Cloud projects.
How it works
Calls to models you haven’t selected are denied. If a play goes over a model’s token limit, the play is stopped.
Set it up
- Open the track’s sandbox settings and go to the cloud account.
- Under Services, enable Bedrock or Vertex AI.
- Select the models the track can use.
- Optionally, set input and output token limits for each model.
You can also configure this under service_limits in config.yml.
Good to know
- Limits are set per model, per cloud account and per track, and each play is counted on its own. There is no org-wide limit.
- Usage is checked by polling the cloud provider’s metrics, so a play can go slightly past its limit before it is stopped. Treat the limit as a cost guardrail, not an exact cutoff.
- Selecting a model doesn’t grant access to it. On AWS, the account still needs a policy that allows Bedrock, such as AmazonBedrockFullAccess. Without that policy, every call is denied.
- Only text models on Bedrock and Vertex AI are supported. Image and video generation models and other providers are not.
- If you need a model that isn’t on the supported list, contact Support and we’ll review it.
- This is available on tracks only. Coming soon to labs