For LLM providers that allow setting the PRNG seed for sampling determinism reasons, Kurt should provide a way to do this.
Current expectations of support across LLM providers:
- OpenAI has this as a documented feature
- VertexAI Gemini doesn't appear to have it, but it may be an undocumented feature
- Anthropic Claude doesn't appear to have it.
- Local LLMs should be able to easily support it.
For LLMs which do not support this feature, if the optional seed parameter is given by the user, Kurt should throw a KurtFeatureIncompatibleError (an error class which doesn't yet exist, but I've been thinking about for some time). We should also include a row for this in the #28 to highlight which LLM API providers have and don't have this feature.
For LLM providers that allow setting the PRNG seed for sampling determinism reasons, Kurt should provide a way to do this.
Current expectations of support across LLM providers:
For LLMs which do not support this feature, if the optional
seedparameter is given by the user, Kurt should throw aKurtFeatureIncompatibleError(an error class which doesn't yet exist, but I've been thinking about for some time). We should also include a row for this in the #28 to highlight which LLM API providers have and don't have this feature.