Skip to content

Requesting retry logic when model is overloaded #164

Description

@sethladd

I'm seeing:

 [WARN] 5 concept(s) planned but only 4 written for foo (ServiceUnavailableError).                        
      summary-rewrite.................openkb.agent.compiler WARNING: summary-rewrite failed for foo:           
  litellm.ServiceUnavailableError: GeminiException - {                                                                                          
    "error": {                                                                                                                                  
      "code": 503,                                                                                                                              
      "message": "This model is currently experiencing high demand. Spikes in demand are usually temporary. Please try again later.",           
      "status": "UNAVAILABLE"                                                                                                                   
    }                                                                                                                                           
  }

Which apparently looks like I lost information? Not sure if there's a retry, but it looks like a retry isn't happening?

Hope this helps!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions