Keboola - Degraded performance of Kai AI features on EU stacks – Incident details

All systems operational

Degraded performance of Kai AI features on EU stacks

Resolved
Degraded performance
Started 4 months agoLasted about 8 hours

Affected

AWS EU (eu-central-1)

Degraded performance from 3:00 AM to 10:40 AM

GCP EU (europe-west3)

Degraded performance from 3:00 AM to 10:40 AM

Updates
  • Resolved
    UTC
    Resolved

    The situation has stabilized and we are no longer observing elevated 429 responses from GCP Vertex AI on the affected EU stacks. Kai AI features are operating normally again. The incident is now resolved, and we will continue to monitor the situation.

    We apologize for the inconvenience.

  • Investigating
    UTC
    Investigating

    Since approximately 05:00 UTC we have been observing degraded performance of Kai AI features on our EU stacks (connection.eu-central-1.keboola.com and connection.europe-west3.gcp.keboola.com). The issue is caused by rate limiting (HTTP 429 Too Many Requests) from our upstream GCP Vertex AI provider and has been intermittently affecting responses since this morning.

    Users may experience slower responses, failed requests, or errors when interacting with Kai. Other platform functionality is not affected.

    Our team is actively monitoring the situation.

    Next update in 60 minutes.

    We apologize for the inconvenience.