Keboola - Degraded performance of Kai AI features on EU stacks – Incident details

All systems operational

Degraded performance of Kai AI features on EU stacks

Resolved
Partial outage
Started 4 months agoLasted about 5 hours

Affected

AWS EU (eu-central-1)

Partial outage from 11:00 AM to 3:33 PM

GCP EU (europe-west3)

Partial outage from 11:00 AM to 3:33 PM

Azure NE (north-europe)

Partial outage from 11:00 AM to 3:33 PM

Updates
  • Resolved
    UTC
    Resolved

    We have seen no more throttled requests since 16:40. The incident is resolved.

  • Monitoring
    UTC
    Monitoring

    We see substantial reduction of throttled requests. Only one request was throttled in the past 15 minutes.

  • Identified
    UTC
    Identified

    Since 11:00 UTC we're observing degraded performance of Kai AI features on our EU stacks (connection.eu-central-1.keboola.com and connection.europe-west3.gcp.keboola.com and connection.north-europe.azure.keboola.com) . The issue is caused by rate limiting (HTTP 429 Too Many Requests) from our upstream GCP Vertex AI provider and has been intermittently affecting responses.

    Users may experience slower responses, failed requests, or errors when interacting with Kai. Other platform functionality is not affected.

    Our team is actively monitoring the situation.

    Next update in 60 minutes.

    We apologize for the inconvenience.