We run a mobile app with an increasing user base. It is hosted on Google App Engine and utilises Google Cloud SQL for storing user data..
Twice in the last 2 months we've woken up to find the CPU usage has spiked to 100%, Memory usage as spiked to near 90% of the available memory (26GB), and the service has become unresponsive despite the database being only around 10GB in size in reality (so it should all fit comfortable in memory). We have been updating the instance to meet memory demands, and have done the basics of indexing commonly queries columns in our tables. We use Redis for Memcaching, so the database is only queried when really necessary.
Resetting the SQL instance does seem to rectify the issue, but of course we'd like them not to happen in the first place.
Unfortunately debugging the failures have now exceeded my capabilities as (predominantly) a mobile application developer, and we're looking for someone to come in and take a look at the logs on CloudSQL and make some database optimisation recommendations with the benefit of actually knowing what they're doing in a slightly more in-depth way than I do.
The errors being reported are basically:
Aborted connection 17843215 to db: 'sleepiest' user: 'sleepiest-34' host: 'cloudsqlproxy' (Got an error reading communication packets)'
We're looking for a MySQL / Database expert that's got experience with Google Cloud Platform, and ideally has debugged issues just like this on GCP before.
About the recuiterMember since May 20, 2018 Kumar Piyush
from al-Fayyum, Egypt