MySQL Keeps Crashing? How to Find the Cause
If your MySQL server is crashing repeatedly, you’re losing data and uptime. This guide walks you through the systematic process of identifying the root cause—from checking error logs and system resources to analyzing query performance and configuration. You’ll learn exactly where to look and what each clue means, so you can fix the issue and prevent future crashes.
What You’ll Need
- SSH (or console) access to the server running MySQL
- sudo or root privileges to read logs and run diagnostic commands
- Basic familiarity with the command line
- Access to the MySQL configuration file (usually
my.cnformy.ini)
Step-by-Step
- Check the MySQL error log. The error log is your first stop. It records startup/shutdown messages, corruption warnings, and crash details. Find its location with
SHOW VARIABLES LIKE 'log_error';or look in the default paths (/var/log/mysql/error.logon Debian/Ubuntu,/var/log/mysqld.logon RHEL/CentOS). Look at the last 100 lines:sudo tail -n 100 /var/log/mysql/error.log. Pay attention to lines containing “[ERROR]”, “[FATAL]”, “InnoDB: Assertion failure”, or “signal SIGSEGV”. - Check the system logs for out-of-memory (OOM) kills. MySQL is often killed by the Linux kernel when memory runs low. Run
sudo dmesg | grep -i -E 'oom|killed'or check/var/log/syslog(or/var/log/messages). Look for lines mentioning the MySQL process (e.g.,mysqld) being killed. If you see it, the crash is due to insufficient RAM or a memory leak. - Inspect the MySQL system variables for clues. If MySQL is running (even briefly), connect with
mysql -u root -pand runSHOW GLOBAL STATUS LIKE 'Uptime';to see how long it’s been up. If it resets often, that confirms frequent restarts. Also runSHOW VARIABLES LIKE 'innodb_buffer_pool_size';andSHOW VARIABLES LIKE 'max_connections';to note current settings—they may be too high for your server’s RAM. - Check disk space and inode usage. A full disk can cause MySQL to crash or fail to write. Run
df -handdf -i. If any mount point is at 100%, free up space or move the MySQL data directory. Also check that the partition containing the data directory (usually/var/lib/mysql) has enough free space. - Review the MySQL configuration for common pitfalls. Open the config file (
sudo nano /etc/mysql/my.cnfor/etc/my.cnf). Look for overly aggressive settings:innodb_buffer_pool_sizelarger than available RAM,max_connectionsset too high, ortmp_table_size/max_heap_table_sizecausing memory exhaustion. A quick rule of thumb: the buffer pool should be about 70-80% of available RAM, and total memory usage (including connections) should not exceed physical RAM. - Enable and examine the slow query log. If crashes happen during heavy load, a runaway query might be the culprit. Enable the slow query log by setting
slow_query_log = 1andslow_query_log_file = /var/log/mysql/mysql-slow.login the config, then restart MySQL. After a crash, check that log for queries with very long execution times or high rows examined. These can indicate missing indexes or inefficient joins that spike memory/CPU. - Monitor system resources in real time. If the crash is intermittent, run
htoportopto watch CPU and memory usage. Also useiostat -x 1to see disk I/O. A sudden spike in memory usage just before the crash often points to a specific query or a connection storm. - Check MySQL’s own status and process list. While MySQL is running, run
SHOW FULL PROCESSLIST;to see active queries. Look for queries that have been running for a long time or are blocking others. Also checkSHOW GLOBAL STATUS LIKE 'Threads_connected';—if it’s hitting the max_connections limit, that can cause aborted connections and crashes. - Test with a minimal configuration. Temporarily comment out all custom settings in
my.cnf(or rename it tomy.cnf.bakand start with defaults). If MySQL runs fine, then one of your custom settings is the problem. Re-add settings one by one to find the culprit. - Check for hardware issues. Run
sudo smartctl -a /dev/sda(replace with your disk) to check for disk errors. Also check RAM withsudo memtest86+(requires reboot). Hardware faults are rare but can cause random crashes.
Common Mistakes
- Ignoring the error log. Many admins focus on system logs or guesswork, but the MySQL error log often contains the exact reason for the crash.
- Assuming it’s a MySQL bug. Most crashes are caused by resource exhaustion (RAM, disk) or misconfiguration, not by MySQL itself.
- Restarting without investigating. If you immediately restart MySQL, you lose the chance to check memory usage, process list, and system logs that contain clues.
- Setting buffer pool too high. Setting
innodb_buffer_pool_sizeto 90% of RAM on a server with other services will cause OOM kills. - Not checking disk space. A full disk can cause InnoDB to crash, and the error log may not be written because the disk is full.
Tips
- Set up log rotation. Ensure error logs and slow query logs are rotated so they don’t fill the disk.
- Use a monitoring tool. Tools like Prometheus, Grafana, or even simple cron scripts can alert you to memory/disk spikes before a crash.
- Keep MySQL updated. Bugs are fixed in newer versions, so staying current can prevent known crash issues.
- Document your config changes. When you change
my.cnf, note why. This helps when reverting changes during troubleshooting. - Test on a staging server. Reproduce the issue on a non-production environment before making changes.
Troubleshooting
If the steps above don’t reveal the cause, try the following:
- If MySQL won’t start at all: Check the error log for the last lines. If it says “InnoDB: Corruption”, you may need to restore from backup or use
innodb_force_recovery(set to 1, then 2, etc.) to start in a limited mode and dump data. - If crashes happen only at a specific time: Set up a cron job to log memory usage (
free -m) and process list every minute to a file. Compare the logs around the crash time. - If the error log mentions “Out of memory”: This is almost always a RAM issue. Reduce
innodb_buffer_pool_size,max_connections, andtmp_table_size. Also consider adding swap space as a temporary measure. - If the error log shows “Segmentation fault”: This can be caused by a bug or a corrupt table. Run
mysqlcheck -u root -p --all-databasesto check and repair tables. If it persists, try upgrading or downgrading MySQL. - If you’re using a shared hosting environment: You may not have access to logs. Contact your host and ask for the error log or ask them to check the OOM killer.
Frequently Asked Questions
How do I find the MySQL error log location?
Run mysql -u root -p -e "SHOW VARIABLES LIKE 'log_error';" to get the path. If MySQL is not running, check common locations like /var/log/mysql/error.log or /var/log/mysqld.log.
What does “Out of memory” in the error log mean?
It means the MySQL server was killed by the operating system because it ran out of RAM. Reduce memory-hungry settings or add more RAM.
Can a single query crash MySQL?
Yes, a badly optimized query can use excessive memory or cause a deadlock that leads to a crash, but it’s more common for it to cause performance issues. Check the slow query log.
Is it safe to use innodb_force_recovery?
It’s a temporary measure to start MySQL in a limited mode to recover data. It should not be used for normal operation. Set it to 1 first and increase gradually, but be aware that some data may be inaccessible.
How often should I check for crashes?
If you’re experiencing crashes, check logs daily. Once resolved, monitor weekly to ensure the issue doesn’t return.
What if the crash is caused by a hardware issue?
Run disk checks (smartctl) and memory tests (memtest86+). If hardware is faulty, replace it and restore from backup.
Related
Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.