RESOURCE_EXHAUSTED: etcdserver: mvcc: database space exceeded

Error message

reason

etcd is full, not enough space

Resolve

1. Compression space, defragmentation
2. Clear old data
3. Extend

Detailed steps

View space details

# View details$ etcdctl --endpoints=ip:port endpoint status --write-out="table"

Manual compression

# 1. Get the current version$ rev=$(etcdctl --endpoints=ip:port endpoint status --write-out="json" | egrep -o '"revision":[0-9]*' | egrep -o '[0-9].*')# 2. Compress all records before the current version$ etcdctl compact $rev# 3. Clean up excess debris space$ etcdctl defrag# 4. Remove the warning$ etcdctl alarm disarm

Auto Compression Settings

# keep an hour of history$ etcd --auto-compaction-retention=1

Official Documentationhttps://etcd.io/docs/v3.4/op-guide/maintenance/#history-compaction