• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

Python小程序扫描清理Redis中的key

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

场景

项目中很多地方使用Redis,有的用于缓存,有的直接做为存储,有的key设置有过期,有的key没有过期时间。
随着时间增长,Redis存储数据越来越多,消耗内存不断增长;
无论测试或生产环境,总内存是有限的;
有的key可能临时或测试使用的;
于是有了清理Redis key的需求。

Redis命令

查看key个数:
dbsize
info keyspace

查看内存情况:
info memory

通配符扫描key:
SCAN cursor [MATCH pattern] [COUNT count]

Python小程序

Linux服务器一般都自带Python,这里用Python编写2个小程序,分别用于扫描key和扫描并删除key。

redis-scan.py(扫描key):

#!/usr/bin/env python
# Scan keys in Redis.
# Author: cdfive
from redis import Redis
import time


def RedisScan(host, port, password, db, cursor, pattern, count):
    start_time = time.time()
    client = Redis(host=host, port=port, password=password, db=db)
    counts, other_cursor_counts = 0, 0
    while True:
        cursor, keys = client.scan(cursor, pattern, count)
        length = len(keys)
        if length > 0:
            for key in keys:
                counts += 1
                print("[%s][%s]%s,cursor=%s" % (
                    int(time.time() - start_time), counts, key.decode("utf-8"), cursor))
        else:
            other_cursor_counts += 1
            print("[%s][other_curosr]other_cursor_counts=%s,cursor=%s" % (
                int(time.time() - start_time), other_cursor_counts, cursor))

        if cursor == 0:
            break
    client.close()
    print("[%s]counts=%s,cursor=%s,other_cursor_counts=%s"
          % (int(time.time() - start_time), counts, cursor, other_cursor_counts))


RedisScan("localhost", 6379, "123456", 0, 0, "*xxx*", 1000)

redis-scan-and-delete.py(扫描并删除key):

#!/usr/bin/env python
# Scan and delete keys in Redis.
# Author: cdfive
from redis import Redis
import time


def RedisScanAndDelete(host, port, password, db, cursor, pattern, count, batch_delete_size):
    start_time = time.time()
    client = Redis(host=host, port=port, password=password, db=db)
    counts, other_cursor_counts = 0, 0
    delete_keys = []
    while True:
        cursor, keys = client.scan(cursor, pattern, count)
        length = len(keys)
        if length > 0:
            index = 0
            for key in keys:
                index += 1
                counts += 1
                print("[%s][%s]%s,cursor=%s" % (
                    int(time.time() - start_time), counts, key.decode("utf-8"), cursor))

                delete_keys.append(key)
                if (len(delete_keys) >= batch_delete_size or index >= length) and len(delete_keys) > 0:
                    client.delete(*delete_keys)
                    delete_keys = []
        else:
            other_cursor_counts += 1
            print("[%s][other_curosr]other_cursor_counts=%s,cursor=%s" % (
                int(time.time() - start_time), other_cursor_counts, cursor))

        if cursor == 0:
            break
    client.close()
    print("[%s]counts=%s,cursor=%s,other_cursor_counts=%s"
          % (int(time.time() - start_time), counts, cursor, other_cursor_counts))


RedisScanAndDelete("localhost", 6379, "123456", 0, 0, "*xxx*", 1000, 100)

注:RedisScanAndDelete最后1个参数为batch_delete_size,用于通过pipeline批量删除key,提高删除效率。


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap