1. Redisson 升级后RateLimiter序列化错误


theme: channing-cyan

某次升级后,RedissonRateLimiter出错,异常信息为:bad argument #2 to 'unpack' (data string too short)

<!-- more -->

问题

问题起因将Redisson从3.15.6升级到了3.16.8,升级原因是解决RedissonJackson版本的冲突

Caused by: java.lang.NoClassDefFoundError: com.fasterxml.jackson.core.JacksonException
 at java.base/java.lang.J9VMInternals.prepareClassImpl(Native Method)
 at java.base/java.lang.J9VMInternals.prepare(Unknown Source)
 at java.base/java.lang.Class.getConstructor(Unknown Source)
 at java.base/java.util.ServiceLoader$1.run(Unknown Source)
 at java.base/java.util.ServiceLoader$1.run(Unknown Source)
 at java.base/java.security.AccessController.doPrivileged(Unknown Source)
 ... 47 common frames omitted

然后升级到3.16.8,这个问题解决了,但是限流开始报错了,报错信息如下

org.redisson.client.RedisException: ERR Error running script (call to f_1378f96e73a4251ef63dadab589a062f8276a044): @user_script:1: user_script:1: bad argument #2 to 'unpack' (data string too short) . channel: [id: 0x6dda80ae, L:/172.16.0.48:33214 - R:redis-9142dab-dcs-rvgi.dcs.huaweicloud.com/192.168.1.90:6379] command: (EVAL), promise: java.util.concurrent.CompletableFuture@b7bb26f0[Not completed, 1 dependents], params: [local rate = redis.call('hget', KEYS[1], 'rate');local interval = redis.call('hget', KEYS[1], 'interval');local type = redis.call('hget', KEYS[1], 'type');assert(rate ~= false and interval ~= false and type ~= false, 'RateLimiter is not initialized')local valueName = KEYS[2];local permitsName = KEYS[4];if type == '1' then valueName = KEYS[3];permitsName = KEYS[5];end;local currentValue = redis.call('get', valueName); if currentValue == false then redis.call('set', valueName, rate); return rate; else local expiredValues = redis.call('zrangebyscore', permitsName, 0, tonumber(ARGV[1]) - interval); local released = 0; for i, v in ipairs(expiredValues) do local random, permits = struct.unpack('Bc0I', v);released = released + permits;end; if released > 0 then redis.call('zremrangebyscore', permitsName, 0, tonumber(ARGV[1]) - interval); currentValue = tonumber(currentValue) + released; redis.call('set', valueName, currentValue);end;return currentValue; end;, 5, xxx-xxx, {xxx-xxx}:value, {xxx-xxx}:value:84b87a92-9cd1-4548-b585-79bb03a9e8f9, {xxx-xxx}:permits, {xxx-xxx}:permits:84b87a92-9cd1-4548-b585-79bb03a9e8f9, 1663684426350]
 at org.redisson.client.handler.CommandDecoder.decode(CommandDecoder.java:370)
 at org.redisson.client.handler.CommandDecoder.decodeCommand(CommandDecoder.java:198)
 at org.redisson.client.handler.CommandDecoder.decode(CommandDecoder.java:137)
 at org.redisson.client.handler.CommandDecoder.decode(CommandDecoder.java:113)
 at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501)
 at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:366)
 at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
 at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
 at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
 at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
 at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
 at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
 at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
 at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
 at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
 at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
 at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
 at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
 at java.base/java.lang.Thread.run(Unknown Source)

实际上不难看出,出错的位置为org.redisson.client.handler.CommandDecoder.decode(CommandDecoder.java:370),基本可以猜测是序列化问题

排查

序列化方式变更

猜测可能是Redisson升级后序列化方式变了(因为Redisson老是在升级版本的时候修改默认的序列化方式),不过经过确认,项目配置了自定义的序列化方式,所以排除此原因

RateLimiter序列化BUG

https://github.com/redisson/redisson/issues/4090

有位老哥遇到了一样的问题,作者说无法复现,建议删除所有键。

这个方案也太暴力了。测试了下,只有RateLimiter有问题,其他的键没有问题,所以只要删除RateLimiter对应的键就行。

删除RateLimiter的键

一般我们都会给Redis的键设置固定前缀,所以我们可以使用AnotherRedisDesktopManager来批量删除指定前缀的键。

RateLimiter会自动给键加上{前缀,所以我们搜索{xxxx,然后直接删除就行

参考资料

作者:IT_Learner

%s 个评论

要回复文章请先登录注册