RedisException: MOVED 3889 / AWS MemoryDB

Resolved shvayapin
(@shvayapin)

2 years, 3 months ago
Hello Till,

first of all thank you for providing this awesome plugin.

Until yesterday, it worked perfectly. We did not change any configuration, did not update plugins or anything else on the page. Today we’ve got this error and a broken WordPress instance:
```
NOTICE: PHP message: RedisException: MOVED 3889 xxx-0001-002.xxx.xxx.memorydb.sa-east-1.amazonaws.com:6379 in /usr/src/wordpress/wp-content/object-cache.php:1922
```
We configured WP_REDIS_HOST with clustercfg.xxx.xxx.memorydb.sa-east-1.amazonaws.com. As far as I’m understanding predis as well as phpredis, both are able to resolve clusters without providing a dedicated cluster configuration.

Obviously xxx-0001-002. is resolved via clustercfg.

Now we’re not able to get the page up and running with object cache enabled.

Redis is running as AWS MemoryDB with multiple nodes and shards. As said – worked until today.

I’m not sure how to continue here. Any ideas?

Thank you in advance and best regards,
Sascha

Viewing 4 replies - 1 through 4 (of 4 total)

Plugin Author Till Krüss
(@tillkruess)

2 years, 3 months ago
That sounds like you’re using a real Redis Cluster (cluster_enabled).

The WP_REDIS_HOST won’t work, because Redis Object Cache needs to know all primary cluster nodes. Instead use:
```
define( 'WP_REDIS_CLUSTER', [
    'tcp://127.0.0.1:6379?alias=primary-01',
    'tcp://127.0.0.2:6379?alias=primary-02',
    'tcp://127.0.0.3:6379?alias=primary-03',
] );
```
Whether you have multiple or a single primary node, you must use the WP_REDIS_CLUSTER configuration, instead of WP_REDIS_HOST.
Thread Starter shvayapin
(@shvayapin)

2 years, 3 months ago
Hi @tillkruess
thank you very much for your fast response. I’m aware of that this is no standard and we’re grateful for that.

Well actually we tried this as well. None of the configurations we tried is working. Once switching to WP_REDIS_CLUSTER, we’re getting as error:
```
epoll_wait() reported that client prematurely closed connection, so upstream connection is closed too while reading upstream
```
We need to connect via tls, with user and password. Logging in into each of our nodes from the WordPress container via redis-cli works fine:
```
redis-cli -h xxx-0001-001.xxx.xxx.memorydb.sa-east-1.amazonaws.com --user $REDIS_USER --pass $REDIS_PASSWORD --tls
```
As said, the connection via WP_REDIS_HOST with the following URL once worked (so I can assume that in general all the required data arrives at the php code:
```
clustercfg.xxx.xxx.memorydb.sa-east-1.amazonaws.com
```
So I’m wondering why this leads to a connection close:
```
define( 'WP_REDIS_CLUSTER', [
    'tcp://xxx-0001-001.xxx.xxx.memorydb.sa-east-1.amazonaws.com:6379',
    'tcp://xxx-0001-002.xxx.xxx.memorydb.sa-east-1.amazonaws.com:6379',
    'tcp://xxx-0002-001.xxx.xxx.memorydb.sa-east-1.amazonaws.com:6379',
    'tcp://xxx-0002-002.xxx.xxx.memorydb.sa-east-1.amazonaws.com:6379',
] );
```
Also tried the protocols redis, rediss and tls.

Any idea how to continue from here?
Thread Starter shvayapin
(@shvayapin)

2 years, 3 months ago
Of course the epoll_wait error is comming from nginx. The fpm does simply time out.

To exclude the option that the epoll_wait timeout is due to missing memory or cpu, we removed the container limits temporarily for the container. Also we raised the fpm pm values:
```
pm.max_children = 25
pm.start_servers = 5
pm.min_spare_servers = 5
pm.max_spare_servers = 10
pm.max_requests = 500
```
Should be sufficient enough…
- This reply was modified 2 years, 3 months ago by shvayapin.
Plugin Author Till Krüss
(@tillkruess)

2 years, 3 months ago

I’d suggest making a redis.php file on your EC2 instance and trying to connect manually via PhpRedis, once you figured out which configuration allow you to connect to your Elasticache Cluster, then I can tell you which WP_REDIS_* constants to use.

Using WP_REDIS_HOST will not work, because MOVED responses aren’t handled.

You may also try using Predis, maybe that works better?

You can bypass the Nginx issues by using php CLI first.

I know that people use Elasticache Clusters with Redis Object Cache, these forums may have some code snippets.

Viewing 4 replies - 1 through 4 (of 4 total)

The topic ‘RedisException: MOVED 3889 / AWS MemoryDB’ is closed to new replies.

Tags