Repository navigation
[Bug] Deleting an application script routing rule makes healthy providers unavailable #16548
Description
Pre-check
- I am sure that all the content I provide is in English.
Search before asking
- I had searched in the issues and found no similar issues.
I also searched for AppScriptStateRouter DELETED, script router rule delete, and No provider available script routing. #16545 covers a separate cross-service router-chain bug; it is not required to reproduce this failure.
Apache Dubbo Component
Java SDK (apache/dubbo)
Affected module: dubbo-cluster; affected router: AppScriptStateRouter.
Dubbo Version
- Apache Dubbo Java SDK 3.3.6 from Maven Central. The error also reproduces in the unmodified 3.3 branch at commit 843b98b using a targeted regression test.
- Amazon Corretto JDK 1.8.0_442 (bundled Nashorn JavaScript engine), Windows 11 build 26200.
- Nacos 2.5.2 standalone as the configuration center, mapped to
127.0.0.1:18848(HTTP) and127.0.0.1:19848(gRPC). - Curator
TestingServer5.9.0 as the ZooKeeper registry at127.0.0.1:23281. - Two provider JVMs and one consumer JVM. No Spring application is needed.
Steps to reproduce this issue
-
Start Nacos and ZooKeeper. The local Nacos command was
docker run -d --name codex-dubbo-script-repro-nacos -e MODE=standalone -e NACOS_AUTH_ENABLE=false -p 127.0.0.1:18848:8848 -p 127.0.0.1:19848:9848 nacos/nacos-server:v2.5.2. Start two separate provider JVMs for the sameOrderProbeService:order-v1at127.0.0.1:23981andorder-v2at127.0.0.1:23982. Both use provider application nameroute-chain-repro-order. The methodString instance()returns the instance ID. -
Publish the following YAML to Nacos group
dubbo, Data IDroute-chain-repro-order.script-router:configVersion: v3.0 scope: application key: route-chain-repro-order runtime: true enabled: true force: true type: javascript script: | (function route(invokers, invocation) { var result = new java.util.ArrayList(); for (var i = 0; i < invokers.size(); i++) { if (invokers.get(i).getUrl().getPort() == 23982) { result.add(invokers.get(i)); } } return result; })(invokers, invocation);
-
Start a single-reference consumer JVM using the same ZooKeeper registry and Nacos config center. Set
roundrobinload balancing andretries=0on the Order reference. Invokeinstance()five times; all five calls returnorder-v2. -
In Nacos, delete that Data ID with
ConfigService.removeConfig("route-chain-repro-order.script-router", "dubbo"). Confirm the Nacos HTTP config endpoint returns 404, wait five seconds for the config notification, then invokeinstance()five more times on the same consumer reference. -
Republish the same YAML, wait five seconds, and invoke
instance()five more times.
The rule lifecycle in the reproducer is:
observe("BEFORE_DELETE", order, 5);
config.removeConfig("route-chain-repro-order.script-router", "dubbo");
waitUntilNacosGetReturns404();
Thread.sleep(5000);
observe("AFTER_DELETE", order, 5);
config.publishConfig("route-chain-repro-order.script-router", "dubbo", rule);
Thread.sleep(5000);
observe("AFTER_RESTORE", order, 5);Each observe call counts the returned provider IDs and caught exceptions. Providers remain registered and healthy throughout. The same failure was also confirmed with a second UserProbeService reference in the consumer when using the independent router-chain fix proposed in PR #16546.
What you expected to happen
Deleting the script rule should disable script filtering. While the rule is absent, both healthy Order providers should remain available for normal load balancing. Republishing the rule should restore v2-only routing. No RPC should fail solely because the rule was deleted.
What actually happened
The original Dubbo 3.3.6 single-reference run returned:
BEFORE_DELETE={order-v2=5}
NACOS_HTTP_AFTER_DELETE=404
AFTER_DELETE={ERROR:RpcException=5}
AFTER_RESTORE={order-v2=5}
The two-reference run with the separate #16545 chain fix had the same 5/5 errors after deletion. The consumer log contains java.lang.NullPointerException at AppScriptStateRouter.doRoute(AppScriptStateRouter.java:69), followed by Failed to execute router and No provider available. The failure happens in a single-reference consumer without the #16545 fix, so the chain fix is not its cause.
Anything else
The DELETED handler in AppScriptStateRouter.process sets scriptRule = null but leaves the compiled scriptRouter non-null. On the next call, doRoute passes the scriptRouter == null check and dereferences scriptRule.isValid(). DynamicDirectory.doList catches the routing exception and returns an empty candidate list, so the cluster reports that no provider is available even though both providers are healthy.
A fix should publish the rule and compiled router together as one state, and clear both on deletion. This also avoids observing mismatched state when config updates race with invocation routing. A regression test can assert routing before deletion, pass-through after deletion, and restored script routing after republishing.
I tested that approach locally against real RPCs using the original Dubbo 3.3.6 router factory and, separately, the #16545 router-chain fix. In both setups, after deletion the five calls reached both providers (order-v1=2, order-v2=3) with zero errors; after republishing, all five reached order-v2. The targeted regression test initially failed with the same NPE on unmodified 3.3 source and passed after the fix.
Do you have a (mini) reproduction demo?
- Yes, I have a minimal reproduction demo to help resolve this issue more effectively!
The local reproducer uses separate provider and consumer JVMs, an embedded ZooKeeper server, Nacos 2.5.2, the YAML above, and the lifecycle code shown above. The provider/consumer setup is the same as the complete runnable source in #16545; the additional dynamic deletion sequence is shown here. PR #16549 includes a regression test.
Are you willing to submit a pull request to fix on your own?
- Yes I am willing to submit a pull request on my own!
The fix and regression test are submitted in PR #16549.
Code of Conduct
- I agree to follow this project's Code of Conduct.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsTodo
Activity