Skip to content

Backport[IMPALA-14466]: Remote client should not cache admissiond's IP when retrying AdmitQuery RPC - #16

Open
giggsoff wants to merge 1 commit into
develop/4.4.0/4.5.0.3from
hotfix/NGSOK-2204
Open

giggsoff wants to merge 1 commit into
develop/4.4.0/4.5.0.3from
hotfix/NGSOK-2204

Conversation

@giggsoff

Copy link
Copy Markdown
Collaborator

The remote admission client's retry logic for AdmitQuery RPC did not handle cases where the admissiond restarts with a new IP address. The client would use the old proxy and retry against the old, stale ip, causing queries to time out.

This change fixes the issue by adding the GetProxy() call inside the retry loop. This forces the client to re-resolve the admissiond's network address on each retry attempt, allowing it to discover the new endpoint and successfully reconnect.

…rying AdmitQuery RPC

The remote admission client's retry logic for AdmitQuery RPC did not
handle cases where the admissiond restarts with a new IP address.
The client would use the old proxy and retry against the old, stale
ip, causing queries to time out.

This change fixes the issue by adding the GetProxy() call inside the
retry loop. This forces the client to re-resolve the admissiond's
network address on each retry attempt, allowing it to discover the
new endpoint and successfully reconnect.

Tests:
Passed admissiond related exhaustive ee tests.
Since automatically change hosts might be difficult, manually test
to change the /etc/hosts with following steps:
1. Start with --admission_service_host=localhost.
2. Change the 'localhost' in /etc/hosts to an inaccessible IP,
like 127.0.0.2.
3. Submit a query, it will block in the retry logic.
4. While the query is blocked, change 'localhost' in /etc/hosts
back to 127.0.0.1.
5. The query succeeded.

Change-Id: I5857de84ce69902b902099f668e87d747f944aff
Reviewed-on: http://gerrit.cloudera.org:8080/23472
Reviewed-by: Abhishek Rawat <arawat@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
(cherry picked from commit 63dee74)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants