Skip to content

httpoison to req - #3928

Open
kinjalh wants to merge 7 commits into
masterfrom
http-migration
Open

httpoison to req#3928
kinjalh wants to merge 7 commits into
masterfrom
http-migration

Conversation

@kinjalh

@kinjalh kinjalh commented Jul 29, 2026

Copy link
Copy Markdown
Member

use req instead of httpoison

Test Plan

Test environment: https://console.your-env.onplural.sh/

Checklist

  • I have added a meaningful title and summary to convey the impact of this PR to a user.
  • If required, I have updated the Plural documentation accordingly.
  • I have added tests to cover my changes.
  • I have deployed the agent to a test environment and verified that it works as expected (required only when changing agent code).

Plural Flow: console

@kinjalh kinjalh added the backend Changes related to the backend label Jul 29, 2026
@soffi-ai

soffi-ai Bot commented Jul 29, 2026

Copy link
Copy Markdown

Soffi AI Summary

This PR migrates the Elixir server's HTTP client layer from HTTPoison to Req across the entire codebase. The motivation is to standardize on a more modern, composable HTTP client library and eliminate the repetitive boilerplate that HTTPoison required for response handling.

The core change introduces a unified Console.Utils.HTTP helper (with a Http.handle/2 wrapper) that centralizes response decoding, error normalization, and JSON parsing — replacing dozens of copy-pasted case HTTPoison.get/post/... blocks that each individually pattern-matched on %HTTPoison.Response{status_code: ..., body: ...}. With Req, all call sites collapse to a single-line request + |> Http.handle(service_name) pattern.

Affected integrations include: Azure DevOps, GitHub, GitLab, Bitbucket (cloud and datacenter), Hydra (OIDC), Prometheus, Loki, Victoria Metrics, Elasticsearch, Ollama, Datadog, Helm repository fetching, Kubernetes version tables, deprecation/compatibility tables, notification delivery, PR automation webhooks, and cloud add-on compatibility fetching. Effectively every external HTTP call in the Elixir server has been migrated.

Supporting configuration keys were also updated — httpoison_*_options app config keys now have req_*_options counterparts, handled by Console.Utils.HTTP.client_options/2 to allow a smooth transition and environment-level tuning.

A small follow-up of test and cron-related fixes accompanies the migration to ensure the test suite passes cleanly after the library swap.

Commits

Commit Summary
74fb44c Initial migration of HTTP calls from HTTPoison to Req across all major integration clients (Azure DevOps, GitHub, GitLab, Bitbucket, Hydra, Prometheus, Loki, Elasticsearch, Ollama, Datadog, etc.), replacing verbose pattern-match boilerplate with a shared Http.handle/2 helper.
d9f6367 Bug fixes discovered during the initial migration pass — correcting response field references (e.g., status_codestatus) and option handling differences between HTTPoison and Req.
ad2c4ad Refactors the HTTP migration for consistency: introduces Console.Utils.HTTP.client_options/2 to unify app-config key lookup for both legacy HTTPoison and new Req option namespaces, and cleans up remaining call sites.
89a9580 Merge from master to incorporate upstream changes before completing the migration branch.
b55c691 Small bug fix to restore test suite compatibility after the HTTPoison→Req swap (adjusting mock/stub expectations for the new response struct shape).
5836363 Broader test suite fixes — updates test helpers and assertions that were still referencing HTTPoison response structures or options.
2330115 Fixes a cron-related test that broke as a side effect of the HTTP client migration.

Deploy in Soffi


Updated: 2026-08-06 14:32 UTC

defp http_opts,
do:
Application.get_env(:console, :httpoison_azure_devops_options, []) ++ [recv_timeout: 60_000]
Application.get_env(:console, :req_azure_devops_options, []) ++ [receive_timeout: 60_000, decode_body: false, retry: false]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Legacy SCM options are dropped

When a deployment supplies proxy, TLS, or timeout settings through the previously supported :httpoison_*_options keys, these clients now read only the renamed :req_*_options keys, causing required connection settings to disappear and the affected SCM integrations to fail.

@greptile-apps

greptile-apps Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR migrates Console’s direct HTTP calls from HTTPoison to Req.

  • Updates response matching, request options, form encoding, redirects, and error handling across AI, deployment, SCM, metrics, log, and utility clients.
  • Adds an HTTPoison-to-Req option translation helper.
  • Reworks Victoria Metrics and AI proxy response streaming around Req’s asynchronous response API.
  • Updates affected mocks and tests to use Req response and transport-error structures.

Confidence Score: 4/5

The SCM option-key compatibility break should be fixed before merging because configured proxy and TLS settings can disappear after an upgrade.

Four workbench SCM clients stop reading their previously supported application environment keys without a fallback or translation path, causing deployment-specific HTTP settings to be silently omitted.

Files Needing Attention: lib/console/ai/tools/workbench/integration/azure_devops/client.ex, lib/console/ai/tools/workbench/integration/bitbucket/client.ex, lib/console/ai/tools/workbench/integration/bitbucket_datacenter/client.ex, lib/console/ai/tools/workbench/integration/gitlab/client.ex

Important Files Changed

Filename Overview
lib/console/utils/http.ex Adds centralized translation of legacy proxy, timeout, TLS, and redirect options into Req-compatible options.
lib/console/logs/stream/exec.ex Correctly migrates the Victoria Metrics streaming state machine to Req asynchronous responses and handles all documented message forms.
lib/console_web/controllers/ai_controller.ex Migrates the streaming AI reverse proxy to Req while filtering hop-by-hop response headers and cancelling timed-out streams.
lib/console/ai/tools/workbench/integration/azure_devops/client.ex Migrates requests to Req but drops compatibility with the existing HTTP option configuration key.
lib/console/ai/tools/workbench/integration/github/client.ex Migrates GitHub workbench requests and pagination headers to Req while translating Tentacat’s legacy request options.
lib/console/clients/prometheus/client.ex Migrates instant and range Prometheus queries to Req form encoding with equivalent connection and receive timeouts.
lib/console/clients/hydra/client.ex Migrates Hydra administration and OAuth request handling to Req without changing the decoded response contracts.

Reviews (1): Last reviewed commit: "httpoison to req" | Re-trigger Greptile

@kinjalh kinjalh added the enhancement New feature or request label Jul 29, 2026

@michaeljguarino michaeljguarino left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this generally makes sense. Have you done any e2e testing, certainly with things like elasticsearch/prometheus queries, and others that route through workbenches

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend Changes related to the backend enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants