Skip to content

Separate health check readiness from Stellar dependency checks #521

Closed
#542
Closed
Separate health check readiness from Stellar dependency checks#521
#542
Assignees
TheHalalHunter
Labels
Stellar WaveIssues in the Stellar wave programenhancementNew feature or request

Description

@DeFiVC
DeFiVC
opened on Sep 30, 2026

Bug Description

The health check endpoint at /health (lines 209-235 in src/server.ts) calls Stellar Horizon and Soroban RPC as part of the health check. If either Stellar service is down, the health check returns 503 even though the API itself is healthy.

Location

src/server.ts lines 209-235

The Problem

The health check performs 4 checks:

  1. Database (SELECT 1)
  2. Redis (PING)
  3. Stellar Horizon (root call)
  4. Soroban RPC (health check)

If any of these fail, the entire health check returns 503 "degraded". This means:

  • A Stellar network outage makes the API appear "down" to load balancers
  • Kubernetes/Docker health checks may restart healthy API instances
  • Users cannot access non-Stellar features (course browsing, quiz taking) during Stellar outages

Recommended Fix

Separate the health check into:

  1. Liveness (/health/live) - Is the process alive? (always 200)
  2. Readiness (/health/ready) - Can it serve requests? (DB + Redis only)
  3. Full Health (/health) - Detailed status including external dependencies

The readiness check should only verify core infrastructure (DB + Redis), not external blockchain services. Stellar-dependent endpoints can handle their own degradation.

Current State

Note: /health/live already exists and returns { status: "ok" }. However, /health/ready has the same 4-check structure as /health, making it equally fragile.

Acceptance Criteria

  • /health/live - Returns 200 if process is alive (already done)
  • /health/ready - Returns 200 if DB and Redis are healthy (remove Stellar checks)
  • /health - Keep as detailed status endpoint for monitoring dashboards
  • Update documentation to explain the difference between readiness and liveness

Severity

medium - Can cause unnecessary container restarts and load balancer failures during Stellar outages.

Activity

  1. DeFiVC
    added
    enhancementNew feature or request
    on Sep 30, 2026

  2. guptakumarranjeet150 commented on Sep 30, 2026

    @guptakumarranjeet150
    guptakumarranjeet150
    on Sep 30, 2026

    Hey team,

    I am interested in working on this issue.
    Plan:

    1. Build out the handlers for Separate health check readiness from Stellar dependency checks ensuring proper HTTP status codes and error propagation.
    2. Add integration test suites for standard invocations and boundary/error conditions.

    Looking forward to collaborating on this—please assign!

  3. drips-wave
    added
    Stellar WaveIssues in the Stellar wave program
    on Sep 30, 2026

  4. Emoji-dot commented on Sep 30, 2026

    @Emoji-dot
    Emoji-dot
    on Sep 30, 2026 – with Drips Wave

    @Emoji-dot has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

    I would like to work on this issue

    ℹ️ Repo Maintainers: To accept this application, review their application or assign @Emoji-dot to this issue.

  5. Peolite001 commented on Sep 30, 2026

    @Peolite001
    Peolite001
    on Sep 30, 2026 – with Drips Wave

    @Peolite001 has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

    I would love to work on this issue. Kindly assign me

    ℹ️ Repo Maintainers: To accept this application, review their application or assign @Peolite001 to this issue.

  6. adeboladee commented on Sep 30, 2026

    @adeboladee
    adeboladee
    on Sep 30, 2026 – with Drips Wave

    @adeboladee has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

    Kindly assign this task to me maintainer,i would fix this bug and conflict free

    ℹ️ Repo Maintainers: To accept this application, review their application or assign @adeboladee to this issue.

  7. kolawoleoluwaferanmi014-a11y commented on Sep 30, 2026

    @kolawoleoluwaferanmi014-a11y
    kolawoleoluwaferanmi014-a11y
    on Sep 30, 2026 – with Drips Wave

    @kolawoleoluwaferanmi014-a11y has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

    Hello Maintainer, I’m interested in working on this issue and have reviewed the requirements carefully. I’m confident I can deliver a quality implementation that satisfies the acceptance criteria. Please consider assigning it to me.

    ℹ️ Repo Maintainers: To accept this application, review their application or assign @kolawoleoluwaferanmi014-a11y to this issue.

  8. Onefie10 commented on Sep 30, 2026

    @Onefie10
    Onefie10
    on Sep 30, 2026 – with Drips Wave

    @Onefie10 has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

    Hello. I want to work on this before 2pm. Thank you.

    ℹ️ Repo Maintainers: To accept this application, review their application or assign @Onefie10 to this issue.

  9. Fayedamz commented on Sep 30, 2026

    @Fayedamz
    Fayedamz
    on Sep 30, 2026 – with Drips Wave

    @Fayedamz has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

    Hello Maintainer, I'd love to work on this issue.

    ℹ️ Repo Maintainers: To accept this application, review their application or assign @Fayedamz to this issue.

  10. official-rex commented on Sep 30, 2026

    @official-rex
    official-rex
    on Sep 30, 2026 – with Drips Wave

    @official-rex has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

    I’m interested in working on this can deliver before the end of wave

    ℹ️ Repo Maintainers: To accept this application, review their application or assign @official-rex to this issue.

  11. joewee-x commented on Sep 30, 2026

    @joewee-x
    joewee-x
    on Sep 30, 2026 – with Drips Wave

    @joewee-x has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

    Hi Maintainer, I'm a smart contract/full-stack dev (Rust, Soroban, Cairo, TypeScript) focused on secure, clean blockchain code. I've reviewed the issue and can deliver a complete solution within 24 hours. Please assign it to me

    ℹ️ Repo Maintainers: To accept this application, review their application or assign @joewee-x to this issue.

  12. ife-daniel commented on Sep 30, 2026

    @ife-daniel
    ife-daniel
    on Sep 30, 2026 – with Drips Wave

    @ife-daniel has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

    I can do the recommended fixes
    Assign me

    ℹ️ Repo Maintainers: To accept this application, review their application or assign @ife-daniel to this issue.

  13. Kao-thar commented on Sep 30, 2026

    @Kao-thar
    Kao-thar
    on Sep 30, 2026 – with Drips Wave

    @Kao-thar has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

    Hi, I have looked at the issue and I understand what's needed. I would love to work on this.

    ℹ️ Repo Maintainers: To accept this application, review their application or assign @Kao-thar to this issue.

  14. dammy-lab commented on Sep 30, 2026

    @dammy-lab
    dammy-lab
    on Sep 30, 2026 – with Drips Wave

    @dammy-lab has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

    I can work on this issue because it is a focused backend reliability and health-check improvement involving API infrastructure, dependency isolation, and service availability. The main work is to separate process liveness from application readiness and external dependency health.

    ℹ️ Repo Maintainers: To accept this application, review their application or assign @dammy-lab to this issue.

  15. oscar24357 commented on Sep 30, 2026

    @oscar24357
    oscar24357
    on Sep 30, 2026 – with Drips Wave

    @oscar24357 has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

    Hello maintainer, can I work on this issue please.

    ℹ️ Repo Maintainers: To accept this application, review their application or assign @oscar24357 to this issue.

  16. TheHalalHunter commented on Sep 30, 2026

    @TheHalalHunter
    TheHalalHunter
    on Sep 30, 2026 – with Drips Wave
    Contributor

    @TheHalalHunter has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

    Hi maintainer
    I’d love to help out with this issue. I’ve checked the current codebase/documentation and know exactly what needs to be updated to make this clearer. I will ensure everything aligns perfectly with your existing contribution guidelines.

    ℹ️ Repo Maintainers: To accept this application, review their application or assign @TheHalalHunter to this issue.

  17. drips-wave commented on Sep 30, 2026

    @drips-wave
    drips-waveBot
    on Sep 30, 2026 – with Drips Wave

    Congratulations, @TheHalalHunter! 🎉 Your application was accepted by the repo's maintainers, and the issue is due on September 30, 2026.

    🧑‍💻 @TheHalalHunter: Please resolve the issue such that the repo's maintainers have enough time to review your contribution before the due date. You'll earn Points for completing the issue on-time, which will make you eligible for a share of the Stellar Wave Program's reward pool.

    Warning

    When opening a PR, please link it to this issue to ensure it gets tracked accurately. Points are awarded when this issue is marked as completed by the maintainer.

    🤠 Repo maintainers: Please keep an eye on the contributor's progress and review their work before the due date. You can manage this issue, including adjusting its complexity and points, here.

    🌊 Happy Wave 🌊

  18. DeFiVC
    assigned
    TheHalalHunter
    on Sep 30, 2026
  19. DeFiVC
    added a commit that references this issue on Oct 1, 2026

    fix: resolve issues #521 #522 #523 #526

    92ceda0
  20. DeFiVC
    closed this as completedin #542on Oct 1, 2026
  21. DeFiVC
    added a commit that references this issue on Oct 1, 2026

    Merge pull request #542 from TheHalalHunter/fix/521-522-523-526-healt…

    6cdfac1

  22. grantfox-oss commented on Oct 1, 2026

    @grantfox-oss
    grantfox-ossBot
    on Oct 1, 2026 – with GrantFox OSS

    🎉 This issue has been marked as completed on GrantFox!

    @TheHalalHunter's PR #542 was approved and merged by @DeFiVC.

    🏆 @TheHalalHunter: You earned 35 FoxPoints for this contribution! Your current tier: Explorer (608 total points). Track your full progress on GrantFox.

    👏 Great work, @TheHalalHunter! Keep contributing to ChainLearnOfficial.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Stellar WaveIssues in the Stellar wave programenhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions