fix: cap HTTP response growth in write_callback

.gitignore

@@ -33,6 +33,7 @@
 
 # CMake / build
 build/
+build-live/
 cmake-build-*/
 CMakeFiles/
 CMakeCache.txt
@@ -57,6 +58,8 @@ _deps/
 # Test artifacts
 downloads/
 test/downloads/
+# written and removed again by the live suite's bulk round trip
+zb-sdk-live.csv
 
 # Misc
 .DS_Store

AGENTS.md

@@ -0,0 +1,80 @@
+# AGENTS.md
+
+Guidance for AI coding agents working in this repository.
+
+## Project
+
+`zerobounce-c-api-wrapper` is the ZeroBounce C SDK. Callers allocate a `ZeroBounce` with `new_zero_bounce_instance`, call `zero_bounce_initialize`, and receive results through success and error callbacks. HTTP goes through libcurl in `src/ZeroBounce.c` and `src/Priority3.c`. There is no package registry: releases are GitHub tags. This is not the C++ repository `zerobounce-cpp-api-wrapper`.
+
+## Layout
+
+- `include/ZeroBounce/`: public headers. `ZeroBounce.h` is the client.
+- `src/ZeroBounce.c`: validation, bulk file, credits, activity, scoring, and the libcurl write callback.
+- `src/Priority3.c`: bulk email finder, domain search, list evaluator, filters, the S3 / SFTP bulk jobs, and the Inbox Placement Tester. The six file jobs share `ZBBulkFileJobOptions` and `p3_bulk_root_file_job`.
+- `src/ZeroBounceWarmup.c` / `include/ZeroBounce/ZeroBounceWarmup.h`: the Warmup API. A separate product on its own host with its own team-scoped key and an `API-Key` header, so it has its own `ZeroBounceWarmup` and never touches the validation key.
+- `src/ZBErrorResponse.c`: turns HTTP failures, HTML pages, and empty or non-JSON bodies into a short `ZBErrorResponse`, with the HTTP status on `status_code`.
+- `test/ZeroBounceTest.c`: existing Unity tests.
+- `test/OfflineApiTest.c`, `test/OfflineWarmupTest.c`: offline suites. `zero_bounce_set_http_perform_for_test` replaces libcurl and records every request.
+- `live/live_api.c`: the opt-in live suite. Built only with `-DZEROBOUNCE_BUILD_LIVE_TESTS=ON`, so the default build and CI never compile it.
+- `CMakeLists.txt`: the only version field, `project(ZeroBounce VERSION x.y.z)`.
+
+## Commands
+
+```bash
+cmake -S . -B build -G "Unix Makefiles"
+cmake --build build          # this is the typecheck; there is no separate lint step
+./build/bin/ZeroBounceTests
+```
+
+CI (`.github/workflows/sdk_ci.yml`) configures, builds, and runs `./build/bin/ZeroBounceTests`. That build is the typecheck. `build/` is generated; do not commit it.
+
+The live suite is opt-in and is never part of that build:
+
+```bash
+cmake -S . -B build-live -DZEROBOUNCE_BUILD_LIVE_TESTS=ON
+cmake --build build-live --target ZeroBounceLiveApi
+export ZEROBOUNCE_API_KEY=... ZEROBOUNCE_WARMUP_API_KEY=...
+ZEROBOUNCE_LIVE_SPEND=1 ZEROBOUNCE_LIVE_CHANGES=1 ./build-live/bin/ZeroBounceLiveApi
+```
+
+Keys come from the environment only. Never write one into a file, a commit, a log, a fixture or a message.
+
+## Conventions
+
+- **One `ZeroBounce` per API key.** `zero_bounce_get_instance()` is a process-wide singleton. Construct a separate object with `new_zero_bounce_instance` per key. The offline concurrency test does this.
+- **Only endpoints documented at https://www.zerobounce.net/docs/.** Do not add a function for an endpoint the public docs do not describe. Check before concluding something is undocumented: the S3 and SFTP bulk endpoints live under https://www.zerobounce.net/docs/api-integrations, not the main API reference.
+- **Four base URLs on `ZeroBounce`.** `api_base_url`, `bulk_api_base_url` (ends in `/v2`), `bulk_api_root_url` (the S3/SFTP jobs, **no** `/v2`), and `list_evaluator_base_url` / `inbox_placement_base_url`. The last two are separate hosts rather than paths built from the bulk base, so `zero_bounce_initialize_with_base_url` alone leaves them on the live API — use the two setters in tests.
+- **`zb_url_escape` is a query encoder.** It turns a space into `+`, which is correct in a query string but a literal plus in a path. Use `p3_escape_path_segment` for anything that goes into a path, such as the list evaluator file id and the Inbox Placement test id.
+- **The live API wins over the docs.** Where they disagree, follow the API and say so in a comment at the call site, prefixed `LIVE:`. The ones found so far: `validatebatch` takes `timeout` as a JSON *string*; the Inbox Placement Tester authenticates with `Authorization: Bearer` and lists tests from `GET /tests` (plural); every S3/SFTP job requires both `file_name` and `return_url`, `validate_sftp_file` included, although the docs call its `file_name` optional; `send_evaluated_file` takes a file id as a form field, not an upload; Warmup's `check-domain-dmarc` takes an *email address*, not a domain; and `test-smtp-configuration` answers a failed SMTP login with HTTP 500.
+- **A `"success": false` field on an HTTP 200 is a failure.** `getfile` and `deletefile` report an unknown or deleted file id that way. `zb_success_means_failure` catches it in `zb_read_success_json`, and `ZBErrorResponse.status_code` carries the HTTP status.
+- **`export_sftp_file` sends `export_path`**, not the `export_bucket` that `export_s3_file` sends.
+- **Safety rails.** `zb_api_key_is_clean` rejects a key with surrounding whitespace or control characters, `zb_check_request_url` allows https only, `p3_escape_path_segment` rejects `.` and `..`, `set_write_callback` turns redirects off and restricts the protocol to https, and `zb_redact_api_key` keeps the key out of messages built from a URL. Use them rather than adding a second rail beside them.
+- **`p3_append_form_field` does not call `zb_note_http_field`.** The offline tests assert on the recorded request body (`OfflineRequest.body`), not on `has_field`, for anything built with it.
+- **Column indexes are 1-based** on bulk send-file name, gender, and IP columns. `0` means the field is left out. `send_file`'s email column is required and is sent as given, including `0`. `ValidateS3FileOptions.email_address_column` uses `-1` for omit, and `0` is sent.
+- **Tri-state ints are `-1` omit, `0` false, `1` true.** `timeout`, `activity_data`, `verify_plus`, `has_header_row`, and `allow_phase_2` follow that. Explicit `0` is sent as `false`.
+- **Booleans on multipart and form bodies are the strings `true` and `false`.**
+- **`allow_phase_2` is validation `send_file` only**, and only when `allow_phase_2_is_set` is true. Scoring `send_file` ignores it. `activity_data` on get-file is validation only.
+- **`validate_email_batch` uses `api.zerobounce.net`**, not `bulkapi.zerobounce.net`. An empty batch IP is JSON `null`.
+- **Failures use the error callback**, not a process exit. The message must be short: no raw HTML, no megabyte body, and no `Error: Error:` prefix. An account-level `validate_email_batch` error (`email_address` `all`) is an error callback. A per-address error stays on the success result.
+- **Tests replace HTTP** with `zero_bounce_set_http_perform_for_test`. Read `zero_bounce_test_method`, `zero_bounce_test_url`, `zero_bounce_test_body`, `zero_bounce_test_extra_header`, and `zero_bounce_test_field_*` from that callback. They are thread-local. Do not call the live API from anything under `test/`.
+- **Never from the live suite:** creating an Inbox Placement test (it cannot be deleted while running), setting the Inbox Placement webhook (it cannot be cleared), or ZB managed Warmup onboarding (it provisions a mailbox). Everything the `ZEROBOUNCE_LIVE_CHANGES=1` tier creates is removed again in the same run.
+- Match the surrounding style. Do not reformat unrelated code.
+
+## Changelog
+
+`CHANGELOG.md` follows [Keep a Changelog 1.1.0](https://keepachangelog.com/en/1.1.0/), and the project adheres to [Semantic Versioning 2.0.0](https://semver.org/spec/v2.0.0.html).
+
+- Add every user-visible change to `## [Unreleased]` in the same commit as the change.
+- Group entries under these headings, in this order, omitting empty ones: `Added`, `Changed`, `Deprecated`, `Removed`, `Fixed`, `Security`.
+- Write for SDK users: name the function, option, or behaviour, and say what changed. Leave out CI-only changes, internal refactors, and fixes to code that was never released.
+- At 2.y.z, a breaking change bumps the major version, a new function or option bumps the minor version, and fixes alone bump the patch version.
+
+## Releasing
+
+Publishing and pushing tags need a maintainer's explicit approval. Do not bump the version, tag, or publish unless they asked.
+
+1. Move `[Unreleased]` entries under `## [x.y.z] - YYYY-MM-DD`, leave an empty `[Unreleased]` section, and update the compare links at the bottom of `CHANGELOG.md`.
+2. Set the same version in `project(ZeroBounce VERSION …)` in `CMakeLists.txt`. That is the only version field. There is no lockfile and no runtime version constant.
+3. `cmake --build build` and `./build/bin/ZeroBounceTests`.
+3a. Run the live suite by hand with real keys and both flags set, and paste its summary line into the release notes. The offline suite cannot catch an API that has changed shape underneath the SDK.
+4. Tag `vx.y.z` on that commit and run the Publish workflow with that tag. It creates the GitHub release. There is no package registry.

CHANGELOG.md

@@ -0,0 +1,130 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+### Added
+
+- `validate_email_with_options` and `validate_email_batch_with_options` take `ZBValidateOptions`: `timeout`, `activity_data`, `verify_plus`, and `use_post` (form-encoded `POST /validate`).
+- `ZBValidateResponse` reads `active_in_days` and `active_first_seen`.
+- `score_email` (`GET /v2/scoring`) and `score_email_post` (form-encoded `POST /v2/scoring`).
+- Bulk email finder (`send_email_finder_file`, `email_finder_file_status`, `email_finder_get_file`, `email_finder_delete_file`).
+- Bulk domain search (`send_domain_search_file`, `domain_search_file_status`, `domain_search_get_file`, `domain_search_delete_file`).
+- List evaluator (`send_list_evaluator_file`, `get_list_evaluator_result`, `send_evaluated_file`).
+- Filters (`add_filter`, `list_filters`, `delete_filter`).
+- S3 bulk (`validate_s3_file`, `score_s3_file`, `export_s3_file`, `list_s3_files`, `email_finder_s3_file`, `domain_search_s3_file`).
+- SFTP/FTP bulk (`validate_sftp_file`, `score_sftp_file`, `email_finder_sftp_file`, `domain_search_sftp_file`, `export_sftp_file`), sharing the new `ZBBulkFileJobOptions`. The server, path and protocol come from your ZeroBounce settings. `validate_sftp_file`'s `file_name` is optional; `export_sftp_file` takes `export_path` rather than the `export_bucket` `export_s3_file` sends.
+- `ZBValidateOptions` accepts `domain_info`, `greylist_processing` and `callback_url`. `callback_url` needs `greylist_processing` set to 1 and is rejected without it, and when blank, before a request is sent.
+- `ZBValidateResponse` reads `domain_website_exists`, `domain_registrant_company_name` and `message`. `message` is the API's `Message` key, returned instead of a result while a greylist retry is still running.
+- Inbox Placement Tester on `https://bulkapi.zerobounce.net/api/ipt`: `get_inbox_placement_providers`, `get_inbox_placement_test_types`, `create_inbox_placement_test`, `get_inbox_placement_test`, `list_inbox_placement_tests`, `delete_inbox_placement_test`, `get_inbox_placement_webhook`, `set_inbox_placement_webhook`. It authenticates with an `Authorization: Bearer` header, bodies are camelCase JSON, and unset fields are left out.
+- `zero_bounce_set_list_evaluator_base_url` and `zero_bounce_set_inbox_placement_base_url`.
+- `zb_make_request_with_method_and_body` and `zb_make_request_with_extra_header`, for the Tester's PUT and DELETE and its Bearer header.
+- `ZeroBounceWarmup`, a client for the [Warmup API](https://www.zerobounce.net/docs/warmup-api): a separate product with its own host, its own team-scoped key (`zero_bounce_warmup_initialize`, sent in an `API-Key` header) and all 26 documented endpoints. Every call hands the raw JSON body to its callback, or an empty string for the documented empty successes (HTTP 204 and an empty 200); `warmup_download_seeds` and `warmup_download_statistics` hand over CSV text and write it to a path when one is given.
+- `ZBErrorResponse.status_code` carries the HTTP status the request came back with, so a caller can tell 429 from 500 without parsing the message. It is 0 when the error was raised before a response arrived.
+- Safety rails: `zb_api_key_is_clean` rejects a key with surrounding whitespace or control characters (both initializers run it), `zb_check_request_url` refuses anything that is not `https://`, `p3_escape_path_segment` rejects a `.` or `..` id, redirects are never followed and only the https protocol is enabled, and `zb_redact_api_key` strips the key out of any message built from a URL. A regional base URL paired with the single global bulk host warns once.
+- `get_file_streaming` and `scoring_get_file_streaming` write a result file straight to disk as it arrives instead of buffering it in memory first, for a download too large to hold. They are opt-in because a streamed body cannot be inspected as a whole: the first 4 KB are held back and checked, so a JSON or HTML error still reaches the error callback and no partial file is left behind.
+- `live/live_api.c`, an opt-in live suite built only with `-DZEROBOUNCE_BUILD_LIVE_TESTS=ON`, so the default build and CI never compile it. Keys come from `ZEROBOUNCE_API_KEY` and `ZEROBOUNCE_WARMUP_API_KEY`; `ZEROBOUNCE_LIVE_SPEND=1` and `ZEROBOUNCE_LIVE_CHANGES=1` opt in to calls that spend credits or make reversible account changes.
+
+### Changed
+
+- Verified against the live API. Where it disagrees with the docs, the API wins and the call site says so:
+  - `validate_email_batch_with_options` sends `timeout` as a JSON **string**. Sent as a number the API rejects the whole batch with the account-level "Invalid API Key or your account ran out of credits".
+  - The Inbox Placement Tester authenticates with `Authorization: Bearer`, not the `api_key` query parameter the rest of the API takes, and its listing is `GET /tests` (plural) — `/test` answers `Method "GET" not allowed`.
+  - Every S3/SFTP job requires both `file_name` and `return_url`, `validate_sftp_file` included, although the docs call its `file_name` optional. Both are checked before the request is sent.
+  - **Breaking:** `send_evaluated_file`'s second argument is the id of a finished evaluation, not a path to upload. The live endpoint takes the file id as a form field.
+- A `"success": false` field on an HTTP 200 now reaches the error callback instead of being returned as a result. `get_file` and `delete_file` report an unknown or deleted file id that way.
+- The list evaluator uses its own base URL (`.../v2/listevaluator`) instead of building the path from the bulk base, and the file id is encoded as a single path segment. `send_evaluated_file` stays on the bulk base.
+
+- API requests time out after 120 seconds.
+- Query values, including the API key, email, and file id, are percent-encoded.
+- JSON API responses are capped at 16 MB and bulk file downloads at 256 MB.
+
+### Fixed
+
+- `get_api_usage` read `sub_status_anti_spam_system`, but the API sends that counter as `sub_status_antispam_system`, so it was always 0. The struct field keeps its name; only the key it reads is corrected.
+- `get_api_usage` now reads `sub_status_leading_period_removed` and `sub_status_alias_address`, two counters the API returns that were silently discarded.
+
+- Form-encoded request bodies no longer start with stray bytes. The builder grew its buffer with `realloc(NULL, n)`, which hands back uninitialized memory, and then appended to it without terminating it first, so whatever was in that block went out in front of the first field. Depending on what the allocator handed back, `add_filter`, `send_evaluated_file` and the S3/SFTP jobs could be answered with a Cloudflare block instead of a result.
+
+- A 403 HTML "Access denied" page is returned as a short error. An HTML page with status 200 is an error as well, including `get_file`, `email_finder_get_file`, and `domain_search_get_file`, instead of a saved file.
+- Empty and non-JSON bodies are a short error. A repeated `Error:` / `[Error]:` prefix is removed.
+- `validate_email_batch` reports an account-level failure (HTTP 200, error for address `all`, such as an invalid API key or no credits) through the error callback. An error for one address stays on the successful result.
+- Bulk email finder and domain search send `has_header_row` when it is set to false.
+- `validate_s3_file` sends `email_address_column` when it is set to 0. `validate_s3_file`, `score_s3_file`, and `export_s3_file` reject an empty API key before a request is sent.
+- `validate_email_batch` sends an empty IP address as JSON `null`.
+
+### Security
+
+- Query parameters are percent-encoded, so a value cannot rewrite the request URL.
+- Error callbacks no longer include the raw HTML access-denied page or a multi-megabyte body.
+
+## [2.1.3] - 2026-06-18
+
+### Fixed
+
+- `validate_email_batch` posts to `api.zerobounce.net/v2/validatebatch` instead of `bulkapi.zerobounce.net`.
+
+## [2.1.2] - 2026-06-10
+
+### Added
+
+- `ZBValidateResponse.catchall_domain`, read from `catchall_domain`.
+
+## [2.1.0] - 2026-04-05
+
+### Added
+
+- `get_file_with_options` and `scoring_get_file_with_options`, with `GetFileOptions.download_type` (`phase_1`, `phase_2`, `combined`) and `activity_data` (validation `get_file` only).
+- `SendFileOptions.allow_phase_2` / `allow_phase_2_is_set`, sent as `allow_phase_2` on validation `send_file` only.
+- `ZBFileStatusResponse.file_phase_2_status`.
+
+### Fixed
+
+- `get_file` calls the error callback when the body is a JSON error, including HTTP 200 with `"success": false`, instead of saving that body as the results file.
+
+## [2.0.10] - 2026-03-03
+
+### Changed
+
+- License the project under MIT.
+
+## [2.0.0] - 2026-03-03
+
+### Added
+
+- `find_email_by_domain_*`, `find_email_by_company_name_*`, `search_domain_by_domain`, and `search_domain_by_company_name`.
+- `zero_bounce_initialize_with_base_url` to select the API base URL.
+- `ZBValidateSubStatus` value `AcceptAll`.
+
+### Changed
+
+- The library builds as C17 and uses libcurl for HTTP.
+
+### Deprecated
+
+- `find_email`. Use the email-finder or domain-search functions instead.
+
+## [1.1.0] - 2023-09-07
+
+### Added
+
+- Email finder response types and `find_email`.
+
+## [1.0.0] - 2023-08-22
+
+### Added
+
+- Initial release: `get_credits`, `get_api_usage`, `validate_email`, `validate_email_batch`, bulk and AI-scoring file functions, and `get_activity_data`.
+
+[Unreleased]: https://github.com/zerobounce/zerobounce-c-api-wrapper/compare/v2.1.4...HEAD
+[2.1.3]: https://github.com/zerobounce/zerobounce-c-api-wrapper/compare/v2.1.2...v2.1.3
+[2.1.2]: https://github.com/zerobounce/zerobounce-c-api-wrapper/compare/v2.1.0...v2.1.2
+[2.1.0]: https://github.com/zerobounce/zerobounce-c-api-wrapper/compare/v2.0.10...v2.1.0
+[2.0.10]: https://github.com/zerobounce/zerobounce-c-api-wrapper/compare/v2.0.0...v2.0.10
+[2.0.0]: https://github.com/zerobounce/zerobounce-c-api-wrapper/compare/v1.1.0...v2.0.0
+[1.1.0]: https://github.com/zerobounce/zerobounce-c-api-wrapper/compare/v1.0.0...v1.1.0
+[1.0.0]: https://github.com/zerobounce/zerobounce-c-api-wrapper/releases/tag/v1.0.0

CMakeLists.txt

@@ -57,6 +57,16 @@ target_include_directories(${PROJECT_NAME} PUBLIC include ${json-c_SOURCE_DIR})
 target_link_libraries(${PROJECT_NAME} PUBLIC json-c libcurl)
 
 enable_testing()
-add_executable(ZeroBounceTests test/ZeroBounceTest.c)
+add_executable(ZeroBounceTests test/ZeroBounceTest.c test/OfflineApiTest.c test/OfflineWarmupTest.c)
 target_link_libraries(ZeroBounceTests PRIVATE ${PROJECT_NAME} unity)
 target_include_directories(ZeroBounceTests PRIVATE ${fff_SOURCE_DIR})
+
+# The live suite calls the real API and spends credits, so it is opt-in and
+# never built by a plain `cmake --build build`:
+#   cmake -S . -B build-live -DZEROBOUNCE_BUILD_LIVE_TESTS=ON
+#   cmake --build build-live --target ZeroBounceLiveApi
+option(ZEROBOUNCE_BUILD_LIVE_TESTS "Build the opt-in live API suite" OFF)
+if(ZEROBOUNCE_BUILD_LIVE_TESTS)
+  add_executable(ZeroBounceLiveApi live/live_api.c)
+  target_link_libraries(ZeroBounceLiveApi PRIVATE ${PROJECT_NAME})
+endif()