Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .fern/metadata.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{
"cliVersion": "4.28.0",
"generatorName": "fernapi/fern-python-sdk",
"generatorVersion": "4.64.0",
"generatorConfig": {
"client": {
"class_name": "BaseNewscatcherApi",
"filename": "base_client.py",
"exported_class_name": "NewscatcherApi",
"exported_filename": "client.py"
}
},
"originGitCommit": "ea3775d5199c3ed3efc0866eb726d2ed2141aeba",
"sdkVersion": "2.0.0"
}
2 changes: 1 addition & 1 deletion .fernignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,4 @@ test-mock-mode.sh
mypy.ini

# CI/CD
# .github/workflows/ci.yml
.github/workflows/ci.yml
63 changes: 11 additions & 52 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@ name: ci

on: [push, pull_request]

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: false

jobs:
compile:
runs-on: ubuntu-latest
Expand All @@ -18,19 +22,14 @@ jobs:
- name: Install dependencies
run: |
poetry install
# Install additional dependencies if they exist in pyproject.toml or requirements
pip install types-tqdm python-dotenv python-dateutil tqdm types-python-dateutil || echo "Optional dependencies not installed"
- name: Compile
run: |
# Check if we have custom mypy config and custom files
if [ -f "mypy.ini" ] && [ -f "src/newscatcher/client.py" ] && [ -f "src/newscatcher/utils.py" ]; then
# Use custom configuration for custom files
poetry run mypy --config-file=mypy.ini src/newscatcher/client.py src/newscatcher/utils.py tests/ || echo "Custom mypy check completed with warnings"
# Check core files with relaxed settings
poetry run mypy --config-file=mypy.ini --disable-error-code=no-any-return src/newscatcher/core/ || echo "Core mypy check completed with warnings"
else
# Use default Fern approach - check everything
poetry run mypy . || echo "Default mypy check completed with warnings"
poetry run mypy .
fi

test:
Expand All @@ -48,54 +47,15 @@ jobs:
- name: Install dependencies
run: |
poetry install
# Ensure all required dependencies are installed
poetry install --with dev || echo "Dev dependencies not available"
# Install the package in development mode
pip install -e .
# Install additional test dependencies for custom tests
pip install pytest pytest-asyncio python-dateutil tqdm python-dotenv httpx || echo "Optional test dependencies not installed"
- name: Generate mock data
run: |
# Only run if mock script exists
if [ -f "test-mock-mode.sh" ]; then
mkdir -p tests/mocks
python -c "$(cat test-mock-mode.sh | sed -n '/python -c/,/EOF/p' | grep -v 'python -c' | grep -v EOF)" || echo "Mock generation skipped"
fi
- name: Test
env:
NEWSCATCHER_API_KEY: ${{ secrets.NEWSCATCHER_API_KEY }}
TEST_MODE: "mock"
run: |
# Run different test strategies based on what exists
if [ -d "tests/utils" ] || [ -d "tests/custom" ] || [ -d "tests/integration" ]; then
echo "Running custom test suite"
# Run utils tests
if [ -d "tests/utils" ]; then
poetry run pytest -rP tests/utils || echo "Utils tests completed with issues"
fi
# Run custom tests
if [ -d "tests/custom" ] || [ -d "tests/integration" ]; then
pytest tests/custom tests/integration -v || echo "Custom tests completed with issues"
fi
if [ -d "tests/utils" ]; then
poetry run pytest -rP tests/utils
else
echo "Running default Fern test approach"
poetry run pytest -rP . || echo "Default tests completed with issues"
poetry run pytest -rP .
fi
- name: SDK validation test
run: |
# Install package in development mode for validation
pip install -e .
# Always run basic SDK validation
python -c "
try:
from newscatcher import NewscatcherApi
print('SDK import successful')
client = NewscatcherApi(api_key='test-key')
print('SDK instantiation successful')
except Exception as e:
print(f'SDK validation failed: {e}')
exit(1)
"

publish:
needs: [compile, test]
Expand All @@ -115,8 +75,7 @@ jobs:
run: poetry install
- name: Publish to pypi
run: |
poetry config repositories.remote https://upload.pypi.org/legacy/
poetry --no-interaction -v publish --build --repository remote --username "$PYPI_USERNAME" --password "$PYPI_PASSWORD"
poetry config pypi-token.pypi "$PYPI_TOKEN"
poetry --no-interaction -v publish --build
env:
PYPI_USERNAME: ${{ secrets.PYPI_USERNAME }}
PYPI_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }}
113 changes: 64 additions & 49 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,28 @@
[![fern shield](https://img.shields.io/badge/%F0%9F%8C%BF-Built%20with%20Fern-brightgreen)](https://buildwithfern.com?utm_source=github&utm_medium=github&utm_campaign=readme&utm_source=https%3A%2F%2Fgithub.com%2FNewscatcher%2Fnewscatcher-python)
[![pypi](https://img.shields.io/pypi/v/newscatcher-sdk)](https://pypi.python.org/pypi/newscatcher-sdk)

The Newscatcher Python library gives you convenient access to the Newscatcher API from Python.
The Newscatcher Python library provides convenient access to the Newscatcher APIs from Python.

## Table of Contents

- [Documentation](#documentation)
- [Installation](#installation)
- [Reference](#reference)
- [Usage](#usage)
- [Async Client](#async-client)
- [Exception Handling](#exception-handling)
- [Retrieving More Articles](#retrieving-more-articles)
- [Query Validation](#query-validation)
- [Advanced](#advanced)
- [Access Raw Response Data](#access-raw-response-data)
- [Retries](#retries)
- [Timeouts](#timeouts)
- [Custom Client](#custom-client)
- [Contributing](#contributing)

## Documentation

View API reference documentation at [newscatcherapi.com/docs](https://www.newscatcherapi.com/docs/v3/api-reference).
API reference documentation is available [here](https://www.newscatcherapi.com/docs/news-api/api-reference/overview).

## Installation

Expand All @@ -17,72 +34,53 @@ pip install newscatcher-sdk

## Reference

A full reference for this library is available [here](./reference.md).
A full reference for this library is available [here](https://github.com/Newscatcher/newscatcher-python/blob/HEAD/./reference.md).

## Usage

Create and use the client:
Instantiate and use the client with the following:

```python
import datetime

from newscatcher import NewscatcherApi

client = NewscatcherApi(
api_key="YOUR_API_KEY",
api_key="<value>",
)

client.search.post(
q="renewable energy",
predefined_sources=["top 50 US"],
lang=["en"],
from_=datetime.datetime.fromisoformat(
"2024-01-01 00:00:00+00:00",
),
to=datetime.datetime.fromisoformat(
"2024-06-30 00:00:00+00:00",
),
additional_domain_info=True,
is_news_domain=True,
q="\"supply chain\" AND Amazon NOT China",
page_size=1,
)
```

## Async client
## Async Client

Use the `AsyncNewscatcherApi` client to make non-blocking calls to the API:
The SDK also exports an `async` client so that you can make non-blocking calls to our API. Note that if you are constructing an Async httpx client class to pass into this client, use `httpx.AsyncClient()` instead of `httpx.Client()` (e.g. for the `httpx_client` parameter of this client).

```python
import asyncio
import datetime

from newscatcher import AsyncNewscatcherApi

client = AsyncNewscatcherApi(
api_key="YOUR_API_KEY",
api_key="<value>",
)


async def main() -> None:
await client.search.post(
q="renewable energy",
predefined_sources=["top 50 US"],
lang=["en"],
from_=datetime.datetime.fromisoformat(
"2024-01-01 00:00:00+00:00",
),
to=datetime.datetime.fromisoformat(
"2024-06-30 00:00:00+00:00",
),
additional_domain_info=True,
is_news_domain=True,
q="\"supply chain\" AND Amazon NOT China",
page_size=1,
)


asyncio.run(main())
```

## Exception handling
## Exception Handling

The SDK raises an `ApiError` when the API returns a non-success status code (4xx or 5xx response):
When the API returns a non-success status code (4xx or 5xx response), a subclass of the following error
will be thrown.

```python
from newscatcher.core.api_error import ApiError
Expand Down Expand Up @@ -180,17 +178,34 @@ For complete validation rules, bulk validation techniques, and troubleshooting,

## Advanced

### Access Raw Response Data

The SDK provides access to raw response data, including headers, through the `.with_raw_response` property.
The `.with_raw_response` property returns a "raw" client that can be used to access the `.headers` and `.data` attributes.

```python
from newscatcher import NewscatcherApi

client = NewscatcherApi(...)
response = client.search.with_raw_response.post(...)
print(response.headers) # access the response headers
print(response.status_code) # access the response status code
print(response.data) # access the underlying object
```

### Retries

The SDK includes automatic retries with exponential backoff. The SDK retries a request when the request is retriable and the number of retry attempts is less than the configured retry limit (default: 2).
The SDK is instrumented with automatic retries with exponential backoff. A request will be retried as long
as the request is deemed retryable and the number of retry attempts has not grown larger than the configured
retry limit (default: 2).

The SDK retries requests when the API returns these HTTP status codes:
A request is deemed retryable when any of the following HTTP status codes is returned:

- [408](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/408) (Timeout)
- [429](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/429) (Too Many Requests)
- [5XX](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500) (Internal Server Errors)

Use the `max_retries` request option to configure this behavior:
Use the `max_retries` request option to configure this behavior.

```python
client.search.post(..., request_options={
Expand All @@ -200,27 +215,23 @@ client.search.post(..., request_options={

### Timeouts

The SDK uses a 60-second timeout by default. Configure timeouts at the client or request level:
The SDK defaults to a 60 second timeout. You can configure this with a timeout option at the client or request level.

```python

from newscatcher import NewscatcherApi

client = NewscatcherApi(
...,
timeout=20.0,
)

client = NewscatcherApi(..., timeout=20.0)

# Override timeout for a specific method
client.search.post(..., request_options={
"timeout_in_seconds": 1
})
```

### Custom client
### Custom Client

Override the `httpx` client to customize it for your use case. Common use cases include proxies and transports:
You can override the `httpx` client to customize it for your use-case. Some common use-cases include support for proxies
and transports.

```python
import httpx
Expand All @@ -229,14 +240,18 @@ from newscatcher import NewscatcherApi
client = NewscatcherApi(
...,
httpx_client=httpx.Client(
proxies="http://my.test.proxy.example.com",
proxy="http://my.test.proxy.example.com",
transport=httpx.HTTPTransport(local_address="0.0.0.0"),
),
)
```

## Contributing

We value open-source contributions to this SDK. This library is generated programmatically, but we can implement custom methods and use `.fernignore` to preserve certain files. However, implementing custom solutions in the SDK involves a complex process. Please open an issue first to discuss your ideas before submitting a PR.
While we value open-source contributions to this SDK, this library is generated programmatically.
Additions made directly to this library would have to be moved over to our generation code,
otherwise they would be overwritten upon the next generated release. Feel free to open a PR as
a proof of concept, but know that we will not be able to merge it as-is. We suggest opening
an issue first to discuss with us!

On the other hand, contributions to the README are always very welcome!
Loading
Loading