Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 37 additions & 32 deletions src/flexmeasures_client/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -459,11 +459,11 @@ def report_rate_limit(message: str) -> None:
"Request retry timeout while connecting to the API."
) from exception

# request_once has already parsed this answer with aiohttp's json(),
# which refuses a content type that is not JSON, and the loop above reports that as a ConnectionError.
# A check that used to follow here refused the JSON variants aiohttp accepts too, such as application/problem+json,
# with an error that could not even be printed, so it is gone.
# A body that is not JSON under a JSON content type is refused by json() itself, as a JSONDecodeError.
# request_once has already parsed this response with aiohttp's json().
# json() refuses a content type that is not JSON, and the loop above reports that as a ConnectionError.
# A check used to follow here, but it also refused JSON variants that aiohttp accepts, such as application/problem+json.
# Its error could not even be printed, so the check was removed.
# A body that is not JSON under a JSON content type makes json() itself raise a JSONDecodeError.
return await response.json(), response.status

async def request_once(
Expand Down Expand Up @@ -677,22 +677,24 @@ async def post_sensor_data(

The method automatically chooses the appropriate API endpoint based on the provided parameters.

Since FlexMeasures 0.33.0, a server with an ingestion worker available answers ``202 Accepted`` with the id of a background job,
rather than ``200 OK``, and ingests the data after the request has returned.
By default this method then waits for that job to finish,
so that a caller which reads the data back, or asks for a schedule over it, sees what it just posted.
Since FlexMeasures 0.33.0, a server with an available ingestion worker responds ``202 Accepted`` instead of ``200 OK``,
with the id of a background job.
The server then ingests the data after the request has returned.
By default, this method waits for that job to finish.
This way, a caller that reads the data back, or asks for a schedule over it, sees the data it just posted.

:param await_ingestion: whether to wait for asynchronous ingestion to finish,
when the server answered ``202 Accepted`` with a job id.
Waiting is polled on the ``ingestion_polling_*`` profile, which is tighter than the one other jobs are polled on.
when the server responded ``202 Accepted`` with a job id.
The job is polled on the ``ingestion_polling_*`` profile, which is tighter than the profile for other jobs.
A job that fails raises :class:`JobFailedError`.
A job still pending when ``self.ingestion_polling_timeout`` is reached is logged as an error and otherwise ignored,
since the data may well land moments later, and a caller that has waited that long is better served by carrying on than by failing.
A job that is still pending when ``self.ingestion_polling_timeout`` is reached is logged as an error and otherwise ignored.
The data may well land moments later, and a caller that has waited that long is better off carrying on than failing.
Pass ``False`` to return as soon as the server has accepted the data.
A synchronous ``200 OK`` response is unaffected either way.
:param confirm_ingestion: whether ingestion that could not be confirmed while waiting for it raises, rather than being logged and ignored:
:class:`JobTimeoutError` where the job was still pending at the timeout, and what the lookup raised where looking the job up failed.
For a caller that remembers having recorded the data, and would otherwise never record it again.
Either way, a synchronous ``200 OK`` response is unaffected.
:param confirm_ingestion: whether to raise when ingestion could not be confirmed while waiting for it, instead of logging and ignoring that.
If the job was still pending at the timeout, this raises :class:`JobTimeoutError`.
If looking the job up failed, this raises the error from the lookup.
Use this if the caller remembers having recorded the data, and would otherwise never record it again.

This function raises a ValueError when an unhandled status code is returned.
"""
Expand Down Expand Up @@ -778,18 +780,21 @@ async def _await_sensor_data_ingestion(
) -> None:
"""Wait for an ingestion job to finish, so that the data it carries can be read back.

Waiting is left to :func:`wait_for_job`, so ingestion is backed off like every other job this client waits for,
and reported through the same ``job_status_notifier``, on the ``ingestion_polling_*`` intervals rather than the ``job_polling_*`` ones.

A failed job propagates as :class:`JobFailedError`, for the caller to handle like any other failed post.
Nothing else does.
A timeout is logged as an error and otherwise ignored, for the reason given in :func:`post_sensor_data`,
and so is a failure to look the job up at all, which says nothing about the data:
the server accepted that, and a job record can be gone, or answered for by something in front of the server, by the time it is asked about.
Raising there would report a post that succeeded as a post that failed,
unless the caller asked for ingestion to be confirmed, in which case both are raised once logged.
A lookup that 404s is reported as a ``ConnectionError`` by :func:`request`, which wraps every client error that way,
so that is among the ones caught here, alongside the ``ValueError`` an unexpected status code raises.
Waiting is delegated to :func:`wait_for_job`.
So ingestion is backed off like every other job this client waits for,
and reported through the same ``job_status_notifier``.
The only difference is that it uses the ``ingestion_polling_*`` intervals, instead of the ``job_polling_*`` ones.

A failed job raises :class:`JobFailedError`, which the caller handles like any other failed post.
By default, no other error is raised.
A timeout is logged as an error and otherwise ignored, for the reason given in :func:`post_sensor_data`.
The same goes for a failure to look the job up at all, which says nothing about the data.
After all, the server did accept the data.
By the time the job is looked up, its record may be gone, or something in front of the server may respond instead.
Raising in these cases would report a successful post as a failed post.
If the caller asked for ingestion to be confirmed, both cases are logged and then raised.
A lookup that returns 404 is reported as a ``ConnectionError`` by :func:`request`, which wraps every client error that way.
So ``ConnectionError`` is among the errors caught here, alongside the ``ValueError`` that an unexpected status code raises.
"""
try:
await self.wait_for_job(
Expand Down Expand Up @@ -1555,10 +1560,10 @@ async def add_asset(
}

This function raises a ValueError when an unhandled status code is returned,
when no generic_asset_type_id is given, and when only one of latitude and longitude is.
when no generic_asset_type_id is given, and when only one of latitude and longitude is given.
"""
if generic_asset_type_id is None:
# Optional in the signature only so that latitude and longitude, before it, can be left out,
# The signature makes this optional only so that latitude and longitude, which come before it, can be left out,
# while every argument keeps its position.
raise ValueError("add_asset needs a generic_asset_type_id.")
if (latitude is None) != (longitude is None):
Expand All @@ -1570,7 +1575,7 @@ async def add_asset(
account_id=account_id,
generic_asset_type_id=generic_asset_type_id,
)
# A location is optional to FlexMeasures, which many assets have no use for, such as one standing for a community.
# FlexMeasures does not require a location, and many assets have no use for one, such as an asset that represents a community.
if latitude is not None:
asset["latitude"] = latitude
if longitude is not None:
Expand Down
4 changes: 2 additions & 2 deletions src/flexmeasures_client/s2/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ def __init__(self, max_size: int = 100, timezone: str = "UTC") -> None:
self.outgoing_messages_status = SizeLimitOrderedDict(max_size=max_size)

self._timezone = ZoneInfo(timezone)
#: The time this handler goes by; a CEM gives each handler it registers its own clock.
#: The clock this handler reads the time from. A CEM sets this to its own clock for each handler it registers.
self._clock: Clock = WallClock()

self.discover()
Expand Down Expand Up @@ -253,5 +253,5 @@ def handle_revoke_object(self, message: RevokeObject) -> ReceptionStatus:
)

def now(self) -> datetime:
"""The current moment by this handler's clock, in its time zone."""
"""Return the current moment by this handler's clock, in the handler's timezone."""
return self._clock.now().astimezone(self._timezone)
Loading
Loading