Skip to content

[google-genai] blob upload failed to parse the path using latest version of fsspec[gcs] #4161

Closed

Description

@JeffLuoo

Describe your environment

OS: (e.g, Ubuntu) Linux
Python version: (e.g., Python 3.9.10)
Package version: (e.g., 0.46.0) latest

What happened?

Using latest version of fsspec[gcs] in ADK agent for prompts and responses upload failed with error:

/usr/local/lib/python3.13/site-packages/google/adk/cli/adk_web_server.py:475: UserWarning: [EXPERIMENTAL] App: This feature is experimental and may change or be removed in future versions without notice. It may introduce breaking changes at any time.
  agentic_app = App(
/usr/local/lib/python3.13/site-packages/opentelemetry/sdk/_logs/_internal/__init__.py:337: LogDeprecatedInitWarning: LogRecord init with `trace_id`, `span_id`, and/or `trace_flags` is deprecated since 1.35.0. Use `context` instead.
  return cls(
INFO:     34.162.183.88:40558 - "GET /debug/trace/session/faa1ef56-6649-440a-b431-1bf5d2422b49 HTTP/1.1" 200 OK
INFO:     10.138.15.196:40559 - "GET /apps/capital_agent/users/user/sessions/faa1ef56-6649-440a-b431-1bf5d2422b49 HTTP/1.1" 200 OK
uploader failed
Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/opentelemetry/util/genai/_upload/completion_hook.py", line 142, in done
    future.result()
    ~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.13/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.13/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.13/concurrent/futures/thread.py", line 59, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.13/site-packages/opentelemetry/util/genai/_upload/completion_hook.py", line 240, in _do_upload
    with self._fs.open(path, "w", content_type=content_type) as file:
         ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/fsspec/spec.py", line 1337, in open
    self.open(
    ~~~~~~~~~^
        path,
        ^^^^^
    ...<4 lines>...
        **kwargs,
        ^^^^^^^^^
    ),
    ^
  File "/usr/local/lib/python3.13/site-packages/fsspec/spec.py", line 1349, in open
    f = self._open(
        path,
    ...<4 lines>...
        **kwargs,
    )
  File "/usr/local/lib/python3.13/site-packages/gcsfs/core.py", line 1256, in _open
    return GCSFile(
        self,
    ...<8 lines>...
        **kwargs,
    )
  File "/usr/local/lib/python3.13/site-packages/gcsfs/core.py", line 1406, in __init__
    det = getattr(self, "details", {})  # only exists in read mode
  File "/usr/local/lib/python3.13/site-packages/fsspec/spec.py", line 1936, in details
    self._details = self.fs.info(self.path)
                    ~~~~~~~~~~~~^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/fsspec/asyn.py", line 118, in wrapper
    return sync(self.loop, func, *args, **kwargs)
  File "/usr/local/lib/python3.13/site-packages/fsspec/asyn.py", line 103, in sync
    raise return_result
  File "/usr/local/lib/python3.13/site-packages/fsspec/asyn.py", line 56, in _runner
    result[0] = await coro
                ^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/gcsfs/core.py", line 834, in _info
    out = await self._ls(path, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/gcsfs/core.py", line 870, in _ls
    out = await self._list_objects(path, prefix=prefix)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/gcsfs/core.py", line 658, in _list_objects
    return [await self._get_object(path)]
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/gcsfs/core.py", line 612, in _get_object
    res = await self._call("GET", "b/{}/o/{}", bucket, key, json_out=True)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/gcsfs/core.py", line 539, in _call
    self.validate_response(status, contents, path, headers)
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/gcsfs/core.py", line 1311, in validate_response
    raise FileNotFoundError
FileNotFoundError
uploader failed
Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/opentelemetry/util/genai/_upload/completion_hook.py", line 142, in done
    future.result()
    ~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.13/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.13/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.13/concurrent/futures/thread.py", line 59, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.13/site-packages/opentelemetry/util/genai/_upload/completion_hook.py", line 240, in _do_upload
    with self._fs.open(path, "w", content_type=content_type) as file:
         ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/fsspec/spec.py", line 1337, in open
    self.open(
    ~~~~~~~~~^
        path,
        ^^^^^
    ...<4 lines>...
        **kwargs,
        ^^^^^^^^^
    ),
    ^
  File "/usr/local/lib/python3.13/site-packages/fsspec/spec.py", line 1349, in open
    f = self._open(
        path,
    ...<4 lines>...
        **kwargs,
    )
  File "/usr/local/lib/python3.13/site-packages/gcsfs/core.py", line 1256, in _open
    return GCSFile(
        self,
    ...<8 lines>...
        **kwargs,
    )
  File "/usr/local/lib/python3.13/site-packages/gcsfs/core.py", line 1406, in __init__
    det = getattr(self, "details", {})  # only exists in read mode
  File "/usr/local/lib/python3.13/site-packages/fsspec/spec.py", line 1936, in details
    self._details = self.fs.info(self.path)
                    ~~~~~~~~~~~~^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/fsspec/asyn.py", line 118, in wrapper
    return sync(self.loop, func, *args, **kwargs)
  File "/usr/local/lib/python3.13/site-packages/fsspec/asyn.py", line 103, in sync
    raise return_result
  File "/usr/local/lib/python3.13/site-packages/fsspec/asyn.py", line 56, in _runner
    result[0] = await coro
                ^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/gcsfs/core.py", line 834, in _info
    out = await self._ls(path, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/gcsfs/core.py", line 870, in _ls
    out = await self._list_objects(path, prefix=prefix)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/gcsfs/core.py", line 658, in _list_objects
    return [await self._get_object(path)]
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/gcsfs/core.py", line 612, in _get_object
    res = await self._call("GET", "b/{}/o/{}", bucket, key, json_out=True)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/gcsfs/core.py", line 539, in _call
    self.validate_response(status, contents, path, headers)
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/gcsfs/core.py", line 1311, in validate_response
    raise FileNotFoundError
FileNotFoundError
uploader failed
Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/opentelemetry/util/genai/_upload/completion_hook.py", line 142, in done
    future.result()
    ~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.13/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.13/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.13/concurrent/futures/thread.py", line 59, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.13/site-packages/opentelemetry/util/genai/_upload/completion_hook.py", line 240, in _do_upload
    with self._fs.open(path, "w", content_type=content_type) as file:
         ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/fsspec/spec.py", line 1337, in open
    self.open(
    ~~~~~~~~~^
        path,
        ^^^^^
    ...<4 lines>...
        **kwargs,
        ^^^^^^^^^
    ),
    ^
  File "/usr/local/lib/python3.13/site-packages/fsspec/spec.py", line 1349, in open
    f = self._open(
        path,
    ...<4 lines>...
        **kwargs,
    )
  File "/usr/local/lib/python3.13/site-packages/gcsfs/core.py", line 1256, in _open
    return GCSFile(
        self,
    ...<8 lines>...
        **kwargs,
    )
  File "/usr/local/lib/python3.13/site-packages/gcsfs/core.py", line 1406, in __init__
    det = getattr(self, "details", {})  # only exists in read mode
  File "/usr/local/lib/python3.13/site-packages/fsspec/spec.py", line 1936, in details
    self._details = self.fs.info(self.path)
                    ~~~~~~~~~~~~^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/fsspec/asyn.py", line 118, in wrapper
    return sync(self.loop, func, *args, **kwargs)
  File "/usr/local/lib/python3.13/site-packages/fsspec/asyn.py", line 103, in sync
    raise return_result
  File "/usr/local/lib/python3.13/site-packages/fsspec/asyn.py", line 56, in _runner
    result[0] = await coro
                ^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/gcsfs/core.py", line 834, in _info
    out = await self._ls(path, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/gcsfs/core.py", line 870, in _ls
    out = await self._list_objects(path, prefix=prefix)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/gcsfs/core.py", line 658, in _list_objects
    return [await self._get_object(path)]
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/gcsfs/core.py", line 612, in _get_object
    res = await self._call("GET", "b/{}/o/{}", bucket, key, json_out=True)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/gcsfs/core.py", line 539, in _call
    self.validate_response(status, contents, path, headers)
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/gcsfs/core.py", line 1311, in validate_response
    raise FileNotFoundError
FileNotFoundError

Steps to Reproduce

Run adk with:

opentelemetry-instrumentation-google-genai==0.4b0
google-adk==1.16.0
fastapi==0.115.13
uvicorn==0.34.0
pydantic==2.11.7
fsspec[gcs]

Expected Result

No error

Actual Result

See error logs above

Additional context

No response

Would you like to implement a fix?

None

Tip

React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.

Activity

  1. JeffLuoo commented on Feb 3, 2026

    @JeffLuoo
    Author
  2. DylanRussell commented on Feb 9, 2026

    @DylanRussell
    Contributor

    Hmm I was not able to reproduce this.. I tried running a vanilla ADK agent with all the instrumentation env vars and dependencies set as per the docs but with fsspec and gcsfs to 2026.2.0 and the upload worked.

  3. aabmass commented on May 5, 2026

    @aabmass
    Member

    Good to close folks?

  4. JeffLuoo commented on May 5, 2026

    @JeffLuoo
    Author

    We can close it for now, I will re-open the ticket if I run into the issue again.

  5. DylanRussell commented on May 5, 2026

    @DylanRussell
    Contributor

    SG... I did make a change to verify the upload works at startup, and if it doesn't to fail, instead of repeatedly failing each upload: #4390 -- also it could be the env var wasn't prefixed with gs:// - that is required to upload to GCS.. I don't know if that could have caused this stacktrace or not..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions