Skip to content

url_to_fs with DirFileSystem #2011

@pjw91

Description

@pjw91

Is the url dir:///usr/lib valid for url_to_fs?

It raises the following exception currently:

Traceback (most recent call last):
  File "/dev/shm/newfs/a.py", line 4, in <module>
    d=fsspec.url_to_fs('dir:///ext/stag')
  File "/dev/shm/newfs/.venv/lib/python3.14/site-packages/fsspec/core.py", line 432, in url_to_fs
    fs = filesystem(protocol, **inkwargs)
  File "/dev/shm/newfs/.venv/lib/python3.14/site-packages/fsspec/registry.py", line 325, in filesystem
    return cls(**storage_options)
  File "/dev/shm/newfs/.venv/lib/python3.14/site-packages/fsspec/spec.py", line 84, in __call__
    obj = super().__call__(*args, **kwargs, **strip_tokenize_options)
  File "/dev/shm/newfs/.venv/lib/python3.14/site-packages/fsspec/implementations/dirfs.py", line 48, in __init__
    self.path = fs._strip_protocol(path)
                ~~~~~~~~~~~~~~~~~~^^^^^^
  File "/dev/shm/newfs/.venv/lib/python3.14/site-packages/fsspec/implementations/local.py", line 256, in _strip_protocol
    if path.startswith("file://"):
       ^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'startswith'

It seems that in url_to_fs() :

    # chain = [('/usr/lib', 'dir', {})]
    for i, ch in enumerate(reversed(chain)):
        urls, protocol, kw = ch
        if i == len(chain) - 1:
            inkwargs = dict(**kw, **inkwargs)
            inkwargs['fo'] = urls    # BUGGY: <--- lack this line
            continue
        inkwargs["target_options"] = dict(**kw, **inkwargs)
        inkwargs["target_protocol"] = protocol
        inkwargs["fo"] = urls
    urlpath, protocol, _ = chain[0]
    fs = filesystem(protocol, **inkwargs)
    return fs, urlpath

When chain = [('/usr/lib', 'dir', {})], neither fo or path is assigned in inkwargs, thus the instantiate fs = filesystem(protocol, **inkwargs) becomes fs = filesystem('dir', **{}) and lacks path for DirFileSystem

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions