PyFilesystem is a Python module I started some time in 2008, and since then it has been very much a part of my personal standard library. I’ve used it in personal and professional projects, as have many other developers and organisations.
If you aren’t familiar with PyFilesystem; it’s an abstraction layer for filesystems. Essentially anything with files and directories (hard-drive, zip file, ftp server, network filesystems etc.) may be wrapped with a common interface. With it, you can write code that is agnostic as to where the files are physically located.
Here’s a quick example that recursively counts the lines of code in a directory:
def count_python_loc(fs): """Count non-blank lines of Python code.""" count = 0 for path in fs.walk.files(filter=['*.py']): with fs.open(path) as python_file: count += sum(1 for line in python_file if line.strip()) return count from fs import open_fs projects_fs = open_fs('~/projects') print(count_python_loc(projects_fs))
fs argument to
count_python_loc is an FS object, which encapsulates everything you would need to do with a filesystem. Because of this abstraction, the same code will work with any filesystem. For instance, counting the lines of code in a zip file is a single line change:
projects_fs = open_fs('zip://projects.zip')