upsies.utils.fs

File system helpers

Module Attributes

upsies.utils.fs.OS_SEP_REGEX = '/'

os.sep escaped with re.escape()

On Windows, this should be "\\" while it should be identical to os.sep on Unix-likes.

Functions

upsies.utils.fs.assert_dir_usable(path, *, check_writable=True)

Raise ContentError if dirpath can not be used as a directory

Parameters:

check_writable (bool) – Whether to ignore if path is not writable

upsies.utils.fs.assert_file_readable(filepath)

Raise ContentError if open(filepath) fails

upsies.utils.fs.basename(path, parents=0)

Return last segment in path

Unlike os.path.basename(), this removes any trailing directory separators first.

>>> os.path.basename('a/b/c/')
''
>>> upsies.utils.basename('a/b/c/')
'c'

Parameters:

parents – How many levels of parent directories to include in the returned path

upsies.utils.fs.clear_cache(cache_directory)

Delete cache_directory

upsies.utils.fs.dirname(path)

Remove last segment in path

Unlike os.path.dirname(), this removes any trailing directory separators first.

>>> os.path.dirname('a/b/c/')
'a/b/c'
>>> upsies.utils.dirname('a/b/c/')
'a/b'

upsies.utils.fs.ensure_path_in_cache(path, cache_directory)

Return path beneath cache_directory

If path points to anywhere inside cache_directory, return it unmodified. Otherwise, return the basename() of path appended to cache_directory.

upsies.utils.fs.file_and_parent(path)

Return path’s filename and parent directoy name if there is one

upsies.utils.fs.file_extension(path, minlen=1, maxlen=None, only=())

Return file extension

Unlike os.path.splitext(), this function expects the extension to consist only of alphanumeric ASCII characters.

Parameters:

• path (str) – Path to file or directory

• minlen (str) – Minimum number of characters in file extension or None for no limit

• maxlen (str) – Maximum number of characters in file extension or None for no limit

• only (sequence of str) – Only return extension if it exists in this sequence

Returns:

file extension

Return type:

str

upsies.utils.fs.file_list(path, *, extensions=(), min_age=None, max_age=None, follow_dirlinks=False)

List naturally sorted files in path and any subdirectories

If path is not a directory, it is returned as a single item in a list unless extensions are given and they don’t match.

If path is any falsy value (e.g. "" or None), return an empty sequence.

Unreadable directories are excluded.

Parameters:

• path (str) – Path to a directory

• extensions (str) – Only include files with one of these extensions (matched case-insensitively) or include all files if this is falsy

• min_age (int or float) – Exclude files that are younger than this

• max_age (int or float) – Exclude files that are older than this

• follow_dirlinks – Whether to include the contents of symbolic links to directories or the links themselves

Returns:

Tuple of file paths

upsies.utils.fs.file_size(path)

Return file size in bytes or None

upsies.utils.fs.filter_files(path, exclude)

Return sequence of files beneath path with some files excluded

If path is not a directory, an empty sequence is returned if it matches anything in exclude.

Parameters:

exclude –

Sequence of glob patterns (str) and re.Pattern (return value from re.compile()) or Regex objects

Files beneath path are excluded from the torrent.

Glob patterns are matched case-insensitively. For case-insensitive matching with regular expressions, use (?i:<pattern>).

Raises:

ContentError – if anything goes wrong, e.g. a directory cannot be read

upsies.utils.fs.filter_main_videos(filepaths)

Exclude irrelevant file paths from filepaths

Irrelevant file paths are non-video file paths (based on VIDEO_FILE_EXTENSIONS) and sample videos.

For season packs, try to find the main episodes by file name first. If that fails, try to find the main video(s) exclude all video files that are smaller than 75 % of the average video file size.

upsies.utils.fs.find_main_video(path)

Return main video file from path

If path is a video file, simply return it. If path is a directory, pass it to find_main_videos() and return the first item.

If no video file is found, raise ContentError.

upsies.utils.fs.find_main_videos(path, exclude=())

Return main video file paths beneath path

This is a simple convenience wrapper around filter_files() and filter_main_videos().

upsies.utils.fs.find_name(name, path, validator=None)

Return first path (in human sort order) to name anywhere beneath path or None

Parameters:

• name (str) – Exact name of the file or directory

• path (str) – Base path to search for name

• validator –

  Callable that gets a path that ends with name and returns whether it is a match or not

  For example, you can use os.path.isfile() or os.path.isdir() to only find files or directories.

upsies.utils.fs.format_file_tree(tree, _parents_is_last=())

Format nested file tree sequence as indented multi-line string

Parameters:

tree – Nested 2-tuples: The first item is the file or directory name, the second item is the file size for files or a tuple for directories

upsies.utils.fs.limit_directory_size(path, max_total_size, min_age=None, max_age=None)

Delete oldest files (by access time) until maximum size is not exceeded

Empty files and directories are always deleted.

Parameters:

• path – Path to directory

• max_total_size – Maximum combined size of all files in path and its subdirectories

• min_age (Unix timestamp) – Preserve files that are younger than this

• max_age (Unix timestamp) – Preserve files that are older than this

upsies.utils.fs.listdir(path)

Return non-recursive sequence of items in directory path

If path is not a directory or getting the directory’s items fails for any reason, return an empty sequence.

upsies.utils.fs.mkdir(path)

Create directory and its parents

Existing directories are ignored.

If path is empty, no directory is created.

Raises:

ContentError – if directory creation fails

upsies.utils.fs.path_size(path)

Combined size of all files beneath path

Returns the same value as file_size() if path is not a directory.

Symbolic links are ignored.

upsies.utils.fs.projectdir(content_path, base=None)

Return path to existing directory in which jobs put their files and cache

Parameters:

• content_path (str) – Path to torrent content

• base (str) – Location of the project directory; defaults to DEFAULT_CACHE_DIRECTORY

Raises:

ContentError – if content_path exists and is not a directory or has insufficient permissions

upsies.utils.fs.prune_cache(cache_directory, *, max_total_size)

Wrapper around limit_directory_size() and prune_empty()

Parameters:

• cache_directory – Path of cache directory to prune

• max_total_size – See limit_directory_size()

upsies.utils.fs.prune_empty(path, *, files=False, directories=True)

Remove empty subdirectories recursively

Parameters:

• path – Path to directory

• files (bool) – Whether to prune empty files

• directories (bool) – Whether to prune empty directories

Dead symbolic links are removed after pruning files and directories.

If path is not a directory, do nothing.

upsies.utils.fs.sanitize_filename(filename)

Replace illegal characters in filename with “_”

Illegal characters include os.sep.

upsies.utils.fs.sanitize_path(path)

Replace illegal characters in each path segment with “_”

path is split at os.sep and the resulting items are passed to sanitize_filename() and joined.

On Windows, the drive (e.g. “C:”) is not sanitized to keep the “:”.

upsies.utils.fs.strip_extension(path, minlen=1, maxlen=None, only=())

Return path without file extension

If path doesn’t have a file extension, return it as is.

A file extension consists only of alphanumeric ASCII characters.

Parameters:

• path (str) – Path to file or directory

• minlen (str) – Minimum number of characters in file extension or None for no limit

• maxlen (str) – Maximum number of characters in file extension or None for no limit

• only (sequence of str) – Only strip extension if it exists in this sequence

Returns:

file name without extension

Return type:

str

upsies.utils.fs.tildify_path(path)

Return path with $HOME replaced by ~