Module fsops

Collect list of files from given directory and file paths.

paths can be any sequence of strings, or a single string. Directories can be searched for files recursively or non-resursively, as requested by the recurse parameter. Parameters sort and unique determine if the resulting paths are sorted alphabetically increasing and if duplicate paths are removed. If relcwd is set to True, absolute file paths which point to files within the current working directory are made relative to it.

Only selected files may be collected by supplying a selection function through selectf parameter. It takes a file path as argument and returns a boolean, True to select the file or False to discard it.

Parameters:

• paths (string or iter(string*)) - paths to search for files
• recurse (bool) - whether to search for files recursively
• sort (bool) - whether to sort collected paths
• unique (bool) - whether to eliminate duplicate collected paths
• relcwd - whether to make collected absolute paths within current working directory relative to it
• relcwd - bool
• selectf ((string)->bool) - test to select or discard a file path

Returns: [string...]

collected file paths

Collect list of files having given extension from given paths.

The extension parameter can be a single extension or a sequence of extensions, without the leading dot. Files with empty extension (i.e. dot at the end of path) are collected by supplying empty string for extension, and files with no extension by supplying another empty sequence.

Other parameters behave in the same way as in collect_files.

Parameters:

• extension (string or sequence of strings) - extension of files to collect

Collect list of catalog file paths from given paths.

Applies collect_files_by_ext with extensions set to ("po", "pot").

Collect list of catalog file paths from directories given by an environment variable.

Other parameters behave in the same way as in collect_catalogs.

Parameters:

• catpathenv (string) - environment variable name

Make all the directories in the path which do not exist yet.

Like shell's mkdir -p.

Parameters:

• dirpath (string) - the directory path to create

Returns: string or None

the path of topmost created directory, if any

Execute command line in a specific working directory.

Like os.system, only switching CWD during execution.

Parameters:

• cmdline (string) - command line to execute
• wdir (path) - working directory for the command (CWD if none given)

Returns: int

exit code from the command

Execute command line and assert success.

If the command exits with non-zero zero state, the program aborts.

cmdline can be either a monolithic string, in which case it is executed through a shell, or a list of argument strings, when the process is started directly with these arguments.

cmdline is processed with unicode_to_str to convert any unicode strings to raw byte strings in expected system encoding.

Parameters:

• cmdline (string) - command line to execute
• echo (bool) - whether to echo the supplied command line
• wdir (path) - working directory for the command (CWD if none given)

Execute command line and collect stdout, stderr, and exit code.

cmdline can be either a monolithic string, in which case it is executed through a shell, or a list of argument strings, when the process is started directly with these arguments.

cmdline is processed with unicode_to_str to convert any unicode strings to raw byte strings in expected system encoding.

Parameters:

• cmdline (string or [string*]) - command line to execute
• echo (bool) - whether to echo the command line, as well as stdout/stderr
• wdir (path) - working directory for the command (CWD if none given)
• env ({string: string}) - environment for the execution (variable name-value pairs)
• instr (string) - string to pass to the command stdin

Returns: (string, string, int)

stdout, stderr, and exit code

Read content of a text file into list of lines.

Only CR, LF, and CR+LF are treated as line breaks.

If the given file path is not readable, or text cannot be decoded using given encoding, exceptions are raised. If encoding is not given, the encoding specified by the environment is used.

Parameters:

• filepath (string) - path of the file to read
• encoding - text encoding for the file
• encoding - string

Returns: [string...]

lines

Join path and normalize it with respect to current working directory.

Path elements are joined with os.path.join and the joined path normalized by os.path.normpath. The normalized path is then made relative to current working directory if it points to a location within current working directory.

Parameters:

• elements (varlist) - path elements

Returns: string

normalized joined path

Convert a raw string value or sequence of values into Unicode.

Strings comming in from the environment are frequently raw byte sequences, and need to be converted into Unicode strings according to system locale (e.g. command-line arguments). This function will take either a single raw string or any sequence of raw strings and convert it into a Unicode string or list thereof.

If the input value is not a single raw or unicode string, it is assumed to be a sequence of values. In case there are values in the input which are not raw strings, they will be carried over into the result as-is.

Parameters:

• strarg (string, unicode, or sequence of objects) - input string or sequence

Returns: unicode string or list of objects

unicode string or sequence of objects

Convert a unicode string into raw byte sequence.

Strings goint to the environment should frequently be raw byte sequences, and need to be converted from Unicode strings according to system locale (e.g. command-line arguments). This function will take either a single Unicode string or any sequence of Unicode strings and convert it into a raw string or list thereof.

If the input value is not a single raw or unicode string, it is assumed to be a sequence of values. In case there are values in the input which are not Unicode strings, they will be carried over into the result as-is.

Parameters:

• strarg (string, unicode, or sequence of objects) - input string or sequence

Returns: raw string or list of objects

raw string or sequence of objects

Guess user's preferred languages from the environment.

Various environment variables are examined to collect the list of languages in which the user may be wanting to read or write in in the environment. The list is ordered from most to least preferred language, and may be empty. Languages are given by their ISO-639 codes.

Returns: [string...]

preferred languages

Get number of columns in the terminal of output stream.

If the output stream is not linked to the terminal, 0 is returned. If the output stream is linked to the terminal, but the number of columns cannot be determined, the supplied default value is returned instead.

Parameters:

• stream (file) - output stream for which the terminal is looked up
• default (int) - value to return if width cannot be determined

Returns: int

width of the terminal in columns

Build a path selection function based on inclusion-exclusion condition.

Frequently a collection of paths needs to be filtered, to pass only specific paths (inclusion), or to block only specific paths (exclusion), or both. Filtering conditions are normally posed on full paths, but frequently file base names without extensions are really tested.

This function builds a selector function which takes a path and returns True to select the path or False to discard it, based on four sets of conditions: inclusions by base name without extension (incnames), inclusion by full path (incpaths), exclusions by base name without extension (excnames), and exclusions by full path (excpaths). Each condition in each of the sets can be a regular expression string, an object with search(string) method returning true or false value (e.g. compiled regular expression), or a general function taking string and returning true or false value.

If ormatch is False, the path is included if there are no inclusion conditions or all inclusion conditions match; the path is excluded if there is at least one exclusion condition and all exclusion conditions match. If ormatch is True, the path is included if there are no inclusion conditions or at least one of them matches; the path is excluded if at least one exclusion condition match.

Parameters:

• incnames (sequence (see description)) - conditions for inclusion by base name without extension
• incpaths (sequence (see description)) - conditions for inclusion by full path
• excnames (sequence (see description)) - conditions for exclusion by base name without extension
• excpaths (sequence (see description)) - conditions for exclusion by full path
• ormatch (bool) - whether conditions are linked with OR

Returns: (string)->bool

path selection function

Collect list of paths from the file.

In general, non-empty lines in the file are taken to be paths, and empty lines are skipped. If cmnts is True, then also the lines starting with '#' are skipped as comments.

The respathf parameter provides a function to be applied to each path and return a list of paths, which then substitute the original path. This function can be used, for example, to recursively collect files from listed directories, or to exclude paths by an external condition.

If incexc is True, then the lines starting with ':' define directives by which files and directories are included or excluded from the final list. Inclusion-exclusion directives are mostly useful when some of the paths are directories, and respathf parameter is used to provide a function to collect subpaths from listed directories; the inclusion-exclusion directives are applied to those subpaths too. The directives are as follows:

• :-REGEX: excludes path if its base name without extension matches the regular expression
• :/-REGEX: excludes path if it matches the regular expression
• :+REGEX: includes path only if its base name without extension matches the regular expression
• :/+REGEX: includes path only if it matches the regular expression

The path is included if there are no inclusion directives, or it matches at least one inclusion directive; the path is excluded if it matches at least one exclusion directive. Inclusion-exclusion directives are given to build_path_selector to create the path selection function (with ormatch set to True), which is then used to filter collected paths (after application of respathf, if given).

If getsel is set to True, the selection function is returned instead of being applied to read paths immediately. This is useful in case the respathf parameter is not sufficient to resolve paths, but more complex processing is required. from directories externally, instead with respathf). If there were no inclusion-exclusion directives in the file, the resulting selection function will return True for any path.

Parameters:

• fpath (string) - the path to file which contains paths
• cmnts (bool) - whether the file can contain comments
• incexc (boolean) - whether the file can contain inclusion-exclusion directives
• respathf ((string)->[string...]) - function to resolve collected paths
• getsel (bool) - whether to return constructed path selection function instead of applying it
• abort (bool) - whether to abort the execution on exceptions from path resolution or selection functions

Returns: [string...] or ([string...], (string)->bool)

collected paths, possibly with path selection function

Collect list of paths from usual sources given on command line.

Scripts that process paths will in general get paths directly (as free command line arguments or on standard input), or indirectly from files containing lists of paths (usually given by a command line option). Sometimes input directory paths will be searched for paths of all files in them, possibly of certain type. Especially when searching directory paths, the script may take options to exclude or include only paths that match something. This function conveniently wraps up these possibilities, to fetch all possible paths in single statement.

The rawpaths parameter provides a list of directly supplied paths, e.g. from command line arguments. incnames, incpaths, excnames, and excpaths are lists of inclusion and exclusion conditions out of which single path selection function is constructed, with ormatch determining how conditions are linked, see build_path_selector for details. filesfrom is a list of files containing lists of paths, cmnts and incexc are options for the file format, see collect_paths_from_file for details. If both rawpaths and filesfrom are not given or empty, elsecwd determines if current working directory is added to list of paths (True) or not (False). respathf is a function which takes a path and returns list of paths, see description of the same parameter in collect_paths_from_file.

The order of path collection is as follows. First all paths from rawpaths are added, applying respathf. Then all paths from all files given by fromfiles are added, by applying collect_paths_from_file on each file (respathf is applied by sending it to collect_paths_from_file). If both rawpaths and fromfiles were None or empty, current working directory is added, possibly applying respathf. Finally, all paths are filtered through inclusion-exclusion tests; if no inclusion tests are given, then all files are included unless excluded by an exclusion test.

If getsel is set to True, the path selection function is returned instead of being applied to collected paths. This function will also include path selection functions constructed from inclusion-exclusion directives found in filesfrom, linked with the top conditions according to ormatch.

Parameters:

• respathf ((string)->[string...]) - function to resolve collected paths
• getsel (bool) - whether to return constructed path selection function instead of applying it
• abort (bool) - whether to abort the execution on exceptions from path resolution or selection functions

Returns: [string...] or ([string...], (string)->bool)

collected paths, possibly with path selection function

Get path of current working directory as Unicode string.

os.getcwd() returns a raw byte sequence, to which the str_to_unicode function is applied to make best guess at decoding it into a unicode string.

Returns: string

path of current working directory

Gracefully exit a Pology script when an exception is received.

Any error message will be printed, any progress lines will be cleared, and keyboard interrupt will exist silently.

The backtrace can be shown instead (on non-keyboard interrupt exceptions) by setting [global]/show-backtrace user configuration field to true.

Parameters:

• func (() -> any) - a zero-argument function
• cleanup (() -> any) - a zero-argument function to execute before exiting

Returns: string

path of current working directory