pology.fsops

Package pology :: Module fsops |

# Module fsops

Operations with environment, file system and external commands.

Author: Chusslove Illich (Часлав Илић) <caslav.ilic@gmx.net>

License: GPLv3

Functions
[string...] |
collect_files(paths, recurse=True, sort=True, unique=True, relcwd=True, selectf=None) Collect list of files from given directory and file paths. |
collect_files_by_ext(paths, extension, recurse=True, sort=True, unique=True, relcwd=True, selectf=None) Collect list of files having given extension from given paths. |
collect_catalogs(paths, recurse=True, sort=True, unique=True, relcwd=True, selectf=None) Collect list of catalog file paths from given paths. |
collect_catalogs_by_env(catpathenv, recurse=True, sort=True, unique=True, relcwd=True, selectf=None) Collect list of catalog file paths from directories given by an environment variable. |
string or `None` |
mkdirpath(dirpath) Make all the directories in the path which do not exist yet. |
int |
system_wd(cmdline, wdir) Execute command line in a specific working directory. |
assert_system(cmdline, echo=False, wdir=None) Execute command line and assert success. |
(string, string, int) |
collect_system(cmdline, echo=False, wdir=None, env=None, instr=None) Execute command line and collect stdout, stderr, and exit code. |
[string...] |
lines_from_file(filepath, encoding=None) Read content of a text file into list of lines. |
string |
join_ncwd(*elements) Join path and normalize it with respect to current working directory. |
unicode string or list of objects |
str_to_unicode(strarg) Convert a raw string value or sequence of values into Unicode. |
raw string or list of objects |
unicode_to_str(strarg) Convert a unicode string into raw byte sequence. |
[string...] |
get_env_langs() Guess user's preferred languages from the environment. |
int |
term_width(stream=sys.stdout, default=None) Get number of columns in the terminal of output stream. |
(string)->bool |
build_path_selector(incnames=None, incpaths=None, excnames=None, excpaths=None, ormatch=False) Build a path selection function based on inclusion-exclusion condition. |
[string...] or ([string...], (string)->bool) |
collect_paths_from_file(fpath, cmnts=True, incexc=True, respathf=None, getsel=False, abort=False) Collect list of paths from the file. |
[string...] or ([string...], (string)->bool) |
collect_paths_cmdline(rawpaths=None, incnames=None, incpaths=None, excnames=None, excpaths=None, ormatch=False, filesfrom=None, cmnts=True, incexc=True, elsecwd=False, respathf=None, getsel=False, abort=False) Collect list of paths from usual sources given on command line. |
string |
getucwd() Get path of current working directory as Unicode string. |
string |
exit_on_exception(func, cleanup=None) Gracefully exit a Pology script when an exception is received. |

Variables
| __package__ = ``'``pology``'``

Function Details

### collect_files(paths, recurse=True, sort=True, unique=True, relcwd=True, selectf=None) |
Collect list of files from given directory and file paths. `paths` can be any sequence of strings, or a single string. Directories can be searched for files recursively or non-resursively, as requested by the `recurse` parameter. Parameters `sort` and `unique` determine if the resulting paths are sorted alphabetically increasing and if duplicate paths are removed. If `relcwd` is set to `True`, absolute file paths which point to files within the current working directory are made relative to it. Only selected files may be collected by supplying a selection function through `selectf` parameter. It takes a file path as argument and returns a boolean, `True` to select the file or `False` to discard it. Parameters: `paths` (string or iter(string*)) - paths to search for files `recurse` (bool) - whether to search for files recursively `sort` (bool) - whether to sort collected paths `unique` (bool) - whether to eliminate duplicate collected paths `relcwd` - whether to make collected absolute paths within current working directory relative to it `relcwd` - bool `selectf` ((string)->bool) - test to select or discard a file path Returns: [string...] collected file paths

### collect_files_by_ext(paths, extension, recurse=True, sort=True, unique=True, relcwd=True, selectf=None) |
Collect list of files having given extension from given paths. The `extension` parameter can be a single extension or a sequence of extensions, without the leading dot. Files with empty extension (i.e. dot at the end of path) are collected by supplying empty string for `extension`, and files with no extension by supplying another empty sequence. Other parameters behave in the same way as in collect_files. Parameters: `extension` (string or sequence of strings) - extension of files to collect See Also: collect_files

### collect_catalogs(paths, recurse=True, sort=True, unique=True, relcwd=True, selectf=None) |
Collect list of catalog file paths from given paths. Applies `collect_files_by_ext` with extensions set to `("po", "pot")`.

### collect_catalogs_by_env(catpathenv, recurse=True, sort=True, unique=True, relcwd=True, selectf=None) |
Collect list of catalog file paths from directories given by an environment variable. Other parameters behave in the same way as in collect_catalogs. Parameters: `catpathenv` (string) - environment variable name

### mkdirpath(dirpath) |
Make all the directories in the path which do not exist yet. Like shell's `mkdir -p`. Parameters: `dirpath` (string) - the directory path to create Returns: string or `None` the path of topmost created directory, if any

### system_wd(cmdline, wdir) |
Execute command line in a specific working directory. Like `os.system`, only switching CWD during execution. Parameters: `cmdline` (string) - command line to execute `wdir` (path) - working directory for the command (CWD if none given) Returns: int exit code from the command

### assert_system(cmdline, echo=False, wdir=None) |
Execute command line and assert success. If the command exits with non-zero zero state, the program aborts. `cmdline` can be either a monolithic string, in which case it is executed through a shell, or a list of argument strings, when the process is started directly with these arguments. `cmdline` is processed with unicode_to_str to convert any unicode strings to raw byte strings in expected system encoding. Parameters: `cmdline` (string) - command line to execute `echo` (bool) - whether to echo the supplied command line `wdir` (path) - working directory for the command (CWD if none given)

### collect_system(cmdline, echo=False, wdir=None, env=None, instr=None) |
Execute command line and collect stdout, stderr, and exit code. `cmdline` can be either a monolithic string, in which case it is executed through a shell, or a list of argument strings, when the process is started directly with these arguments. `cmdline` is processed with unicode_to_str to convert any unicode strings to raw byte strings in expected system encoding. Parameters: `cmdline` (string or [string*]) - command line to execute `echo` (bool) - whether to echo the command line, as well as stdout/stderr `wdir` (path) - working directory for the command (CWD if none given) `env` ({string: string}) - environment for the execution (variable name-value pairs) `instr` (string) - string to pass to the command stdin Returns: (string, string, int) stdout, stderr, and exit code

### lines_from_file(filepath, encoding=None) |
Read content of a text file into list of lines. Only CR, LF, and CR+LF are treated as line breaks. If the given file path is not readable, or text cannot be decoded using given encoding, exceptions are raised. If encoding is not given, the encoding specified by the environment is used. Parameters: `filepath` (string) - path of the file to read `encoding` - text encoding for the file `encoding` - string Returns: [string...] lines

### join_ncwd(*elements) |
Join path and normalize it with respect to current working directory. Path elements are joined with `os.path.join` and the joined path normalized by `os.path.normpath`. The normalized path is then made relative to current working directory if it points to a location within current working directory. Parameters: `elements` (varlist) - path elements Returns: string normalized joined path

### str_to_unicode(strarg) |
Convert a raw string value or sequence of values into Unicode. Strings comming in from the environment are frequently raw byte sequences, and need to be converted into Unicode strings according to system locale (e.g. command-line arguments). This function will take either a single raw string or any sequence of raw strings and convert it into a Unicode string or list thereof. If the input value is not a single raw or unicode string, it is assumed to be a sequence of values. In case there are values in the input which are not raw strings, they will be carried over into the result as-is. Parameters: `strarg` (string, unicode, or sequence of objects) - input string or sequence Returns: unicode string or list of objects unicode string or sequence of objects

### unicode_to_str(strarg) |
Convert a unicode string into raw byte sequence. Strings goint to the environment should frequently be raw byte sequences, and need to be converted from Unicode strings according to system locale (e.g. command-line arguments). This function will take either a single Unicode string or any sequence of Unicode strings and convert it into a raw string or list thereof. If the input value is not a single raw or unicode string, it is assumed to be a sequence of values. In case there are values in the input which are not Unicode strings, they will be carried over into the result as-is. Parameters: `strarg` (string, unicode, or sequence of objects) - input string or sequence Returns: raw string or list of objects raw string or sequence of objects

### get_env_langs() |
Guess user's preferred languages from the environment. Various environment variables are examined to collect the list of languages in which the user may be wanting to read or write in in the environment. The list is ordered from most to least preferred language, and may be empty. Languages are given by their ISO-639 codes. Returns: [string...] preferred languages

### term_width(stream=sys.stdout, default=None) |
Get number of columns in the terminal of output stream. If the output stream is not linked to the terminal, 0 is returned. If the output stream is linked to the terminal, but the number of columns cannot be determined, the supplied default value is returned instead. Parameters: `stream` (file) - output stream for which the terminal is looked up `default` (int) - value to return if width cannot be determined Returns: int width of the terminal in columns

### build_path_selector(incnames=None, incpaths=None, excnames=None, excpaths=None, ormatch=False) |
Build a path selection function based on inclusion-exclusion condition. Frequently a collection of paths needs to be filtered, to pass only specific paths (inclusion), or to block only specific paths (exclusion), or both. Filtering conditions are normally posed on full paths, but frequently file base names without extensions are really tested. This function builds a selector function which takes a path and returns `True` to select the path or `False` to discard it, based on four sets of conditions: inclusions by base name without extension (`incnames`), inclusion by full path (`incpaths`), exclusions by base name without extension (`excnames`), and exclusions by full path (`excpaths`). Each condition in each of the sets can be a regular expression string, an object with `search(string)` method returning true or false value (e.g. compiled regular expression), or a general function taking string and returning true or false value. If `ormatch` is `False`, the path is included if there are no inclusion conditions or all inclusion conditions match; the path is excluded if there is at least one exclusion condition and all exclusion conditions match. If `ormatch` is `True`, the path is included if there are no inclusion conditions or at least one of them matches; the path is excluded if at least one exclusion condition match. Parameters: `incnames` (sequence (see description)) - conditions for inclusion by base name without extension `incpaths` (sequence (see description)) - conditions for inclusion by full path `excnames` (sequence (see description)) - conditions for exclusion by base name without extension `excpaths` (sequence (see description)) - conditions for exclusion by full path `ormatch` (bool) - whether conditions are linked with OR Returns: (string)->bool path selection function

### collect_paths_from_file(fpath, cmnts=True, incexc=True, respathf=None, getsel=False, abort=False) |
Collect list of paths from the file. In general, non-empty lines in the file are taken to be paths, and empty lines are skipped. If `cmnts` is `True`, then also the lines starting with `'#'` are skipped as comments. The `respathf` parameter provides a function to be applied to each path and return a list of paths, which then substitute the original path. This function can be used, for example, to recursively collect files from listed directories, or to exclude paths by an external condition. If `incexc` is `True`, then the lines starting with `':'` define directives by which files and directories are included or excluded from the final list. Inclusion-exclusion directives are mostly useful when some of the paths are directories, and `respathf` parameter is used to provide a function to collect subpaths from listed directories; the inclusion-exclusion directives are applied to those subpaths too. The directives are as follows: `:-REGEX`: excludes path if its base name without extension matches the regular expression `:/-REGEX`: excludes path if it matches the regular expression `:+REGEX`: includes path only if its base name without extension matches the regular expression `:/+REGEX`: includes path only if it matches the regular expression The path is included if there are no inclusion directives, or it matches at least one inclusion directive; the path is excluded if it matches at least one exclusion directive. Inclusion-exclusion directives are given to build_path_selector to create the path selection function (with `ormatch` set to `True`), which is then used to filter collected paths (after application of `respathf`, if given). If `getsel` is set to `True`, the selection function is returned instead of being applied to read paths immediately. This is useful in case the `respathf` parameter is not sufficient to resolve paths, but more complex processing is required. from directories externally, instead with `respathf`). If there were no inclusion-exclusion directives in the file, the resulting selection function will return `True` for any path. Parameters: `fpath` (string) - the path to file which contains paths `cmnts` (bool) - whether the file can contain comments `incexc` (boolean) - whether the file can contain inclusion-exclusion directives `respathf` ((string)->[string...]) - function to resolve collected paths `getsel` (bool) - whether to return constructed path selection function instead of applying it `abort` (bool) - whether to abort the execution on exceptions from path resolution or selection functions Returns: [string...] or ([string...], (string)->bool) collected paths, possibly with path selection function

### collect_paths_cmdline(rawpaths=None, incnames=None, incpaths=None, excnames=None, excpaths=None, ormatch=False, filesfrom=None, cmnts=True, incexc=True, elsecwd=False, respathf=None, getsel=False, abort=False) |
Collect list of paths from usual sources given on command line. Scripts that process paths will in general get paths directly (as free command line arguments or on standard input), or indirectly from files containing lists of paths (usually given by a command line option). Sometimes input directory paths will be searched for paths of all files in them, possibly of certain type. Especially when searching directory paths, the script may take options to exclude or include only paths that match something. This function conveniently wraps up these possibilities, to fetch all possible paths in single statement. The `rawpaths` parameter provides a list of directly supplied paths, e.g. from command line arguments. `incnames`, `incpaths`, `excnames`, and `excpaths` are lists of inclusion and exclusion conditions out of which single path selection function is constructed, with `ormatch` determining how conditions are linked, see build_path_selector for details. `filesfrom` is a list of files containing lists of paths, `cmnts` and `incexc` are options for the file format, see collect_paths_from_file for details. If both `rawpaths` and `filesfrom` are not given or empty, `elsecwd` determines if current working directory is added to list of paths (`True`) or not (`False`). `respathf` is a function which takes a path and returns list of paths, see description of the same parameter in collect_paths_from_file. The order of path collection is as follows. First all paths from `rawpaths` are added, applying `respathf`. Then all paths from all files given by `fromfiles` are added, by applying collect_paths_from_file on each file (`respathf` is applied by sending it to collect_paths_from_file). If both `rawpaths` and `fromfiles` were `None` or empty, current working directory is added, possibly applying `respathf`. Finally, all paths are filtered through inclusion-exclusion tests; if no inclusion tests are given, then all files are included unless excluded by an exclusion test. If `getsel` is set to `True`, the path selection function is returned instead of being applied to collected paths. This function will also include path selection functions constructed from inclusion-exclusion directives found in `filesfrom`, linked with the top conditions according to `ormatch`. Parameters: `respathf` ((string)->[string...]) - function to resolve collected paths `getsel` (bool) - whether to return constructed path selection function instead of applying it `abort` (bool) - whether to abort the execution on exceptions from path resolution or selection functions Returns: [string...] or ([string...], (string)->bool) collected paths, possibly with path selection function

### getucwd() |
Get path of current working directory as Unicode string. `os.getcwd()` returns a raw byte sequence, to which the str_to_unicode function is applied to make best guess at decoding it into a unicode string. Returns: string path of current working directory

### exit_on_exception(func, cleanup=None) |
Gracefully exit a Pology script when an exception is received. Any error message will be printed, any progress lines will be cleared, and keyboard interrupt will exist silently. The backtrace can be shown instead (on non-keyboard interrupt exceptions) by setting `[global]/show-backtrace` user configuration field to true. Parameters: `func` (() -> any) - a zero-argument function `cleanup` (() -> any) - a zero-argument function to execute before exiting Returns: string path of current working directory