Shared utilities

The atomistic-cookbook-utils package provides small helpers used across many recipes, so that recipe code can stay focused on the science. Add it to your recipe’s environment.yml under pip: (e.g. atomistic-cookbook-utils >=0.1,<0.2) and import the helpers you need.

The package source lives at src/atomistic-cookbook-utils/ in this repository.

On CI, the ATOMISTIC_COOKBOOK_CACHE environment variable makes download_with_retry() go through a download cache shared by all recipes, so that a file fetched by one recipe can be reused by the others. When the variable is not set (e.g. when running a recipe locally), files are downloaded directly to their destination.

API reference

atomistic_cookbook_utils.download_with_retry(url: str, destination: str | Path, *, retries: int = 5, backoff_factor: float = 1.0, status_forcelist: Tuple[int, ...] = (429, 500, 502, 503, 504), overwrite: bool = False, chunk_size: int = 65536) → Path[source]

Download url to destination with automatic retries.

The destination’s parent directories are created as needed. If the file already exists and overwrite is false (the default), the download is skipped and the existing path is returned.

Retries follow urllib3’s Retry with an exponential backoff. With the defaults (retries=5, backoff_factor=1) the waits between attempts are roughly 1, 2, 4, 8, 16 seconds. If the connection breaks while the file is being received, the download restarts from the beginning, with the same number of attempts and waits.

If the ATOMISTIC_COOKBOOK_CACHE environment variable points to a directory, downloads go through a cache in that directory, keyed by URL: a file that was already downloaded (e.g. by another recipe) is copied from the cache instead of being fetched again. This is meant for CI, and is off when the variable is unset. Any problem with the cache directory falls back to a direct download, and overwrite=True always bypasses the cache.

Parameters:

  • url – URL to fetch.

  • destination – Where to save the file.

  • retries – Maximum number of retry attempts on the status codes listed in status_forcelist (and on connect/read errors), and of restarts of a download interrupted while the file is being received.

  • backoff_factor – Exponential backoff factor passed to urllib3 Retry.

  • status_forcelist – HTTP status codes that trigger a retry.

  • overwrite – If true, re-download even if the destination already exists.

  • chunk_size – Streaming chunk size in bytes.

Returns:

The resolved destination path.

Return type:

pathlib.Path

atomistic_cookbook_utils.run_command(command: str, *, cwd: str | Path | None = None, check: bool = True, env: Mapping[str, str] | None = None, print_output: bool = False) → CompletedProcess[source]

Run a shell-like command string.

The string is split with shlex.split() and passed as a list of arguments to subprocess.run(). Shell features such as pipes, redirection, and glob expansion are not supported — for those, use subprocess directly.

By default the child process inherits the parent’s stdout and stderr, so output appears wherever the parent’s output goes. Pass print_output=True to capture the command’s combined stdout and stderr and re-emit it via print(), which makes the output show up in sphinx-gallery rendered cells (sphinx-gallery captures sys.stdout but not raw file-descriptor writes from children).

Parameters:

  • command – The command to run, e.g. "ls -la".

  • cwd – Working directory in which to run the command.

  • check – If true (the default), raise subprocess.CalledProcessError on non-zero exit status. When print_output is true, any captured output is printed before the exception is raised.

  • env – Optional environment mapping. If omitted, the current process environment is inherited.

  • print_output – If true, capture the child’s combined stdout and stderr and re-emit it via print() so it is visible to sys.stdout-based capture systems like sphinx-gallery. The captured text is also accessible as .stdout on the returned CompletedProcess.

Returns:

The completed process. When print_output is true, result.stdout holds the captured combined output as text; otherwise it is unset.

Return type:

subprocess.CompletedProcess