mirar.utils package

Module for util/helper functions

Subpackages

• mirar.utils.docs package
  • Submodules
  • mirar.utils.docs.auto_config_docs module
    • auto_rst_config()
    • auto_rst_pipeline()
    • auto_top_level_rst()
    • get_rst_config_path()
    • get_rst_pipeline_path()
    • iterate_rst_generation()
  • mirar.utils.docs.pipeline_visualisation module
    • flowify()
    • get_save_path()
    • iterate_flowify()

Submodules

mirar.utils.dockerutil module

Module containing docker integration (beta-stage)

mirar.utils.dockerutil.docker_batch_put(container: Container, local_paths: str | list)

Function to copy multiple files into a Docker container

Parameters

container: A docker.models.container.Container object local_paths: Local paths of each file to copy

Returns

Returns a list of files in the docker container after the copying is done

mirar.utils.dockerutil.docker_get(container: Container, local_path: str | Path)

Function to cope one file from the Docker container ‘container’ to ‘local_path’. The file in the container should have the same name as the base file in ‘local_path’.

Parameters

container: A docker.models.container.Container object local_path: Local path of file to copy to

Returns

mirar.utils.dockerutil.docker_get_new_files(container: Container, output_dir: str | Path, ignore_files: list[str | Path])

Function to copy new files out of a container. All files in the work directory of ‘container’ will be copied out to ‘output_dir’, unless they appear in the ‘ignore_files’ list.

The normal procedure is to run this in tandem with docker_batch_put(), so that only new files are copied out of the container. For example:

ignore_files = docker_batch_put(

container=container, local_paths=list_of_files_to_copy_into_container

)

container.exec_run(some_docker_command)

docker_get_new_files(

container=container, output_dir=output_dir, ignore_files=ignore_files

)

Parameters

container: A docker.models.container.Container object output_dir: A local directory to save the output files to. ignore_files: List of files to ignore (i.e to not copy)

Returns

mirar.utils.dockerutil.docker_path(file_path: str | Path) → Path

Converts a local path to the corresponding path in the docker container

Parameters:

file_path – file path

Returns:

mirar.utils.dockerutil.docker_put(container: Container, local_path: str | Path)

Function to one file, at ‘local_path’ into the Docker container ‘container’

Parameters

container: A docker.models.container.Container object local_path: Local path of file to copy

Returns

mirar.utils.dockerutil.new_container()

Generate a new docker.models.containers.Container object, using the default docker daemon and the docker image. If the image is not found locally, the image will first be pulled from DockerHub.

This function requires a Docker daemon to first be running.

Returns

A docker container built with the {DOCKER_IMAGE_NAME} image

mirar.utils.execute_cmd module

Module for executing bash commands

exception mirar.utils.execute_cmd.ExecutionError

Bases: Exception

Error relating to executing bash command

exception mirar.utils.execute_cmd.TimeoutExecutionError

Bases: Exception

Error relating to timeout when executing bash command

mirar.utils.execute_cmd.execute(cmd: str, output_dir: Path | str = '.', local: bool = True, timeout: float = 300.0)

Generically execute a command either via bash or a docker container

Parameters:

• cmd – command

• output_dir – output directory for command

• local – boolean whether use local or docker

• timeout – timeout for local execution

Returns:

None

mirar.utils.execute_cmd.run_docker(cmd: str, output_dir: Path | str = '.')

Function to run a command via Docker. A container will be generated automatically, but a Docker server must be running first. You can start one via the Desktop application, or on the command line with `docker start’.

After the specified ‘cmd’ command has been run, any newly-generated files

will be copied out of the container to ‘output_dir’

Parameters

cmd: A string containing the base arguments you want to use to run sextractor. An example would be:

cmd = ‘image01.fits -c sex.config’

output_dir: A local directory to save the output files to.

Returns

mirar.utils.execute_cmd.run_local(cmd: str, timeout: float = 300.0)

Function to run on local machine using subprocess, with error handling.

After the specified ‘cmd’ command has been run, any newly-generated files will be copied out of the current directory to ‘output_dir’

Parameters

cmd: A string containing the command you want to use to run sextractor. An example would be:

cmd = ‘/usr/bin/source-extractor image0001.fits -c sex.config’

timeout: Time to timeout in seconds

Returns

mirar.utils.execute_cmd.temp_config(config_path: str | Path, output_dir: str | Path) → Path

Get a

Parameters:

• config_path

• output_dir

Returns:

mirar.utils.ldac_tools module

Functions to convert FITS files or astropy Tables to FITS_LDAC files and vice versa.

mirar.utils.ldac_tools.convert_hdu_to_ldac(hdu: BinTableHDU | TableHDU) → tuple[BinTableHDU, BinTableHDU]

Convert an hdu table to a fits_ldac table (format used by astromatic suite)

Parameters

hdu: astropy.io.fits.BinTableHDU or astropy.io.fits.TableHDU

HDUList to convert to fits_ldac HDUList

Returns

tbl1: astropy.io.fits.BinTableHDU

Header info for fits table (LDAC_IMHEAD)

tbl2: astropy.io.fits.BinTableHDU

Data table (LDAC_OBJECTS)

mirar.utils.ldac_tools.convert_table_to_ldac(tbl: Table) → HDUList

Convert an astropy table to a fits_ldac

Parameters

tbl: astropy.table.Table

Table to convert to ldac format

Returns

hdulist: astropy.io.fits.HDUList

FITS_LDAC hdulist that can be read by astromatic software

mirar.utils.ldac_tools.get_table_from_ldac(file_path: str | Path, frame: int = 1) → Table

Load an astropy table from a fits_ldac by frame (Since the ldac format has column info for odd tables, giving it twce as many tables as a regular fits BinTableHDU, match the frame of a table to its corresponding frame in the ldac file).

Parameters

file_path: str

Name of the file to open

frame: int

Number of the frame in a regular fits file

mirar.utils.ldac_tools.save_table_as_ldac(tbl: Table, file_path: str | Path, **kwargs)

Save a table as a fits LDAC file

Parameters

tbl: astropy.table.Table

Table to save

file_path: str

Filename to save table

kwargs:

Keyword arguments to pass to hdulist.writeto

mirar.utils.security module

Util functions for password generation

mirar.utils.security.generate_key(length: int = 20) → str

Generate an alphanumeric password of length N, from https://docs.python.org/3/library/secrets.html#recipes-and-best-practices

Parameters:

length – length

Returns:

Password

mirar.utils.send_email module

Module containing gmail integration functions

mirar.utils.send_email.send_gmail(email_recipients: str | list[str], email_subject: str, email_text: str, email_sender: str = None, email_password: str = None, attachments: str | list[str] | None = None, auto_compress: bool = True)

Function to send an email to a list of recipients from a gmail account.

Parameters:

• email_recipients – recipients for email

• email_subject – subject for the email

• email_text – Text to send

• email_sender – Gmail to send from

• email_password – Password for sender gmail account

• attachments – Any files to attach

• auto_compress – Boolean to compress large attachments before sending

Returns: