Getting Started

The galileoQC package is run from a Jupyterlab notebook. You should create a virtual working environment (called whizz in this documentation) with python venv.

Installation

To use galileoQC, first install it into your whizz environment using pip (conda installations of galileoQC can work but conda has not been tested as fully as pip):

(whizz) $ pip install galileoQC

Then start up a jupyterlab notebook. For more information about this, see [1] . The Jupyterlab-Desktop app [2] also works.

For more information on installation see Install Details.

Session Setup

In a jupyterlab notebook, set up your session with the following commands.

Firstly import the necessary Python packages …

from pathlib import Path # useful for file names
%matplotlib widget # needed to, for example, zoom into plots

… then import galileoQC …

import galileoQC as qc

… and finally set the path to your field and plan data. This might look something like this:

# Setup the path to the acquired data, ...
data_root = r'.source/tutorials/'
dx = Path(data_root + r'CanobieData/Canobie.xyz')
dh = dx.with_suffix('.hdf5')

# ..., and plan data.
plan_root = data_root
px = Path(plan_root + r'CanobieData/CanobiePlan.xyz')
ph = px.with_suffix('.hdf5')

The variables dx, dh, px and ph are often used to represent the four key data filenames. Of course, you can use any variable names for your projects but it is worth noting these because you will see them occasionally in the documentation.

If you want to see how to use galileoQC, go to Tutorials and follow along with the examples. You can also download the python tutorial notebooks and example data from github and experiment with the functions yourself.

Now you are ready to QC your data!

Installation on NCI Australia

Finally, some users might want to install galileoQC on their NCI (National Computational InfraStructure) environment. The following notes might prove useful.

WARNING - some of the following information might be out of date.

Using the instructions in this link, set up a python3 environment in your specific location. Due to memory/space limitations for each user, it is suggested that you make this custom python environment in your project directory in NCI. The environment will have a folder with the same name in your working directory after it is built:

For the QAQC python scripts, we need to install these extra python3 packages within our new python environment:

  •   Xarray (version 2022.12.0 or higher)

  •   Rioxarray

  •   netCDF4

  •   Ipympl

  •   Colorcet

  •   Filebrowser

  •   Matplotlib(version 3.6.2)

In addition to the above packages, two customised scripts called graphics.py and colors.py can be downloaded from here. These packages should be placed in site_packages available in your python environment.

Also make sure that you have all the following packages as well - if anything missing:

  •   H5py

  •   Verde

  •   Pooch

Finally, in order to run the current QAQC python scripts on NCI, the following steps need to be taken:

  •   Analyst need to have an NCI account

  •   Login to https://ood.nci.org.au/pun/sys/dashboard

  •   Choose Virtual Desktop

  •   Except project that needs to be changed to your active project the rest of settings should be accepted as default

  •   Launch the VDI app

  •   From the menu choose the Terminal

  •   Navigate to your working directory

  •   Load python3 package available in NCI

[user@ood-vn30 user]$ module load python3/3.9.2
	Loading python3/3.9.2
	 Loading requirement: intel-mkl/2020.4.304

In your working directory you will have two active subfolders:

  •   Directory that contains your code and notebooks

  •   Directory that contains your specific python environment Load your specific python3 environment for instance loading environment called QA

[user@ood-vn30 user]$ . QA/bin/activate
     (QA) [user@ood-vn30 user]$

  •   Run jupyter notebook in your VDI using the command:

Jupyter notebook

At this stage, you will be redirected to a web browser that you can load the specific notebook and quality check your airborne gravity dataset.