Skip to content

AGDC v1.0.0 – Initial Open Source Release Notes

Jump to bottom
alex-ip edited this page Sep 26, 2014 · 11 revisions

#AGDC v1.0.0 – Initial Open Source Release Notes

##GitHub Repository The GitHub repository has been renamed from “ga-datacube” to “agdc”, so the new URL is now https://github.com/GeoscienceAustralia/agdc. Note that the old “ga-datacube” URL will still continue to work with no changes.

##Major Changes

  • All code has been refactored under the Python module “agdc” which is installed using “python setup.py”. This means that the new code is now accessed at the NCI using the command “module load agdc”.
  • Import statements for shared modules have all been changed from the private ga-neo-landsat-processor repository to the open-source EO-tools repository (available at https://github.com/GeoscienceAustralia/EO_tools). At the time of writing, the AGDC requires EOtools version 0.1 or later.
  • Obsolete, un-used code has been removed (e.g. JR-STACK directory). Other code which still may be of interest has been moved to the deprecated directory.
  • New ingester code has been refactored into agdc.abstract_ingester and agdc.landsat_ingester directory modules.
  • Default configuration file has been changed from datacube.conf to agdc_default.conf
  • Example stacker subclass implementations (e.g. ndvi_stacker.py, rgb_stacker.py and index_stacker.py) have been moved to the examples directory

##Dependencies The following dependencies will need to be installed for the agdc module to work:

##Using the code at NCI ###Accessing the Latest Stable Release at NCI - Users To access the latest stable release of the new AGDC code on raijin at the NCI, you must do the following beforehand:

export MODULEPATH=/projects/el8/opt/modules/modulefiles:$MODULEPATH

module load agdc

This should load all of the required dependencies. Note that the old datacube module has been deprecated and will be removed shortly.

###Testing development code in-situ (without running setup.py) - Developers If you wish to contribute to the development of the AGDC code, you will need to clone the AGDC repository (https://github.com/GeoscienceAustralia/agdc.git), and if you wish to test the code in-situ (without running setup.py), then you can do something like the following:

cd <my_code_directory>

git clone https://github.com/GeoscienceAustralia/agdc.git

cd agdc

ln -s src agdc

export PYTHONPATH=<my_code_directory>/agdc:$PYTHONPATH

export PATH=<my_code_directory>/agdc/bin:$PATH

This will set up a copy of the repository under <my_code_directory>/agdc and then access both the bash script files and Python scripts directly from under that directory.

###Changes to user code Please note that there may be some minor changes required to import statements, as all code is now gathered under the "agdc" Python module rather than just sitting loose in a directory. For example,

from stacker import Stacker

becomes

from agdc.stacker import Stacker

##Installation Instructions Once the dependencies have been installed and the repository has been cloned from GitHub, run the following commands:

python setup.py build [sudo] python setup.py install [--prefix=<installation_root>]

For example, the command to install the code to the el8 module directory on raijin was as follows:

python setup.py install --prefix=/projects/el8/opt/modules/agdc/1.0.0

To install to the default Python library directory (i.e. /usr/local/lib/python2.7/dist_packages) on the workshop VM, the command was as follows:

sudo python setup.py install

##Known Issues

At this stage, we still need to set up a streamlined procedure to update the code on the AGDC VMs on the NCI partner cloud. At the time of writing, the AGDC code on these machines was out of date.

Clone this wiki locally