==SIMPLEG-WBM coupling ==
=== Current status ===
  * Now you can run a series of Singularity containers automatically by submitting a job script from Terminal. 
  * The script runs four Singularity containers sequentially and uses previous stage's outputs
  * Step 4 ( SIMPLEG_to_WBM container) has some issues. We are looking into the issue now.

[[Image(coupling_flow.jpg, desc="Submit workflows and inputs/outputs. Red highlighted files are not generated correctly as of 10/19/2021. ")]]
Most recent status document is available at [https://docs.google.com/presentation/d/1xqOf9TrTKhif3ieuS-bc3gqlqx52jwFFL-zrzQdIv8g/edit?usp=sharing  WBM-SIMPLEG coupling status]

== Set up an environment to run coupled models ==
Singularity containers and their inputs are available at ''/depot/ssg/geoshare/data/GLASS/INFEWS/SIMPLEG_WBM_COUPLING'' directory.  [[BR]]
Please make sure you have READ privilege to that directory. Otherwise the script will fail. [[BR]]
The following script will  copy all necessary files to Brown's scratch space, namely ''/scratch/brown/$USER/SIMPLEG_WBM_COUPLING''. [[BR]]
Here ''$USER'' stands for your account. 

{{{
wooj@brown-fe02:~$ /depot/ssg/geoshare/data/GLASS/INFEWS/setup_wbm_simpleg_env.sh
}}} 

Please copy/paste this command to the terminal and execute it.[[BR]]
This may take long as it copies about 40GB zip files  and uncompress them. [[BR]]
Once the set up is done, you will see these directories under ''/scratch/brown/$USER/''[[BR]]

[[Image(SIMPLEG_WBM_COUPLING_directory.jpg, desc="Subdirectories under SIMPLEG_WBM_COUPLING folder")]]

== Submit a job ==
Log and error files are created at the directory where the submit executable is called. [[BR]]
Please look into "slurm-jobid.out" file for investigating errors. [[BR]]

{{{
wooj@brown-fe02:~$ sbatch /scratch/brown/wooj/SIMPLEG_WBM_COUPLING/submit/simpleg-wbm.sub
Submitted batch job 12837875
}}}

=== Checking status of a job ===

[[Image(jobstatus_check.jpg, desc="Job status is shown as 'Running' or 'R' at the ST (Status) column  ")]]


== Submit file ==

===  The sequence submit file executes containers ===
One iteration consist of four steps.  The script finishes after a single iteration. [[BR]]
Multiple iterations can be done by modifying ''for loop'' in the script later. [[BR]]
For example, to run three iterations, script can be updated to [[BR]]
{{{
    for i in 1 2 3
}}}


 # WBM Singularity container 
   * WBM container's output is written under wbm_output directory, not job-specific directory.
 # WBM to SIMPLEG Singularity container 
  * It reads daily files and aggregates them into yearly resolution. 
  * Start year and end year are currently hard-coded to 2000-2002. These will be configurable soon.
 # SIMPLEG Singularity container 
  * Each iteration's output is written to job specific directory such as Iteration_1_S3 or Iteration_2_S3 ... 
 # SIMPLEG to WBM container 
  * It is having some issues creating three netcdf files (s1_rice_irr_fr_av.nc and so on)
  * Its output is directly written to WBM's input directory 

[[Image(submitfile.jpg, , desc="A submit file executing four containers sequentially. If you want to run a specific step only, comment out the others with '#' at the beginning of the line "))]]

== Editing submit file ==
  * If you want to execute a certain Singularity only, comment out the other steps. 
  * DO NOT forget to replace the $SLURM_ID with the actual job id you want to refer to.
  * Please contact Jungha Woo (wooj@purdue.edu) for any questions.