INFEWS Logo
  • Overview
  • Collections
  • Members
  • Wiki3
  • Resources
  • Forum
  • Blog
  • Wish List
  • Projects
  • Usage
  • Announcements
  • Courses
  • Files
  • Activity
  • Discoverability Visible
  • Join Policy Invite Only
  • Created 02 Dec 2019

INFEWS

►

Wiki

Submitting a SIMPLEG-WBM coupling job to the Purdue Brown cluster

Version 17
by (unknown)
Version 18
by (unknown)

Deletions or items before changed

Additions or items after changed

1 ==SIMPLEG-WBM coupling ==
2 === Current status ===
3 * Now you can run a series of Singularity containers automatically by submitting a job script from Terminal.
4 * The script runs four Singularity containers sequentially and uses previous stage's outputs
5 * Current workflow assumes some convergence conditions. Workflow orchestration will be changed to alternative approach proposed by Iman.
6 {{{
7 There are some minor changes but Jungha should be able to create the shocks for SIMPLEG using the code:
8 Spin-up:
9 We need an initial run of WBM for 2000-2005
10 Then we calculate the shocks for SIMPLEG using WBMtoSIMPLE_v3. The new R script takes the “sim.year1” (here 2004) and “sim.length” (here 4) as inputs. After the test run, the year will be updated at each run.
11 Then we solve SIMPLEG. It will create a new land area to be sent to WBM. for the test run, I like to label it with the “2005” run.
12 Consecutive WBM-SIMPLEG runs:
13 Here we need to run WBM only for the next year (the year 2005).
14 Then we calculate the shocks for SIMPLEG (for 2005). It creates a new land area to be sent to WBM for the next year, it will be labeled with the next year (here as 2006).
15 Then we repeat (a) and (b) for all the required years.
16 }}}
17
18 [[Image(SIMPLE-G_WBM_coupling_workflow.jpg, desc="Submit workflows and inputs/outputs.")]]
19 Most recent status document is available at [https://docs.google.com/presentation/d/1xqOf9TrTKhif3ieuS-bc3gqlqx52jwFFL-zrzQdIv8g/edit?usp=sharing WBM-SIMPLEG coupling status]
20
21 == Set up an environment to run coupled models ==
22 Singularity containers and their inputs are available at ''/depot/ssg/geoshare/data/GLASS/INFEWS/SIMPLEG_WBM_COUPLING'' directory. [[BR]]
23 -
Please make sure you have READ privilege to that directory. Otherwise the script will fail. [[BR]]
+
Please make sure you have privilege to read that directory. Otherwise the following script will fail. [[BR]]
24 -
The following script will copy all necessary files to Brown's scratch space, namely ''/scratch/brown/$USER/SIMPLEG_WBM_COUPLING''. [[BR]]
+
The script will copy all necessary files to your Brown's scratch space, namely ''/scratch/brown/$USER/SIMPLEG_WBM_COUPLING''. [[BR]]
25 Here ''$USER'' stands for your account.
26
27 {{{
28 wooj@brown-fe02:~$ /depot/ssg/geoshare/data/GLASS/INFEWS/setup_wbm_simpleg_env.sh
29 }}}
30
31 -
Please copy/paste this command to the terminal and execute it.[[BR]]
+
Please copy and paste this command to the terminal and execute it.[[BR]]
32 This may take long as it copies about 40GB zip files and uncompress them. [[BR]]
33 -
Once the set up is done, you will see these directories under ''/scratch/brown/$USER/''[[BR]]
+
Once the set up is done, you will see SIMPLEG_WBM_COUPLING directory in ''/scratch/brown/$USER/''[[BR]]
34
35 [[Image(SIMPLEG_WBM_COUPLING_directory.jpg, desc="Subdirectories under SIMPLEG_WBM_COUPLING folder")]]
36
37 == Submit a job ==
38 Log and error files are created at the directory where the submit executable is called. [[BR]]
39 Please look into "slurm-jobid.out" file for investigating errors. [[BR]]
40
41 {{{
42 wooj@brown-fe02:~$ sbatch /scratch/brown/wooj/SIMPLEG_WBM_COUPLING/submit/simpleg-wbm.sub
43 Submitted batch job 12837875
44 }}}
45
46 +
You can use alternative submit file for running the workflow two times.
47 +
{{{
48 +
wooj@brown-fe02:~$ sbatch /scratch/brown/wooj/SIMPLEG_WBM_COUPLING/submit/simpleg-wbm-r2.sub
49 +
}}}
50 === Checking status of a job ===
51
52 [[Image(jobstatus_check.jpg, desc="Job status is shown as 'Running' or 'R' at the ST (Status) column ")]]
53
54
55 == Submit file ==
56
57 === The sequence submit file executes containers ===
58 One iteration consist of four steps. The script finishes after a single iteration. [[BR]]
59 Multiple iterations can be done by modifying ''for loop'' in the script later. [[BR]]
60 For example, to run three iterations, script can be updated to [[BR]]
61 {{{
62 for i in 1 2 3
63 }}}
64
65
66 # WBM Singularity container
67 * WBM container's output is written under wbm_output directory, not job-specific directory.
68 # WBM to SIMPLEG Singularity container
69 * It reads daily files and aggregates them into yearly resolution.
70 -
* Start year and end year are currently hard-coded to 2000-2002. These will be configurable soon.
+
* Default Start year and end year are set to 2000-2004. Users can supply start and end year when running container. These are specified from the submit file.
71 # SIMPLEG Singularity container
72 * Each iteration's output is written to job specific directory such as Iteration_1_S3 or Iteration_2_S3 ...
73 # SIMPLEG to WBM container
74 -
* It is having some issues creating three netcdf files (s1_rice_irr_fr_av.nc and so on)
75 * Its output is directly written to WBM's input directory
76
77 [[Image(submitfile.jpg, , desc="A submit file executing four containers sequentially. If you want to run a specific step only, comment out the others with '#' at the beginning of the line "))]]
78
79 == Editing submit file ==
80 * If you want to execute a certain Singularity only, comment out the other steps.
81 * DO NOT forget to replace the $SLURM_ID with the actual job id you want to refer to.
82 * Please contact Jungha Woo (wooj@purdue.edu) for any questions.
83
84 == Container recipes ==
85 Four Singularity recipes are available at GitHub [https://github.com/junghawoo/simpleg-wbm-coupling],
86 but they are private repositories at this moment. Please contact Jungha Woo ( wooj@purdue.edu).
87
88 == Container 101 ==
89 Slides from Yucheng Zhang@RCAC are available at [https://www.rcac.purdue.edu/files/training/containers101.pdf]
90 * Why we should use containers
91 * Build once, run almost anywhere
92 * Ensure reproducibility
93 * Why we use Singularity containers
94 * Secure!
95 * Users do not need to ask their system admin to install software for them
96 * Does not require root privilege