One issue that comes up very often are system load issues related to matlab.
Matlab will happily use extra cpus or cpu-cores on the system if/when available.
I would suggest that you use the -singleCompThread option when batching matlab jobs - as otherwise, matlab will try running its process in parallel using all the system’s CPU cores … and if you have another matlab process running it will try to do the same (ditto for 3rd and so on) and you can quickly exceed the system’s resources, while these processes will likely be swapped out (of memory) more often than necessary and will translate into longer processing times.
You can also limit or control matlab’s parallel nature with the maxNumCompThreads or parpool functions:
maxNumCompThreads(<num_threads>);
or
parpool(<num_threads>);
when doing so, you must also give SGE (Sun Grid Engine) some consideration regarding multi-threading in order for SGE to properly manage it’s resources … and this is accomplished with parallel environments. Most SGE queues at the BIC have some parallel environments defined and one way to find out would be with the following commands, e.g. :
$ qconf -sq all.q | grep pe_list
pe_list make all.pe
$ qconf -sp all.pe
pe_name all.pe
slots 48
user_lists NONE
xuser_lists NONE
start_proc_args /bin/true
stop_proc_args /bin/true
allocation_rule $pe_slots
control_slaves FALSE
job_is_first_task TRUE
urgency_slots min
accounting_summary FALSE
without getting into too much detail, please contact bicadmin if in doubt.
to give you an example, supposing that you used ‘maxNumCompThreads(4)′ within your matlab script, you would then submit this to all.q with:
qsub -q all.q -pe all.pe 4 ...
For more information, start matlab and type the following:
help maxNumCompThreads
or
help parpool