How to use SGE with Matlab

One issue that comes up very often are system load issues related to matlab.

Matlab will happily use extra cpus or cpu-cores on the system if/when available.

I would suggest that you use the -singleCompThread option when batching matlab jobs - as otherwise, matlab will try running its process in parallel using all the system’s CPU cores … and if you have another matlab process running it will try to do the same (ditto for 3rd and so on) and you can quickly exceed the system’s resources, while these processes will likely be swapped out (of memory) more often than necessary and will translate into longer processing times.

You can also limit or control matlab’s parallel nature with the maxNumCompThreads or parpool functions:




when doing so, you must also give SGE (Sun Grid Engine) some consideration regarding multi-threading in order for SGE to properly manage it’s resources … and this is accomplished with parallel environments. Most SGE queues at the BIC have some parallel environments defined and one way to find out would be with the following commands, e.g. :

$ qconf -sq all.q | grep pe_list
pe_list               make

$ qconf -sp
slots              48
user_lists         NONE
xuser_lists        NONE
start_proc_args    /bin/true
stop_proc_args     /bin/true
allocation_rule    $pe_slots
control_slaves     FALSE
job_is_first_task  TRUE
urgency_slots      min
accounting_summary FALSE

without getting into too much detail, please contact bicadmin if in doubt.

to give you an example, supposing that you used ‘maxNumCompThreads(4)′ within your matlab script, you would then submit this to all.q with:

qsub -q all.q -pe 4 ...

For more information, start matlab and type the following:

help maxNumCompThreads


help parpool