GNU Queue load-balancing system:
How do I set up Queue for cluster-wide use?

The GNU Queue system consists of two components, `queued' which runs as a daemon on every host in the cluster, and `queue' is a user program that allows users to submit jobs to the system.

The 'queue' binary contacts queued to learn the relative virtual load averages (explained in 'profile') on each host, and specifies one on which to run the job. Queued then forks off a process and works together with queue on the local end to control the remote job.

Look over the sample 'profile' file See section Configure a Job Queue's profile File to learn how to customize batch queues and load balancing. 'profile' has many options. Among others, you can configure certain hosts to be submit-only hosts for all or only certain job classes by turning off job execution in these queues.

Add the name of each host in the cluster to the access control list. The default location for this is either share/qhostsfile or /usr/local/share/qhostsfile depending on how ./configure was invoked.

Finally, if you are installing GNU Queue cluster-wide with root privileges (./configure --enable-root option), make sure the spool directory (default is /usr/local/com/queue) is NFS exported root-writable on all systems in your cluster. The default for many NFS installations is to map the root UID to "nobody", who is an unprivileged user; it is important that the spool directory be restricted to allow access by privileged users. Consequently, it is necessary to override the default NFS options for the spool directory to allow queue processes on NFS clients to write to this directory safely. In GNU/Linux, this is done by setting the no_root_squash option in /etc/exports (and then running /usr/etc/exportfs to cause the system to acknowlege the changes; if /usr/etc/exportfs is not available on your system, restart nfsd and the portmapper.) Other operating system flavors have different names for this option. Read nfs(4), exports(4) and other man pages for information on setting the no_root_squash equivalent on your operating system flavor.

This isn't an issue if you are installing GNU Queue as an ordinary user, because ordinary, unprivileged users generally do not have their UIDs mapped between client and server. Consequently, your queue process should have no problem writing to the NFS spool directory as long as you can.


werner.krebs@yale.edu
Previous: How do I compile and/or install Queue?
Next: What are the command-line options to "queue"?
This document is: http://bioinfo.mbb.yale.edu:80/cgi-bin/fom?file=8
[Search] [Appearance] [Show Edit Commands]
This is Faq-O-Matic 2.606.