This section discusses the use of qremote to support distributed processing or running tools limited to specific hosts.
mimk's ability to invoke multiple processes in parallel, combined with qremote, qdsrv, qhost, and cush facilities support distributed or remote processing.
All one has to do is to specify the list of hosts that may be used to execute specific tools.
To specify that a tool is be distributed or run on specific hosts, one sets a qvrs associative array variable called _Servers_ with the index being the name of the tool, to the list of hosts which may be used to run the tool.
set _Servers_[cc] host1 host2 host3indicates that cc commands are to be distributed to host1, host2, and host3.
So if the recipe was:
_T_cc -c ... echo.c ...
mimk will actually execute:
qremote -DRScc cc -c ... echo.c ...
The target hosts must be able to access the directory on the
current host in which qef is being run.
They must have an installed Q-Tree and qhost
must be running on it.
The cost of running qremote, rsh, cush and the overhead of the messages to the qdsrv and qhost servers is not trivial. The savings in running a process on another host will have to offset these overheads, thus its use should be limited to expensive processes.
|Where to set _Servers_
The _Servers_[service] can be set in any of the
qvrs files, include the traits file for the host.
It can be modified as required in the more local qvrs file
(e.g., qeffile), but should probably be limited to platform
specific files as there are no checks (as yet) to ensure that the
target platform is compatible with the current host.
||Least Recently Used Host
qremote needs to pick a host from the list named by
the _Servers_ list.
It first sends messages to the target host's qhost server
which also returns the host's Q-Tree directory and the
user's home directory.
This determines which hosts are available.
qremote then sends the list to qdsrv which maintains
an internal list of hosts and when they were last selected.
qdsrv then returns the name of the least recently selected
It might be preferable to select a host based on its current load,
however, that is difficult to determine on some systems.
Furthermore the return on investment is minimal as it is likely
that one is cycling through the list of server hosts and the
least-recently-selected scheme does that effectively.
This qvrs variable specifies the maximum number of parallel
processes to be run by mimk.
If using the distributed processing facility, this variable
could be increased to take advantage of the available servers.
For example, one could set MAXMKPROCS to three times the number of
servers available as in:
set MAXMKPROCS @(expr 3 * @_Servers_[cc]~l)
|cook30.qh - 1.4 - 03/11/06|