Home

DESCRIPTION

xjobs reads job descriptions line by line and executes them in parallel. It
limits the number of parallel executing jobs and starts new jobs when
jobs finish. Therefore, it combines the arguments from every input line
with the utility and arguments given on the command line. If
no utility is given as an argument to xjobs, then the first
argument on every job line will be used as utility. To execute
\fIutility\fR xjobs searches the directories given in the PATH
environment variable and uses the first file found in these directories.

xjobs is most useful on multiprocessor machines when one needs to
execute several time consuming commands that could possibly be run in
parallel. With xjobs this can be achieved easily, and it is possible to
limit the load of the machine to a useful value. It works similar to
xargs, but starts several processes simultaneously and gives only
one line of arguments to each utility call.

By using I/O redirectors the standard input, output, and error stream of
executed jobs can be redirected. Use < to redirect standard input, > to
redirect standard output, >! to redirect standard output and overwrite
an existing file, >> to append standard output to an existing file, >&
to redirect both standard output and standard error output to the same
file, and >>& to append both standard output and standard error output
to the same file.

If passed on the command line, these operators specify the default I/O
redirection that can be overwritten by specifying another redirector to
a specific job on its argument line. After all these operators a
filename is expected. See EXAMPLES below for an example. If you need
more advanced shell features than the redirection operators supported by
xjobs, then use as utilility a shell of your preference.

Every job line can be preceeded by a "cd directory;" command that
tells xjobs in which directory the job shall be executed. For every line
this can only be used once. For more complex scripting, please pass the
line to execute to a shell of your choice.

xjobs constructs the arguments of the jobs to execute from each input
line. Each input line will create a seperate job, whereas newline
character are handled as regular whitespace by xargs. To be able to
include whitespace charakters in arguments, either preceed them with a
backslash or quote them with single or doublequote charakters. A
backslash charakter preceeding a newline will make xjobs ignore the
newline character, thus giving you the ability to pass arguments for a
single job across multiple lines. To include quotation marks in quoted
arguments, preceed them with a backslash. Lines passed to xjobs
beginning with a # charakter are interpreted as comments.

Finally, xjobs also includes a mechanism for serializing the execution.
Like this it is possible to parallelize independent jobs and sequence
jobs that have a dependency. This can be achieved by inserting a line
that only consists of two percentage charakters in sequence (%%). All
jobs before this sequence point are executed at the requested number
of jobs in parallel. When hitting the sequence point xjobs waits for all
processes to finish and then continues starting jobs that follow the
sequence point.

When passing a named pipe (i.e. a file name created by mkfifo) via
option -s as an input, xjobs will close and reopen the fifo when
reaching end-of-file. Like this it is possible to setup an xjobs server
and sending jobs to this server from muliple programs. See section
EXAMPLES below for an example.

OPTIONS

-j <jobs>
Sets the maximum number of jobs that are started in parallel. The
default value is to limit the number executing jobs is equal to the
number of online processors in the system. If the number passed as
<jobs> is followed by an 'x' charakter (e.g. 2.5x), the value is
multiplied with the number of online processors before setting the job
limit. I.e. having a machine with 4 online processors and passing 2.5x
as an argument to option -j will yield a joblimit of 10 jobs.

-s <script>
Use file script instead of the standard input to read the job
descriptions.

-n
Redirect standard output and standard error output of executed jobs to
/dev/null.

-l <num>
Combine the arguments of <\fInum\fR> input lines for a single job.

-L <log>
Set log file of xjobs to <log>.

-p
Start jobs interactively, prompting the user.

-q <num>
Limits the number of queued jobs to num elements. Normally xjobs reads
in jobs from standard input or the give script and queues them if they
cannot be started at once. With this option, xjobs will stop reading as
soon as num jobs are queued and restart reading when a new job has been
started. Like this xjobs allocates less memory. Use this option, if you
pass huge number of jobs to xjobs, to limit memory consumption. It can
also increase performance of xjobs, but be sure that jobs get fed fast
enough to xjobs.

-1
Pass one argument per job, which is expected to be terminated by a
new-line character. No argument parsing is performed. That way it is
more easy to process jobs where arguments may include whitespace
character or other tokens that influence argument parsing.

-0
Same as -1, but as a job and argument termination character a
null-character (\0) is expected instead of a new-line character. That
way also arguments with new-line character can be processed without
escape sequences.

-V
Print the version number of xjobs and exit.

-v <level>
Set verbosity of xjobs to level. Valid leves are: 0=silent, 1=error,
2=warning, 3=info, 4=debug. The default level of verbosity is 3.

-c <color>
Set color mode to <color>. Valid modes are none, auto, ansi,
pipe. none disables color mode, auto uses the TERM
environment variable to determine the terminal mode, and ansi
forces ANSI escape sequences for setting color attributes. pipe
enables color mode on terminals in auto mode and on pipes in ANSI mode.

EXAMPLES

If you have a lot of .zip files that you want to extract, then use xjobs
like this:

$ ls -1 *.zip | xjobs unzip
If you want to do the same without getting the output of each unzip task
on your terminal, then try this:

$ ls -1 *.zip | xjobs -n unzip

To gzip all *.bak files in a given directory hierarchy, use it the
following way:

$ find . -name '*.bak' | xjobs gzip

To generate index files for a set of *.jar files, you can use the
redirection feature of xjobs, and do the following:

$ ls -1 .jar | sed 's/\(.\)/\1 > \1.idx/' | xjobs jar tf

If you also want to capture the error output, than use >& instead
of >.

You can also use it to execute several different commands. Therefore,
write a script file that contains every job you want to execute and pass
it to xjobs with the option -s:

$ xjobs -s script

To be able to queue up jobs from multiple sources with xjobs, use a
named pipe and pass it explicitly as input script. Then write the jobs
to the named pipe:

$ mkfifo /var/run/my_named_pipe

$ xjobs -s /var/run/my_named_pipe &

$ echo unzip 1.zip >> /var/run/my_named_pipe

The wiki uses Markdown syntax.

Project Members:

• Thomas Maier-Komor (admin)