npipe v1.8.4 README file:
Manual & Command line options:
npipe.exe --help
This readme:
npipe.exe --readme
npipe v1.8.4
Usage: npipe [OPTIONS] [INPUT] [OUTPUTS] [FILE FLAGS]
File flags affect only the next input/output specified on the command line
-h, -?, --help
Display this help
--more-help
More options and further help
-i input-file, --input-file input-file
Read INPUT from file: input-file
-t, --tee
Write OUTPUT to STDOUT
-s, --stderrtee
Write OUTPUT to STDERR
output-file, --of output-file, --output-file output-file
Write OUTPUT to file: output-file
-a append-file, --append-file append-file
Append OUTPUT to file: append-file
-c conversion-function, --convert conversion-function
--tc function parameters, --text-convert function parameters
Use --convert-help for more information on these switches
INPUT defaults to STDIN
More help text: --more-help
Further options and command line details
--readme
Display readme file, (contains all help texts)
--file-flag-help
Display file flag help
--convert-help
Display help for the -c/--convert and --tc/--text-convert switches
--socket-help
Display socket help
--variables-help
Displays help for variable replacements and Scheme scripting
--adv-help
Display advanced options help
--deprecated-help
Display deprecated options help
--return-codes
Display list of return codes
-v, --version
Display version info
--credits
Display program credits
-, --stdin
Read INPUT from STDIN
--ig file-globs [--end], --input-seq-glob file-globs [--end]
Sequentially read INPUT from the contents of the files matching one or
more of the DOS/glob-style file-globs following it up until a matching
--end or the end of the command line.
If used with --roll, after reading the last file, reading resumes from
the first.
--tg file-globs --end commands [--end]
--input-thread-glob file-globs --end commands [--end]
Read INPUT from each of the files matching one or more DOS/glob-style
file-globs in a separate thread. Each thread will execute the following
commands up until a matching --end or the end of the command line.
--og file-globs [--end], --output-glob file-globs [--end]
--ag file-globs [--end], --append-glob file-globs [--end]
Sequentially write or append OUTPUT to the EXISTING files matching one
or more of the DOS/glob-style file-globs following it up until a
matching --end or the end of the command line.
--ofir[=options] search replace
--output-file-input-regex[=options] search replace
Write OUTPUT to the file whose name is given by the result of the PCRE
regular expression replacement of the input file name.
The input must have been declared previously on the command line.
This is most useful with switches such as --tg/--input-thread-glob.
See --convert-help and --variables-help for more details on
regular expressions and replacements.
Options, comma/whitespace seperated:
g[lobal] Global (permits multiple replacements)
d[up] Duplicate named subpatterns allowed
a[lways] Write even if no substitutions made (no match)
--rp count commands [--end], --repeat count commands [--end]
Evaluates commands consecutively, count number of times.
--rr commands [--end], --round-robin commands [--end]
Executes one output or group amongst commands in a round-robin fashion.
--group commands [--end]
Groups a set of commands together within a round-robin rotation.
File flags which affect every input/output file may be placed before these
switches. In the case of --input-thread-glob, other commands, switches, etc
should be in the thread commands. See --thread under --adv-help.
Files are read/written in the order of their corresponding file-glob.
-@ command-file, @command-file
Read command line parameters from command-file
-V, -VV, -VVV, -VVVV
Increasing verbosity, by default only parameter error info
1: Any non-critical warnings, or minor run-time errors
2: Sparse informational output included.
3: Periodic debugging/progress and detailed information output included
4+: Largely for detailed debugging purposes.
-P, --progress
Displays a progress bar on the active console screen buffer (Windows)
or /dev/tty (Unix)
-R, --report
Print a status report to the error handle (default STDERR, set with -e)
This is overridden by the progress-bar flag (-P).
-e error-file, --error-file error-file
Write errors to file: error-file
-O[=queue length], --overlapped[=queue length]
Reads from the input handle (not file mapping) using asynchronous
overlapped IO.
This will probably improve the overall read/process speed.
This is incompatible with the block-read flag (-B), and file mapping
inputs (-F). This is only implemented on Windows.
'Queue length' defaults to 16, and must be between 1 and 256.
-b buffer-length, --buffer-length buffer-length
Set the IO buffer size (default is 64k, unless console handle read IO
is taking place, in which case it is 256)
-m max-bytes, --maximum max-bytes
Read a maximum of max-bytes from INPUT
-j max-reads, --max-reads max-reads
Read from INPUT a maximum of max-reads times
-X, --always-read
Read input even if no output specified.
-B, --block-read
Read an entire buffer before writing.
Likely only affects pipes or other communication handles,
or when -I/--no-read is used with --roll.
-I, --no-read
Do not read from an input, but instead read once (unless --roll
is used) the initial contents of buffer 'A'.
--roll
Upon reaching the end of the input file, resume from the beginning.
When using with -I/--no-read, read the initial contents of
buffer A repeatedly, up to limits given by -j/--max-reads, and
-m/--maximum, if any. -B/--block-read should be used when using
this method as a pattern filler, to avoid poor performance.
This switch is ignored for other input types (including file maps).
--eof-char value
Treat the character given by "value" as an End Of File marker, and
stop reading as soon as it is encountered. This is often useful with
consoles, in particular with --console-lineless under --adv-help.
This switch cannot be used with overlapped inputs.
"value" can be one of:
#c The literal character c
^c The control character of c: ((c&0xDF)-0x40)&0x7F
num The character given by the integer num
-w, --wait
Wait for all pipes to be connected to before starting transfer
--zero
Read INPUT as all zeros
--random
Read INPUT as cryptographically random bytes
On Unix this reads from /dev/urandom
-f text, --input-text text
--fcs charset text, --input-text-cs charset text
-x data, --input-hex data
Use ANSI/locale, "charset" 'text' or hexadecimal
'data' as the input.
Equivalent to:
--no-read --pre-fill-buffer[-cs/-hex] A text/data
-l/--l8/-L, --write-console-cp/--write-console-utf8/--write-console-unicode
--wccs charset, --write-console-cs charset
Write OUTPUT to the active console screen buffer (Windows) or /dev/tty
(Unix), transcoding as necessary from the OEM/locale charset, UTF-8,
UTF-16LE, or a character set 'charset' respectively.
The //auto and //autoh flags can be used with 'charset'.
-o/--o8/-u, --read-console-cp/--read-console-utf8/--read-console-unicode
--rccs charset, --read-console-cs charset
Read INPUT from the active console screen buffer (Windows) or /dev/tty
(Unix), transcoding to the OEM/locale charset, UTF-8, UTF-16LE,
or a character set 'charset' respectively.
The above switches correctly handle Unicode on all platforms, where supported.
On Windows, simply using --tee, --stderrtee or --stdin when connected to a
console will write/read in the OEM character set.
-J command, --temp command
Write OUTPUT to new temporary file and then execute command on it,
when finished. Executes: "command" "file-name"
-# label-name, --label label-name
Define branch label at this point in the output chain.
--jump label-name
Unconditionally jump to label-name.
Label matching is case-sensitive, and only a-z, A-Z, 0-9 and _ are permitted.
Care should be taken to avoid infinite loops.
Command line string arguments may be delimited by " or '
Quote-string prefixes:
In the special case that the delimited string is preceded by a $ character,
the \ character acts as an escape character.
Recognised escape sequences include:
\a Beep 0007
\b Backspace 0008
\t Tab 0009
\n Line feed 000A
\v Vertical tab 000B
\f Formfeed 000C
\r Carriage return 000D
\xHEX Where HEX is 1-4 hexadecimal characters, Unicode codepoint
\XHEX Where HEX is 1-6 hexadecimal characters, Unicode codepoint
Any other escaped character produces itself (particularly useful for
including delimiting characters in a string).
This is not activated by default as it breaks the windows convention for
backslash-delimited filenames. Otherwise, no instances of the delimiting
character may be present in the string.
In the case where the delimited string is preceded by a ! character,
variable replacements are performed upon parsing of the associated command.
This is useful for dynamically generating the argument to a switch, or a
file name.
In the case case where the delimited string is preceded by a # character,
variable replacements are performed upon tokenisation of the command line.
(This is only useful in a narrow range of circumstances).
$ and # or ! may be used together. # and ! may not be used together.
Note that the $ prefix, the Scheme string parser, the PCRE library and Unix
shells (see below) all implement (slightly differing) backslash-escaping, and
hence it may be necessary to escape literal backslashes multiple times, or use
an alternative character/sequence.
NOTE: Unix shells implement their own command-line tokenisation. The $, # and
! quoted string prefixes will *NOT* work on Unix command lines, a command file
must be used. These and other "special" characters (including backslashes)
are likely to be interpreted by the shell. In all instances they should be
used with caution.
It should also be noted that Unix shells strip quotes from parameters before
passing them to executed processes. This is problematic in the following case:
npipe "-t"
On Windows this writes to a file named -t, on Unix this writes to STDOUT.
To mitigate this issue, consider using the --of/--output-file switch when
specifying a file name which might begin with a - character.
Unix shell wildcards should also be used with caution. In the instance that
there are two files: foo1 and foo2, the following will not work as expected:
npipe -i foo*
This expands to: npipe -i foo1 foo2, (not necessarily in that order), which
reads from foo1 and overwrites foo2.
To use filename wildcards, use one of the *-glob switches:
--ig/--input-seq-glob, --tg/--input-thread-glob, --og/--output-glob or
--ag/--append-glob, as these work as expected on all platforms.
Numbers may be prefixed with 0x for hexadecimal, and postfixed with the
following SI prefixes: k,M,G,T,P,E. These stand for increasing powers of 2^10
(1024) and can be used to specify kilobytes, megabytes, etc.;
Or to be more pedantic and correct: kibibytes, mebibytes, etc.
One letter arguments may be clumped, however only the last argument in the
clump may have an argument. - (--stdin) may not be clumped.
eg. -V -V -t -s -b64k <--> -VVtsb64k
Options with a separate argument can be given as one of:
--maximum arg
--maximum=arg
-m arg
-m=arg (not clumpable)
-marg
Some options require more than one argument (some conversions for example).
These subsequent arguments must be separated by whitespace.
--convert rr search-string replace-string
-crr search-string replace-string
On Windows:
Volume/drive names should be in the format: \\.\x:
Hard disk names should be in the format: \\.\PHYSICALDRIVEx
or NTAPI names could be used (-D/--device-input, -d/--device-output)
File flag help text: --file-flag-help
File flags are options which affect the next input/output specified on the
command line. Entirely superfluous or unrelated file flags are ignored, with an
error message at a verbosity of one where applicable.
Socket-specific file flags are detailed under --socket-help
General file flags:
--rate-limit num
For positive values of num, limits the IO speed to num bytes/s,
For negative or zero values, sets the minimum delay between IO
operations in ms.
Rate-limiting is by nature approximate.
--trickle num
Read/write at most num bytes at a time.
Useful in conjunction with --rate-limit, or for some devices.
Cannot be used with any of: -Q/--overlapped-output,
-F/--file-mapping, --sparse.
When applied to an input it is synonymous with -b/--buffer-length.
-Q, --overlapped-output
Writes using asynchronous overlapped IO.
This is only implemented on Windows.
The current buffer is cleared before advancing to the next output.
This will probably improve the overall write/process speed
This is incompatible with file mapping and sparse output.
Use -O/--overlapped for inputs.
-p, --persistent
Continue writing to next handle, even if write errors occur.
An unwritten gap will be left if appropriate.
Read unmappable/unreadable input blocks as zeros.
-S skip-bytes, --skip skip-bytes
Skip the first skip-bytes bytes of the file, (not pipe or socket)
-W, --file-share-write
Windows: Open with write-shared access. The file may be opened by
others for read or write access.
By default files are opened with read-share access, such that others
may read from but not write to the file whilst it is opened.
Unix: Do not flock the file at all. The default is a shared lock.
--exclusive
Windows: Open with exclusive access. The file may not be opened by
others for reading or writing whilst it is in use.
Unix: flock using an exclusive lock.
-r, --read-write
Open with read and write access
-F, --file-mapping
Windows: open a named file mapping object if it exists. Otherwise open
the file using file mapping IO on Windows or mmap on Unix.
For best performance: ensure that the buffer size and the skip offset
are a multiple of the page size, usually 4k.
--no-trunc
Do not truncate output before writing.
--open-existing
Read/write from existing file only, implies --no-trunc.
Required for certain device/com/serial/console handles.
--create-new
Write to new file only, fails if the file already exists.
Not useful when applied to inputs.
--no-buffer
Perform file IO without OS intermediary buffering.
This might produce a small performance boost in some situations.
Requirements: the buffer size must be a multiple of the sector size.
-B/--block-read should not be used when reading with this flag.
For writing, each operation must also be a multiple of the sector size,
INCLUDING THE LAST WRITE! Read/write memory buffers are aligned to 64k
(hence this is the maximum supported sector size).
The standard sector size is 512, but may vary across systems/volumes.
If in doubt, do not use.
--write-through
Data written to disk is flushed immediately, rather than lazily flushed
This may increase or decrease performance.
--sparse
Write fully zero buffers as sparse, where possible.
Buffers should be at least 64k and preferably aligned.
This cannot be used with: -Q/--overlapped-output.
Using this switch with --no-trunc may incur a small performance penalty
--mod "mode", --creation-mode-bits "mode"
Created files will have their mode bits (permissions) set to "mode",
an octal value. The process umask applies in the usual way.
This only has an effect on Unix. Default: 666.
--chmod "mode", --change-mode-bits "mode"
This is the same as --mod, except that the umask does not apply, and
all files are affected, not just created ones.
--flush-write/--flush-end
Flush output buffers after each write, or at termination, respectively.
Preamble and postamble file flags:
These switches set extra data to be read or written from/to the next
input/output on the command line, before or after all other data respectively.
--preamble text, --postamble text
--preamble-cs charset text, --postamble-cs charset text
--preamble-hex data, --postamble-hex data
--preamble-file filename, --postamble-file filename
Sets the preamble or postamble data to: text transcoded to ANSI/locale
or a given character set, data transcoded from hexadecimal, or data
stored in a file, respectively.
In the event of a write error, it may not be possible to write any
postamble data.
Note that input pre/postamble data is equivalent to any other data read
from the input, whereas output pre/postamble data does not affect any
other input/output.
Flags affecting the next file on the command line (pipe/fifo only)
--pipe
Strong hint that the file is a named pipe (Windows) or fifo (Unix).
This switch will adjust filenames to/from Windows format as required.
Windows pipe format is: \\.\pipe\[pipename]
On unix the above would be adjusted to: /tmp/[pipename]
On windows, unix filenames are prefixed with \\.\pipe\ and any / or \
characters converted to: _
This switch automatically creates the pipe/fifo (using mkfifo) where
appropriate.
-z, --force-pipe-connect
Equivalent to: --pipe --open-existing
-Z, --force-pipe-create
Equivalent to: --pipe --create-new
If --wait is set, synchronous/non-overlapped pipes which have --persistent set
wait for another client to connect upon being disconnected.
Windows: This may necessitate switching from client and server mode, which
requires that --open-existing/--force-pipe-connect is not set.
Flags which affect the next file on the command line (com/serial only)
-M "mode-string", --mode "mode-string"
Set mode of serial device to "mode-string". This is the same
format as the mode command and the BuildCommDCB winapi function.
This is not implemented on Unix. Try setserial.
Flags affecting the next file on the command line (communication devices only)
-T "timeout-string", --timeout "timeout-string"
Set timeout states of communications devices to a numeric value.
Format: val1=num [,] val2=num ...
Values: rito,rttm,rttc,wttm,wttc
These are effectively the fields of the COMMTIMEOUTS structure
All values are in milliseconds.
rito=max delay between arrival of input characters
(starting from first character read in a single read operation).
rttm=max delay per byte to read
rttc=max delay per read operation
wttm=max delay per byte to write
wttc=max delay per write operation
For a read/write operation, the timeout value is *ttm * number of bytes
to read/write + *ttc
Zero means no timeout, all unset values are left as default
To see (default) timeouts, run (with -T "") at a verbosity of 2 (-VV)
On Unix, only rttc is used. Try setserial.
Flags affecting the next file on the command line (console output only)
--control-code
Parse CSI control codes in the output when writing. Codes are started
using ESC[ or character 0x9B. When writing to a non-console device, or
when encountering a valid but unsupported escape sequence, the sequence
is removed and ignored. Supported escape sequence functions are: A-G,
f/H, J, K, S, T, s, u, m (sub-functions: 0, 1, 7, 8, 22, 28, 30-37,
39, 40-47, 49, 90-97, and 100-107), and n (sub-function: 6).
Private sequences are unsupported.
This only has an effect on Windows. Unix terminals do their own parsing.
Conversion help text: --convert-help
-c conversion-function, --convert conversion-function
Converts the buffer using one of the following functions:
All output commands after this point will be processed after the
conversion, those before will be unaffected.
x Copy (useful with -k/--convert-to-buffer, see --adv-help)
k Concatenate to existing buffer contents (-k must be used)
X Exchange current buffer with that specified by -k
Exchanging or concatenating a buffer with itself has no effect
d[+] Delete contents of buffer [and begin next read cycle]
l ANSI/locale To Lower Case
L UTF-16LE To Lower Case
u ANSI/locale To Upper Case
U UTF-16LE To Upper Case
c Clean non-ASCII to space
c:l Clean non-ASCII to letter l
cnum Clean non-ASCII to ASCII char num
c- Delete non-ASCII
s[num] Swap order of every num bytes, default 2, where num is a power
of two
hb[+-#] Binary to Hex by byte
hw[+-#] Binary to Hex by word
hd[+-#] Binary to Hex by dword
+: Adds text dump
-: Removes address header
#: Disables coalescing, and zeroes each buffer start address
hr[+] Binary to Hex by byte raw dump [with spaces]
hu Hex by byte raw dump to Binary
[n/N]?[#] Newline operations, n=ASCII/UTF-8/etc, N=UTF-16LE
- Delete newlines
u Convert newlines to Unix (LF,0xA)
d Convert newlines to DOS (Windows) (CRLF,0xD,0xA)
m Convert newlines to MAC (used up to MAC OS 9) (CR,0xD)
Nn[#] Convert UTF-16LE newlines to NEL (NEL,0x0085)
Note that newline conversion functions split the buffer into line
chunks and hence in effect are automatically preceded by ns/Ns
[n/N]s[#] Split input stream into lines
#: Disables coalescing of line fragments.
Useful for datagram/discrete type inputs.
[r/R]?[*][+l][-l][#bs] PCRE Regular Expression, r=ANSI/locale, R=UTF-8
Requires PCRE dll of same architecture.
64 bit: pcre64.dll or pcre3.dll or pcre.dll
32 bit: pcre3.dll or pcre.dll
s... [PATTERN] Search
r... [PATTERN] [REPLACEMENT] Search and replace
Where [*] is zero or more of the following
g Global (permits multiple search/replacements)
s, [+l], [-l], and [#bs] cannot be used
d Duplicate named subpatterns allowed
This might not work ideally with named [#bs]
c Line style: CR
l Line style: LF
C Line style: CRLF
a Line style: any (this is the default)
Note that c, l, C and a are mutually exclusive
Where [#bs] (repeatable) stores subpattern 's' in buffer 'b'
s=0 is whole match, s can be a name or integer
Where [+l]/[-l] are labels to branch to on match/non-match.
f? [STRING] Format buffer according to the given string
?=a/u/w for ANSI/locale, UTF8, UTF-16LE, respectively
Variable expansions are performed. See --variables-help.
[y/Y][source][dest] Substitution, y=single-byte charset, Y=UCS-2
Source and dest can be one of the following
+ String specified on the command line
a-z Pre-filled buffer
Note that the buffer must have been pre-loaded
by a command input read earlier
- Interlaced (source (first) field only)
Indicates that the second field contains
source-destination character pairs
This conversion processes 'Unicode' text as UCS-2.
Bd Base64 Decode
Be[#] Base64 Encode [Do not coalesce buffers]
z[+]? # Cyclically operate on existing buffer contents without
changing their size. The read buffer is in effect rolled over
the buffer being operated on.
+ Resume reading from the end position of the last
conversion. This is useful when the read buffer is
fixed and the buffer to operate on is not an integral
multiple of the read buffer size.
? is one of the following
c Copy, useful for pattern-filling.
--roll with --no-read may be more suitable.
x Xor
# is the buffer to read from.
S? size Buffer size operations
p Pad buffers with zeros to a minimum of size
t Truncate buffers to a maximum of size
Ss min max Split/merge buffers to be within the size range min-max
For conversion functions: a, A, b, O, w, W, see --deprecated-help
--tc function parameters, --text-convert function parameters
Operates on text using one of the following functions:
All output commands after this point will be processed after the
conversion, those before will be unaffected.
trans input-charset output-charset
Transcodes text from one character set to another.
toupper charset
tolower charset
Converts text in the given character set to upper or lower case
set charset string
Set buffer to "string" in the given character set.
Variable expansions are performed. See --variables-help.
setnv charset string
Same as set, but without variable expansions.
regex charset type[,options] [parameters]
PCRE Regular Expression operation in the given character set.
Windows: Requires PCRE dll of same architecture.
64 bit: pcre64.dll or pcre3.dll or pcre.dll
32 bit: pcre3.dll or pcre.dll
Variable expansions are performed. See --variables-help.
type: parameter:
s[earch] PATTERN Search
r[eplace] PATTERN REPLACEMENT Search and replace
Options, comma/whitespace seperated:
g[lobal] Global (permits multiple replacements)
d[up] Duplicate named subpatterns allowed
line=* Line style:
CR CR (0D)
LF LF (0A)
CRLF CRLF (0D0A)
any Any line style (default)
#b=s Stores subpattern 's' in buffer 'b', repeatable
s=0 is whole match, s can be a name or integer
m[atch]=L Branch to label 'L' on match
n[onmatch]=L Branch to label 'L' on non-match
--tr input-charset output-charset, --trans input-charset output-charset
Synonymous with: --text-convert trans input-charset output-charset
All character set parameters must be identifiers recognised by iconv, or a windows
code page number. On Unix/Windows, only windows code pages/iconv identifiers known
to npipe at compile time will be accepted, this list is taken from the win-iconv
codebase, see credits. Otherwise see "iconv -l" or MSDN for valid identifiers.
The current ANSI/locale charset can be specified by using "char".
NB: BY DEFAULT ICONV WRITES UCS-2/UCS-4/UTF-16/UTF-32 AS BIG ENDIAN.
BY DEFAULT A BOM WILL NOT BE ADDED.
npipe will automatically append the //translit flag, unless //ignore is used.
The //ignore flag is currently unsupported by win-iconv.
//auto can be used on the input charset, to automatically detect the charset
being used, based on the BOM present (if any).
//autoh is an extension of //auto which also uses heuristic methods and is more
likely to be successful. This uses libcharsetdetect.
Note that in order for the heuristic methods to be successful, the first buffer
processed must contain a suitably significant sample of representative text.
The charset detected by //auto or //autoh is shown at a verbosity of 2.
//bom, //bomall and //nobom can be used on the output charset, to force a BOM,
a BOM on every converted buffer, or no BOM, respectively.
Supported BOMs: UTF-8 and the forms of UTF-16, UTF-32.
//lastauto can be used on the input and/or output charsets, to use the charset
detected in the previous instance of the //auto or //autoh option in the
current thread, if any.
Regular expressions:
On Windows this requires a PCRE dll of the same architecture as npipe (32/64 bit).
64 bit builds search, in order: pcre64.dll, pcre3.dll, pcre.dll
32 bit builds search, in order: pcre3.dll, pcre.dll
On Unix pcre is a necessary dependency for compilation.
The syntax for regular expressions is described in detail on the PCRE website:
http://www.pcre.org/
Replacements for (sub)patterns are in the syntax: $num, ${num}, or ${name}.
The entire match is subpattern 0. To specify a literal '$' use '$$'.
Variable are also expanded, see --variables-help.
Socket help text: --socket-help
-g, --socket-read
Read INPUT from local socket
--socket-read-interface socket-addr
Read INPUT from socket: socket-addr
-G socket-addr, --socket-write socket-addr
Write OUTPUT to socket: socket-addr
--gm commands [--end], --socket-read-multi commands [--end]
--socket-read-interface-multi socket-addr commands [--end]
Read INPUT from each connection made to the socket in a separate
thread. Each thread will execute the following commands up until a
matching --end or the end of the command line. File flags for the
socket should be placed before this switch. Other commands, switches,
etc should be in the thread commands. See --thread under --adv-help.
Socket-addr can be a domain/recognisable name, dotted quad IPv4 or hex IPv6
Flags which affect the next file on the command line (network socket only)
-Y port-num, --port port-num
Port number (decimal or service name, defaults to 65042)
-U, --udp, --sock-dgram
Datagram socket mode, changes default protocol from TCP to UDP
--sock-rdm
Reliable message datagram socket mode, changes default protocol from
TCP to PGM
--sock-seqpacket
Sequential packet socket mode, changes default protocol from TCP to
unspecified (OS decides)
--sock-raw
Raw socket mode (note also IP(V6)_HDRINCL socket option must be set to
enable IP packet header modification)
One buffer/write operation per packet
Changes default protocol from TCP to RAW
-4, --ipv4
Force IPv4 address family
-6, --ipv6
Force IPv6 address family, requires OS support
--addrf address_family/number
Set socket address family to address_family name, or to address_number,
if numeric
--protocol protocol_name/number
Set socket protocol to protocol_name or to protocol_number, if numeric
The chosen protocol should be compatible with the socket type,
(set with --sock_dgram, --sock_raw, etc.).
--socket-option "option_name param1 [param2 [param3 [param4]]]"
Sets socket option or IOCTL option_name to value(s) params
See socket option/IOCTL list for list of available options, and
parameter details
If no address family is specified, the system default for the given
address (not present in the case of -g) and socket type is used.
To check the type of sockets, increase verbosity to at least 2 using -VV.
If --persistent is used, connection-orientated read sockets wait for another
client to connect upon being disconnected.
For duplex sockets see the --duplex* switches under --adv-help.
Address family listing: (case insensitive)
AF_UNSPEC 0
AF_UNIX 1
AF_INET 2
AF_IMPLINK 3
AF_PUP 4
AF_CHAOS 5
AF_IPX 6
AF_NS 6
AF_ISO 7
AF_OSI 7
AF_ECMA 8
AF_DATAKIT 9
AF_CCITT 10
AF_SNA 11
AF_DECnet 12
AF_DLI 13
AF_LAT 14
AF_HYLINK 15
AF_APPLETALK 16
AF_NETBIOS 17
AF_VOICEVIEW 18
AF_FIREFOX 19
AF_BAN 21
AF_ATM 22
AF_INET6 23
AF_CLUSTER 24
AF_12844 25
AF_IRDA 26
AF_NETDES 28
AF_TCNPROCESS 29
AF_TCNMESSAGE 30
AF_ICLFXBM 31
AF_BTM 32
Socket options:
No bounds checking of input values is done before invoking setsockopt, and
superfluous parameters are ignored.
Only the options listed below are recognised.
Bool indicates that a non-zero integer sets the option and zero unsets it.
IPv4 and IPv6 refer to recognisable/resolvable IP addresses of the given type.
Socket options and IOCTLs should be used with caution.
Listing: (case insensitive)
Socket options:
SOL_SOCKET:
SO_BROADCAST Bool
SO_CONDITIONAL_ACCEPT Bool
SO_DEBUG Bool
SO_DONTLINGER Bool (Windows only)
SO_DONTROUTE Bool
SO_GROUP_PRIORITY Integer (Windows only)
SO_KEEPALIVE Bool
SO_LINGER Integer Integer
SO_OOBINLINE Bool
SO_RCVBUF Integer
SO_REUSEADDR Bool
SO_EXCLUSIVEADDRUSE Bool (Windows only)
SO_SNDBUF Integer
SO_PORT_SCALABILITY Bool
SO_RCVTIMEO Integer
SO_SNDTIMEO Integer
IPPROTO_TCP:
TCP_NODELAY Bool
TCP_BSDURGENT Bool (Windows only)
TCP_EXPEDITED_1122 Bool (Windows only)
NSPROTO_IPX:
IPX_PTYPE Integer
IPX_FILTERPTYPE Integer
IPX_STOPFILTERPTYPE Integer
IPX_DSTYPE Integer
IPX_EXTENDED_ADDRESS Bool
IPX_RECVHDR Bool
IPX_RECEIVE_BROADCAST Bool
IPX_IMMEDIATESPXACK Bool
IPPROTO_IP:
IP_ADD_MEMBERSHIP IPv4 IPv4
IP_ADD_SOURCE_MEMBERSHIP IPv4 IPv4 IPv4
IP_BLOCK_SOURCE IPv4 IPv4 IPv4
IP_DONTFRAGMENT Bool (Windows and BSD only)
IP_DROP_MEMBERSHIP IPv4 IPv4
IP_DROP_SOURCE_MEMBERSHIP IPv4 IPv4 IPv4
IP_HDRINCL Bool
IP_MULTICAST_IF Bool
IP_MULTICAST_LOOP Bool
IP_MULTICAST_TTL Integer
IP_OPTIONS String
IP_ORIGINAL_ARRIVAL_IF Bool
IP_PKTINFO Integer
IP_RECEIVE_BROADCAST Integer
IP_TOS Integer
IP_TTL Integer
IP_UNBLOCK_SOURCE IPv4 IPv4 IPv4
IP_UNICAST_IF IPv4
IPPROTO_IPV6:
IP_ORIGINAL_ARRIVAL_IF_V6 Bool
IPV6_ADD_MEMBERSHIP IPv6 IPv6
IPV6_DROP_MEMBERSHIP IPv6 IPv6
IPV6_HDRINCL Bool
IPV6_HOPLIMIT Bool
IPV6_JOIN_GROUP IPv6 IPv6
IPV6_LEAVE_GROUP Bool
IPV6_MULTICAST_HOPS Integer
IPV6_MULTICAST_IF Bool
IPV6_MULTICAST_LOOP Bool
IPV6_PKTINFO Bool
IPV6_PROTECTION_LEVEL Integer
IPV6_UNICAST_HOPS Integer
IPV6_UNICAST_IF Integer
IPV6_V6ONLY Bool
IPPROTO_UDP:
UDP_CHECKSUM_COVERAGE Bool
UDP_NOCHECKSUM* Bool
On Unix platforms this is mapped to SOL_SOCKET->SO_NO_CHECK
IPPROTO_RM: (Windows only)
RM_ADD_RECEIVE_IF IPv4
RM_DEL_RECEIVE_IF IPv4
RM_HIGH_SPEED_INTRANET_OPT Integer
RM_LATEJOIN Integer
RM_RATE_WINDOW_SIZE Integer Integer Integer
RM_SEND_WINDOW_ADV_RATE Integer
RM_SET_MCAST_TTL Integer
RM_SET_MESSAGE_BOUNDARY Integer
RM_SET_SEND_IF IPv4
RM_USE_FEC Integer Integer Integer Bool
Socket IOCTLs: (Windows only)
SIO_UDP_CONNRESET Bool
SIO_RCVALL Integer*
SIO_RCVALL_MCAST Integer*
SIO_RCVALL_IGMPMCAST Integer*
SIO_ENABLE_CIRCULAR_QUEUEING Bool
SIO_MULTICAST_SCOPE Integer
SIO_MULTIPOINT_LOOPBACK Bool
* 0=off, 1=on, 2=socket level only, 3=IP level
Variables help text: --variables-help
Variable replacement and Scheme scripting
The escape character used for variable replacement is the '$' symbol
Each instance of a variable referenced using a '$' is replaced whenever the
string is evaluated. This occurs on every processing loop for format
conversions and regular expression replacements, and on tokenisation of the
command line for the # quote-string prefix.
See --more-help for quote-string prefixes.
$[#BUFFER] is replaced verbatim with the given buffer contents.
$[#BUFFER:charset] is replaced with the given buffer contents, transcoded from
the given 'charset', to the matching charset of the containing text.
$(scheme-code) is replaced with the atom->string evaluation of the given
Scheme code. The delimiting brackets are included in the code
evaluation. Any instances of the delimiting character must be escaped
using backslashes, which requires the $ quote-string prefix.
To specify a literal '$' use '$$'.
Scheme code
npipe uses a modified version of TinyScheme, a lightweight Scheme interpreter.
Scheme is a variant of lisp.
The most notable modifications to TinyScheme include:
Exact numbers are signed 64 bit integers, instead of 32 bit.
All errors are reported at a verbosity of 1 through the npipe error handle.
Scheme strings are stored in UTF-8 format. String offsets and character
constants properly deal with (correctly formatted) UTF-8.
String-ports, character classifiers, ASCII character names, and external
module (dll plugin) loading have been removed.
The interpreter is otherwise a reasonably-complete R5RS type.
Instances of the Scheme interpreter and all statistic properties are local to
their thread.
A number of additional functions have been defined:
(string-map proc s)
Defined as: (list->string (map proc (string->list s)))
(npipe-get-prop string), (ngp string)
Returns a read-only npipe property, see below
(npipe-jump-to-label string), (jump string)
Jumps to a label defined using --label/-#
(npipe-start-next-read-cycle), (next-cycle)
Starts the next read cycle without writing to subsequent outputs
(npipe-quit)
Equivalent to --quit switch
(npipe-quit-thread)
Quits the current thread without explicitly quitting other threads
(npipe-next-input)
Equivalent to --next-input. This does not work as expected when called
before processing of the input(s) has begun (typically when used with
--scheme-immediate/--si).
(npipe-command string)
Process the given string as part of the command line
(string->buffer[-type] string [buffer])
(buffer[-type]->string [buffer])
Sets or retrieves the contents of the npipe buffer given by the
character: [buffer], or if not given, the current buffer.
[-type] can be one of:
*blank* Copies without transforming the text charset
-ansi Transcodes to/from ANSI/the locale charset
-utf8 Transcodes to/from UTF-8
-utf16 Transcodes to/from UTF-16LE
(string->buffer-cp charset string [buffer])
(buffer-cp->string charset [buffer])
Same as the above, except the text is transcoded to/from "charset".
See --text-convert under --convert-help for valid identifiers.
The //bom, //nobom, //auto, //lastauto flags are not valid, and all
BOMs are removed during transcoding.
(console-out string), (conout string)
Writes string to the active console screen buffer.
(console-out-with-control-code string), (conout-cc string)
Parses CSI control codes in string, and writes to the active console
screen buffer. See --control-code under --file-flag-help.
CSI codes may not be split across consecutive writes.
(pcre-search regexp string [flag-string])
Searches string using PCRE regular expression: regexp.
flag-string is optional and may be any combination of the following:
d Duplicate named subpatterns allowed
c Line style: CR
l Line style: LF
C Line style: CRLF
a Line style: any (this is the default)
If a match is found, returns a vector with elements as follows:
Substring prior to the matched section
Substring following the matched section
Vector of captured subpatterns, subpattern 0 is the whole match
List of (named-pattern . substring) pairs
Otherwise, returns #f.
(pcre-replace proc regexp string [flag-string])
(pcre-replace-once proc regexp string [flag-string])
Replaces each/a successful match of PCRE regular expression: regexp, in
string with the string returned by the passed procedure: proc.
proc is called with one parameter, the vector returned by pcre-search.
The following functions take a vector x as returned by pcre-search and return
a string. They are a convenient shorthand for using vector-ref, assv, etc.
(pcre-before-match x), (pbm x)
(pcre-after-match x), (pam x)
(pcre-subpattern x n), (ps x n)
(pcre-subpattern-name x string), (psn x string)
(message-box text [caption] [flags])
Only implemented on Windows.
Produces a message box.
Flag Values:
0: OK
1: OK and Cancel
2: Abort, Retry and Ignore
3: Yes, No and Cancel
4: Yes and No
5: Retry and Cancel
6: Cancel, Try Again and Continue
16: Stop sign icon
32: Question mark icon
48: Exclamation mark icon
64: Information icon
For details of additional flags, see MessageBox.
Return values:
0: Failed
1: OK
2: Cancel
3: Abort
4: Retry
5: Ignore
6: Yes
7: No
8: Close
10: Try Again
11: Continue
(sleep num)
Sleeps the current thread for num milliseconds
The jump, quit, next input and next cycle functions take effect after the
current conversion or scheme command-line switch has completed.
npipe property list:
verbosity The current verbosity, integer
numactiveoutputs The number of currently active outputs, integer
This includes non-outputs such as -X/--always-read
numtotaloutputs The total number of "output" objects
Includes all conversions, etc
lastwrite? Does this seem to be the last write operation, boolean
bytesread Bytes read so far, integer
bytestoread Predicted bytes left to read, integer
byteswritten Bytes written so far, integer
timestarted Start timestamp, integer
numreads Number of reads so far, integer
numwrites Number of writes so far, integer
nummaps Number of file mappings so far, integer
numreaderr Number of read errors so far, integer
numwriteerr Number of write errors so far, integer
nummaperr Number of file mapping errors so far, integer
numconverts Number of converions so far, integer
threadcount Number of npipe threads at present, integer
file-[id]-[file property]
id is one of: input
outputN, where N is an integer from 0 to numtotaloutputs-1
file property is one of:
curpos Position within the file or bytes read, integer
initfilesize Initial file size, integer
ischardev If the file is a character device, boolean
handle The file handle, if defined and valid, integer
name The display name of the file, string
sockfarend The name of the far end of the connection, or
of the last received datagram's source, string
Scheme evaluation switches:
--si "code", --scheme-immediate "code"
Runs the Scheme code "code" once as the command-line is processed.
This occurs after tokenisation, in the first pass of the command-line.
(npipe-command string) is useful with this switch.
--sp "code", --scheme-process "code"
Runs the Scheme code "code" upon every iteration, similar to a
conversion.
--sprog "code", --scheme-progress "code"
Runs the Scheme code "code" whenever a progress report would be
generated (by --progress and/or --report). It is not necessary to
also use one of the aforementioned switches. The code should refrain
from reading from/writing to the buffers and from calling jump or
next-cycle, as the effect of attempting to do so is undefined. If this
switch is used more than once, the most recent definition is used.
--si-file "file", --scheme-immediate-file "file"
--sp-file "file", --scheme-process-file "file"
--sprog-file "file", --scheme-progress-file "file"
These are equivalent to the above, but read "code" from the contents
of "file" and transcode as necessary.
Advanced option help text: --adv-help
Advanced, miscellaneous, rarely used and/or arcane options
Buffer control:
-k buffer, --convert-to-buffer buffer
Output the result of the next conversion operation to buffer: 'buffer'.
Append a plus '+', to switch to that buffer following the conversion.
-K buffer, --switch-to-buffer buffer
Subsequent operations read from buffer: 'buffer'.
-q buffer text, --pre-fill-buffer buffer text
--pfbc buffer charset data, --pre-fill-buffer-cs buffer charset data
-H buffer data, --pre-fill-buffer-hex buffer data
Pre-fill buffer: 'buffer' with 'data' transcoded to: ANSI/locale,
UTF-8, UTF-16LE, or a given "charset", or transcoded from hexadecimal.
--pfbf buffer file, --pre-fill-buffer-file buffer file
Pre-fill buffer: 'buffer' with contents of 'file'.
'file' must have a size less than 4GB, and there must be sufficient
memory to fit the full file contents. File flags cannot be used.
For non-file/more complicated inputs, use ---/--begin-new-zone.
'buffer' is defined as a letter from A-Z, A is the initial default buffer.
---, --begin-new-zone
The command line up to this point is executed in it's entirety
Following commands are then read and processed without regard for
previously specified inputs/outputs. The buffer contents and general
flags (-V, -e, etc.) are preserved.
Useful for pre-filling buffer contents with dynamic content.
-+, --begin-new-input
The command line up to this point is executed in its entirety
All information regarding the input and read method is discarded
A new, subsequent input and read method can then be specified.
The output/conversion chain info and buffer contents are preserved.
The output/conversion chain can be appended to. Useful for running
the exact same functionality on multiple inputs, consecutively.
--thread commands [--end]
Executes the commands following it up until a matching --end or
the end of the command line, in a separate thread.
Named buffers (a-z) are not shared between threads.
The thread must have its own input and output(s).
--interleave buffer commands [--end]
This is the same as thread except that the output of the created thread
is moved into the specified buffer of the main thread.
Termination of one processing chain terminates both of them.
The incoming buffer is not guaranteed to be the same size as
the current buffer. Buffers leaving the created thread are
not available at the start of the next read cycle of that thread even
if not in buffer 'a', as they are handed over to the main thread.
If necessary, make a copy.
--interleave-stream buffer commands [--end]
This is the same as --interleave, except that the output of the created
thread is split/merged such that it is the same size of the current
buffer of the main thread, and thus is guaranteed to be the same size.
In the event that the created thread cannot return a buffer that large
the interleaving ends without processing any remainders.
--branch commands [--end]
This is again similar to --thread. The output of the main thread at
this point becomes the input of the created thread.
The outgoing buffer is transferred and hence not available for
subsequent writes.
--branch-dup commands [--end]
This is the same as --branch, except that a duplicate of the buffer is
sent to the created thread, such that the original is still available
for subsequent writes.
--exec program-command
This is again similar to --thread. The main thread writes to the
STDIN handle of the executed program-command. The program's STDERR
and STDOUT handles are the same of that as the current error handle.
The current buffer in the main thread is unchanged.
--chain program-command commands [--end]
This is similar to --exec. The main thread writes to the
STDIN handle of the executed program-command. The created thread reads
from the STDOUT handle of the executed program-command. The STDERR
handle is the same of that as the current error handle.
Note that some programs buffer their output if they detect that their
STDOUT has been redirected to a pipe.
The current buffer in the main thread is unchanged.
--end
Ends the current command line block. If used without an accompanying
block open, it will be treated as the end of the command line.
--dupr commands [--end], --duplex-read commands [--end]
Similar to --thread, except that the created thread reads from the
return channel of the previous output.
This is only implemented for sockets.
--dupw, --duplex-write
Write OUTPUT to the return channel of the current thread's input.
This is only implemented for sockets.
--thread and similar switches are nestable and cascadable (though doing
so is typically unnecessary).
--branch(-dup) and --interleave(-stream) are implemented using a
queue between threads. If the queue would under or overflow, one thread
will have to wait. The default queue length is 32 buffers.
The size and behaviour of the queue may be modified by preceding the switch
with the --qp/--queue-params switch.
--qp "param string", --queue-params "param string"
Set parameters of inter-thread queues.
Format: val1=num [,] val2=num ...
Values: qlen,pfau,hylt,hyht
qlen=Queue Length: Maximum number of buffers to store in queue.
pfau=Pre-Fill Anti-Underflow: Do not start the target thread reading
until there are at least num buffers in the queue.
hylt=Hysteresis Low Threshold: Upon the writer thread waiting
as the queue is full, wait until there are num or less
buffers in the queue before resuming.
hyht=Hysteresis High Threshold: Upon the reader thread waiting
as the queue is empty, wait until there are num or more
buffers in the queue before resuming.
By default: qlen is 32 and all other parameters are unset.
--quit
Terminate process without performing any further IO.
--next-input
Proceed to next input without continuing current input IO
The above two switches are useful in conjunction with conditional branching
Mailslots and NTAPI functions only exist on Windows platforms.
These switches will fail on non-Windows platforms.
If in doubt, you do not need to use mailslots.
-D ntapi-file, --device-input ntapi-file
Read INPUT from native api/kernel file: ntapi-file
-d ntapi-file, --device-output ntapi-file
Write OUTPUT to native api/kernel file: ntapi-file
--mailslot mailslot-name
Create and read INPUT from mailslot: mailslot-name
Mailslot names must be in the format: \\.\mailslot\[[path\]name]
To write to a mailslot, write to it as if it were a normal file
(but not asynchronously). When writing replace '.' by a domain
for remote sending or a '*' for all domains (noting that remote sending
imposes a datagram-length limit of 424 bytes as opposed to the local 64k).
-n handle-num, --handle handle-num
Write OUTPUT to handle/file descriptor: handle-num
-N handle-num, --input-handle handle-num
Read INPUT from handle/file descriptor: handle-num
--console-write-byte-max bytes
Change the maximum number of bytes to output at once to a console/tty.
Default: 8k. If in doubt do not change this value.
Too large a value can cause failure.
-C, --new-console
Create a new console. This only has an effect on Windows.
--console-lineless
Console input is read immediately, rather than at the end of a line,
and is not echoed as it is entered.
The --eof-char switch under --more-help may be useful.
--console-linemode
Console input is read line by line, this is usually the default.
--detach-console
Detach the process from its current console/tty.
--read-memory
Reads INPUT directly from memory starting at an offset given by --skip.
Note that this switch is *dangerous*, if in doubt, do not use it.
If used with --persistent, unreadable sections are filled with zeroes.
It is not guaranteed that attempting to read an unreadable area of
memory will not trigger a general protection fault/segfault (crash).
Deprecated option help text: --deprecated-help
Deprecated switches:
These should not be used, but are still supported for backwards compatibility.
Superfluous buffer pre-fill switches:
--q8 buffer text, --pre-fill-buffer-utf8 buffer text
-A buffer text, --pre-fill-buffer-unicode buffer text
Equivalent to: --pfbc/--pre-fill-buffer-cs buffer charset text
Where 'charset' is UTF-8 or UTF-16LE respectively.
--f8 text, --input-text-utf8 text
-E text, --input-text-unicode text
Equivalent to: --fcs/--input-text-cs charset text
Where 'charset' is UTF-8 or UTF-16LE respectively.
File flags:
--open-reparse-point
If the target file is a reparse point, open it instead of following it.
--open-backup-semantics (below) may also be required.
--open-backup-semantics
Open using FILE_FLAG_BACKUP_SEMANTICS, this is required for directories.
Neither of the above two switches work usefully (or at all on Unix).
Deprecated conversion functions, for usage information try: --convert-help.
O ANSI to OEM
A OEM to ANSI
w[+?] Charset codepage number to UTF-16LE
W[+?] Charset codepage number to UTF-16LE, with BOM
a[+?] UTF-16LE to charset codepage number
b[+?] UTF-16LE to charset codepage number, with BOM (where applicable)
[+?] [+Codepage specifier]
[+A] ANSI (default)
[+M] MAC
[+O] OEM
[+U7] UTF-7
[+U8] UTF-8
[+num] Codepage: num
Precede codepage specifier with C and/or G, to force
composite characters or glyph characters respectively.
C/G are implemeted on Windows only.
Also valid codepage specifiers, C/G have no effect:
[+U16] UTF-16LE (NB: this is mostly a null-operation)
[+U16BE]UTF-16BE
[+U32] UTF-32LE
[+U32BE]UTF-32BE
Only valid for charset to UTF-16LE (with BOM):
[+BOM[+def]] Auto select source codepage based on BOM
If no BOM, use 'def' which defaults to ANSI
Detected BOMs on the input to the conversion are discarded
On Unix platforms the ANSI, MAC and OEM codepages are defined to be the
current locale character set.
The --text-convert switch should be used instead of the above where possible.
Return codes:
0: No error occurred
1: Quit command used
2: Trailing quotation mark on command line
3: Generic error in parsing command line
4: Excessive recursion of command files
5: Input bad
6: No output
7: Sector query failed
8: Memory allocation failure
9: Catastrophic internal logic error
10: Forked child could not exec command (Unix only)
11: Ran out of available handles/file descriptors
Credits: --credits
npipe was created by Jonathan Rennison. http://npipe.sourceforge.net/
npipe uses the following libraries:
The SimpleOpt and SimpleGlob command-line processing libraries
by Brodie Thiesfield. http://code.jellycan.com/simpleopt/
The PCRE regular expression library by Philip Hazel, for regular expression
support (optional on Windows, uses a dll). http://www.pcre.org/
The libb64 library by Chris Venter, for Base64 support.
http://sourceforge.net/projects/libb64/
A modified version of the TinyScheme library, a lightweight
Scheme interpreter, by D. Souflis et al.
http://tinyscheme.sourceforge.net/home.html
The win-iconv wrapper library by Yukihiro Nakadaira, for iconv support on
Windows, and Windows codepage to iconv identifier mapping on Unix.
http://code.google.com/p/win-iconv/
A slightly modified version of libcharsetdetect, derived by Max Bolingbroke
https://github.com/batterseapower/libcharsetdetect
from the Mozilla Universal Character Set Detector.
http://mxr.mozilla.org/seamonkey/source/extensions/universalchardet/
Download Latest Version
npipe-source-v1.8.12.zip (519.8 kB)
