path: root/doc/bbe.info

This is bbe.info, produced by makeinfo version 4.8 from bbe.texi.

INFO-DIR-SECTION Utilities
START-INFO-DIR-ENTRY
* bbe: (bbe).                 Binary Block Editor.
END-INFO-DIR-ENTRY

   This file documents version 0.2.2 of `bbe', a binary block editor.

   Copyright (C) 2005 Timo Savinen

     Permission is granted to make and distribute verbatim copies of
     this manual provided the copyright notice and this permission
     notice are preserved on all copies.

     Permission is granted to copy and distribute modified versions of
     this manual under the conditions for verbatim copying, provided
     that the entire resulting derived work is distributed under the
     terms of a permission notice identical to this one.

     Permission is granted to copy and distribute translations of this
     manual into another language, under the above conditions for
     modified versions.


File: bbe.info,  Node: Top,  Next: Overview,  Prev: (dir),  Up: (dir)

bbe
***

This file documents version 0.2.2 of `bbe', a binary block editor.

   Copyright (C) 2005 Timo Savinen

     Permission is granted to make and distribute verbatim copies of
     this manual provided the copyright notice and this permission
     notice are preserved on all copies.

     Permission is granted to copy and distribute modified versions of
     this manual under the conditions for verbatim copying, provided
     that the entire resulting derived work is distributed under the
     terms of a permission notice identical to this one.

     Permission is granted to copy and distribute translations of this
     manual into another language, under the above conditions for
     modified versions.

* Menu:

* Overview::                    Preliminary information.
* Samples::                     Samples using `bbe'.
* Invoking bbe::                How to run `bbe'.
* bbe programs::                How `bbe' works.
* Problems::                    Reporting bugs.


File: bbe.info,  Node: Overview,  Next: Samples,  Prev: Top,  Up: Top

1 Preliminary information
*************************

The `bbe' program is a sed-like editor for binary files. `bbe' performs
basic byte related transformations on blocks of input stream. `bbe' is
non-interactive command line tool and can be used as a part of a
pipeline. `bbe' makes only one pass over input stream.

   `bbe' contains also grep-like features, like printing the filename,
offset and block number.


File: bbe.info,  Node: Samples,  Next: Invoking bbe,  Prev: Overview,  Up: Top

2 Samples using `bbe'
*********************

Few examples of running `bbe':

`bbe -b "/\x80\x50\x0e/:12" -e "d 0 3" -e "c BCD ASC" -e "A \x0a" -e "w /tmp/numbers" -o /dev/null /tmp/bdata'
     Task here is to extract BCD coded numbers from the file
     `/tmp/bdata' and write them in ascii format with newline to file
     `/tmp/numbers'.  12 bytes long blocks containing the BCD-numbers
     start with three byte sequence of values `0x80', `0x50' and
     `0x0e'. First three bytes (the block start sequence) are removed
     (`d 0 3') rest of the block is transformed from BCD to Ascii (`c
     BCD ASC') and a newline character is appended at the end of the
     block (`A \x0a').  All transformed blocks are written to
     `/tmp/numbers' (`w /tmp/numbers'). Nothing is written to the
     output (`-o /dev/null').
`bbe -b ":525" -e "i 524 \x0a" -o /tmp/data_with_nl /tmp/data'
     A newline is added after every 525'th byte of the file
     `/tmp/data'. Data with newlines is written to `/tmp/data_with_nl'.
`bbe -b ":526" -e "d 525 1" -o /tmp/data /tmp/data_with_nl'
     Every 526'th byte (inserted newline in previous example) is
     removed from the file `/tmp/data_with_nl'. Data without newlines
     is written to `/tmp/data'.
`bbe -e "s/\x0d\x0a/\x0a/"'
     Same effect as has command `dos2unix'.


File: bbe.info,  Node: Invoking bbe,  Next: bbe programs,  Prev: Samples,  Up: Top

3 How to run `bbe'
******************

`bbe' accepts several commands to operate on blocks. Commands are
executed in the same order as they appear in command line or in a
script file.  Order is significant, because the changes made to current
byte by previous commands are seen by next commands.

* Menu:

* Invocation::      Program invocation
* Block::           Block definition
* Commands::        bbe commands
* Limits::          Limitations


File: bbe.info,  Node: Invocation,  Next: Block,  Up: Invoking bbe

3.1 Program invocation
======================

The format for running the `bbe' program is:

     bbe OPTION ...

   `bbe' supports the following options:

`-b BLOCK'
`--block=BLOCK'
     Block definition.

`-e COMMAND'
`--expression=COMMAND'
     Add command(s) to the commands to be executed. Commands must be
     separated by semicolon.

`-f SCRIPT-FILE'
`--file=SCRIPT-FILE'
     Add commands from SCRIPT-FILE to the commands to be executed.

`-o FILE'
`--output=FILE'
     Write output to FILE instead of standard output.

`-s'
`--suppress'
     Suppress printing of normal output, print only block contents.

`-?'
`--help'
     Print an informative help message describing the options and then
     exit successfully.

`-V'
`--version'
     Print the version number of `bbe' and then exit successfully.

   All remaining options are names of input files, if no input files
are specified or `-' is given, then the standard input is read.


File: bbe.info,  Node: Block,  Next: Commands,  Prev: Invocation,  Up: Invoking bbe

3.2 Block definition
====================

`bbe' devides the input stream to blocks defined by the `-b' option. If
block is not defined, the whole input stream is considered as one
block. Commands have effect only inside a block, rest of the input
stream remains untouched. Currently `bbe' supports only one block
definition per invocation. If input stream consists of different blocks,
several `bbe's can be combined in a pipeline.

   A block can be defined several ways:

`N:M'
     Block starts at offset N of input stream (first byte is 0). Block
     is M bytes long. This definition allows only one block to be
     defined.

`:M'
     The whole input stream is divided to M-size blocks.

`/START/:/STOP/'
     Blocks start with sequence START and end with sequence STOP. Both
     START and STOP are included to blocks.

`/START/:'
     Blocks start with sequence START and ends at next occurrence of
     START. Only the first START is included to block.

`:/STOP/'
     Blocks start at the beginning of input stream or after the end of
     previous block. Block ends at first occurrence of STOP.  Only the
     last STOP is included to blocks.

   It is possible to use c-like byte values in N, M, START and STOP.
Values in START and STOP must be escaped with `\', `\' can be escaped
as `\\'.

   Byte values can be expressed in decimal, octal or hexadecimal e.g.
in START and STOP:

`\123, \32 or \0'
     Decimal values

`\012, \08 or \0278'
     Octal values

`\x0a, \x67 or \xff'
     Hexadecimal values

   Also escape code `\y' can be used. Decimal values of `\y''s:

`\a'
     7

`\b'
     8

`\t'
     9

`\n'
     10

`\v'
     11

`\f'
     12

`\r'
     13

`\;'
     59
     Semicolon must be escaped, because it is a command delimitter.

   Values of N and M can be given in decimal, octal and hexadecimal:

`123, 32 or 112232'
     Decimal values

`0128, 08123 or 0'
     Octal values

`x456a, x167 or xffde'
     Hexadecimal values


File: bbe.info,  Node: Commands,  Next: Limits,  Prev: Block,  Up: Invoking bbe

3.3 `bbe' commands
==================

Commands in `bbe' can be divided in two groups: Block related commands
and byte related commands. Block related commands operate at block
level e.g. remove a block.  Byte related commands work allways inside a
block and they don't have effect beyond the block boundaries.

   Same escape codes for byte values in STRINGs can be used as in START
and STOP of block definition.

Block commands are:
-------------------

`I STRING'
     Write the STRING to output stream before the block.

`D [N]'
     Delete the N'th block. If N is not defined all blocks are deleted
     from output stream.  *Note*: First block is number one.

`A STRING'
     Write the STRING to output stream after the block.

`J N'
     Commands appearing after this command have no effect until N
     blocks are found. Means "Jump first N blocks".  *Note*: Commands
     that are defined before this command have effect on every block.

`L N'
     Commands appearing after this command have no effect after N
     blocks are found. Means "Leave blocks after N'th block".  *Note*:
     Commands that are defined before this command have effect on every
     block.

`N'
     Before block contents the file name where the current block starts
     is printed with colon.

`F F'
     Before block contents the current stream offset and colon is
     printed in format specified by F.  Stream offset starts at zero. F
     can have one of following values:
    H
          Hexadecimal.

    D
          Decimal.

    O
          Octal.

`B F'
     Before block contents the current block number and colon is
     printed in format specified by F.  Block numbering starts at one.
     F can have one of the sames codes as `F'-command.

`> `file''
     Before printing a block, the contents of file `file' is printed.

`< `file''
     After printing a block, the contents of file `file' is printed.

Byte commands are:
------------------

*Note*: The N in byte commands is offset from the beginning of current
block, first byte is number zero.

`c FROM TO'
     Converts bytes from FROM to TO. Currently supported formats are:
    `ASC'
          Ascii

    `BCD'
          Binary Coded Decimal
     *Note*: Bytes, that cannot be converted are passed through as they
     are. e.g. in ASC -> BCD conversion, ASCII characters not in range
     `'0'' - `'9'' are not converted.

`d N M|*'
     Delete M bytes starting from the offset N. If * is defined instead
     of M, then all bytes of the block starting from N are deleted.

`i N STRING'
     Insert STRING after byte number N.

`j N'
     Commands appearing after `j'-command have no effect concerning
     bytes 0-N of the block.

`l N'
     Commands appearing after `l'-command have no effect concerning
     bytes starting from the byte number N of the block.

`u N C'
     All bytes from start of the block to offset N are replaced by C.

`f N C'
     All bytes starting from offset N to the end of the block are
     replaced by C.

`p FORMAT'
     Contents of block is printed in formats specified by FORMAT.
     FORMAT can contain following format codes:
    H
          Hexadecimal.

    D
          Decimal.

    O
          Octal.

    A
          Ascii, nonprintable characters are printed as space.

    B
          Binary.

     FORMAT can contain several codes, values are then separated by
     hyphen.

`r N STRING'
     Replace bytes with STRING starting at the byte number N of the
     block.

`s/SEARCH/REPLACE/'
     All occurences of SEARCH are replaced by REPLACE. REPLACE can be
     empty. Separator `/' can be replaced by any character not present
     in SEARCH or REPLACE.

`w `file''
     Contents of blocks are written to file `file'. *Note*: Data
     inserted by commands `A', `I', `>' and `<' are written to file
     `file' and `j' and `l' commands have no effect on `w'-commands.
     Zero size files are not preserved.
     Filename can contain format string `%B' or `%nB', these format
     strings are replace by current block number (starting from one),
     causing every block to have it's own file.  In `%nB', the `n' is
     field width in range 0-99. If `n' has a leading zero, then the
     block numbers will be left padded with zeroes.

`y/SOURCE/DEST/'
     Translate bytes in SOURCE to the corresponding bytes in DEST.
     SOURCE and DEST must have equal length.  Separator `/' can be
     replaced by any character not present in SOURCE or DEST.

`& C'
     Performs binary and with C on block contents.

`| C'
     Performs binary or with C on block contents.

`^ C'
     Performs exclusive or with C on block contents.

`~'
     Performs binary negation on block contents.

`x'
     Exchange the contents of nibbles (half an octet) of bytes.


File: bbe.info,  Node: Limits,  Prev: Commands,  Up: Invoking bbe

3.4 Limitations
===============

At least in GNU/Linux `bbe' should be able to handle big files (> 4
GB), other systems are  not tested.

   There are however, some limitations in block and command definitions:

_Strings in block definition_
_Search string in `s' command_
     are limited to _16384_ bytes.


File: bbe.info,  Node: bbe programs,  Next: Problems,  Prev: Invoking bbe,  Up: Top

4 How `bbe' works
*****************

`bbe' scans the input stream just once, so the last block may differ
from the block definition, because `bbe' doesn't 'peek' the end of the
input stream. Last block may be shorter than defined, e.g. if block is
defined as `/string/:128' and if the end of input stream is found
before 128'th byte of the last block is reached, the last block remains
shorter.

Basic execution cycle:
----------------------

  1. Start of the block is searched. If found, data before block is
     written to output stream (unless `-s' is defined) and step 2 is
     executed.

  2. Block commands affecting the start of the block (`I', `D', `J',
     `N', `F', `>' and `B') are executed.

  3. The block is scanned byte by byte and all byte commands (lower
     case letters) are executed.  *Note*: Commands are executed on
     results of previous commands, if e.g. the first byte of the block
     is deleted, the following  commands don't 'see' the removed byte.

  4. When end of the block is reached the end of the block commands
     (`A' and `<') are executed.

  5. Next block is searched, data between the blocks, if not suppressed
     with `-s', is written to output stream.

Few examples:
-------------

`echo "The quick brown fox jumps over a lazy dog" | bbe -b "/The/:21" -e "j 4" -e "s/ /X/"'
     Output is
          The quickXbrownXfoxXjumps over a lazy dog
     The only block in this is
          The quick brown fox j
     All spaces in the block are converted to X's, before conversion
     first 4 bytes are skipped.
`echo "The quick brown fox jumps over a lazy dog" | bbe -b ":/ /" -e "J 1" -e "A \x0a"'
     Output is:
          The quick
          brown
          fox
          jumps
          over
          a
          lazy
          dog
     All blocks end at space, a newline character is inserted after
     every block except the first block.
`echo "The quick brown fox jumps over a lazy dog" | bbe  -e "r 4 fast\x20" -e "s/f/c/"'
     Output is:
          The cast  brown cox jumps over a lazy dog
     Also the `f' in `fast' is converted to `c'.
`echo "1234567890" | bbe -b ":1"  -e "L 9" -e "A -"'
     Output is
          1-2-3-4-5-6-7-8-9-0
     Hyphen is inserted after every 1 byte long block,but not after
     9'th block.
`bbe -s -b "/First line/:/Last line/" /tmp/text'
     Print lines between sentences `First line' and `Last line'.
`bbe -s -b "%<a %:%</a>%" -e "s/\x0a/ /" -e "A \n" ./index.html'
     Extract all links from `./index.html'. To get one link per line,
     all newlines are converted to spaces and newline is added after
     every link.
`bbe -b "/\x5f\x28\x02/:10" -s  -e "F d"  -e "p h" -e "A \n" ./bindata'
     10 bytes long sequences starting with values `x5f' `x28' and `x02'
     are printed as hex values.  Also the file offset is printed before
     each sequence and new line is added after every sequence.  Example
     output:
          52688:x5f x28 x02 x32 x36 x5f x81 x64 x01 x93
          68898:x5f x28 x02 x39 x46 x5f x81 x64 x41 x05
          69194:x5f x28 x02 x42 x36 x5f x81 x64 x41 x05

`bbe -b "/Linux/:5" -s -e "N;D;A \x0a" /bin/* | uniq'
     Print the file names of those programs in /bin directory which
     contains word `Linux'.  Example output:
          /bin/loadkeys:
          /bin/mkbimage:
          /bin/ps:
          /bin/uname:

`bbe -b "/\x5f\x81\x18\x06/:10" -s -e "B d;d 0 4;c BCD ASC;A \n" ./bindata'
     Print BCD numbers and their block numbers in ascii format. Numbers
     start with sequence `x5f' `x81' `x18' `x06'.  The start sequence
     is not printed.
`bbe -b "/\x5f/:2" -e "j 1;& \xf0" -o newdata bindata'
     The least significant nybble of bytes after `x5f' is cleared.
`bbe -b "/\xff\xd8\xff/:/\xff\xd9/" -s -e "w pic%02B.jpg" -o /dev/null manual.pdf'
     Extract jpg-images from pdf-file to separate jpg-files (assuming
     that the jpg start/stop sequences does not appear in other context
     than jpg-images).  Files will be named as `pic01.jpg',
     `pic02.jpg', `pic03.jpg',...
`bbe -b "_<body>_:_</body>_" -s -o temp nicebody.html'
`bbe -b "_<body>_:_</body>_" -e "D;< temp" -o tmpindex.html index.html'
`mv tmpindex.html index.html'
     The body part of the html-document `index.html' is replaced by the
     body of the document `nicebody.html'.


File: bbe.info,  Node: Problems,  Prev: bbe programs,  Up: Top

5 Reporting Bugs
****************

If you find a bug in `bbe', please send electronic mail to
<tjsa@iki.fi>.  Include the version number, which you can find by
running `bbe --version'.  Also include in your message the output that
the program produced and the output you expected.

   If you have other questions, comments or suggestions about `bbe',
contact the author via electronic mail to <tjsa@iki.fi>.  The author
will try to help you out, although he may not have time to fix your
problems.



Tag Table:
Node: Top900
Node: Overview1960
Node: Samples2447
Node: Invoking bbe3845
Node: Invocation4379
Node: Block5394
Node: Commands7440
Node: Limits12278
Node: bbe programs12656
Node: Problems17041

End Tag Table