[Top] [Contents] [Index] [ ? ]

Zlibc

Introduction

The zlibc package allows transparent on the fly uncompression of gzipped files. Your programs will be able to access any compressed file, just as if they were uncompressed. Zlibc will transparently uncompresses the data from these files as soon as they are read, just as a compressed filesystem would do. No kernel patch, no recompilation of these executables and no recompilation of the libraries is needed.

It is not (yet) possible execute compressed files with zlibc. However, there is another package, called tcx, which is able to uncompress executables on the fly. On the other hand tcx isn't able to uncompress data files on the fly. Fortunately, the both zlibc and tcx may coexist on the same machine without problems.

1. Where to get zlibc  Where to get zlibc and early bug fixes
2. Installing zlibc  How to install zlibc
3. Using zlibc  How to configure your system to use zlibc
4. How it works  The general principles behind zlibc
5. Customization  How to customize zlibc to your needs
6. Porting zlibc  How to port zlibc to new architectures
Variable index  Variable Index
Concept index  Concept Index
This documentation looks most pretty when printed or as html. Indeed, in the info version certain examples are difficult to read due to the confusing quoting conventions of info.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

1. Where to get zlibc

Zlibc can be found at the following places (and their mirrors):
 
ftp://zlibc.linux.lu/zlibc-0.9k.tar.gz
ftp://www.tux.org/pub/knaff/zlibc/zlibc-0.9k.tar.gz
ftp://ibiblio.unc.edu/pub/Linux/libs/compression/zlibc-0.9k.tar.gz
ftp://ftp.gnu.org/gnu/zlibc/compression/zlibc-0.9k.tar.gz

Before reporting a bug, make sure that it has not yet been fixed in the Alpha patches which can be found at:
 
http://zlibc.linux.lu/
http://www.tux.org/pub/knaff/zlibc

These patches are named zlibc-version-ddmm.taz, where version stands for the base version, dd for the day and mm for the month. Due to a lack of space, I usually leave only the most recent patch.

There is an zlibc mailing list at zlibc @ www.tux.org . Please send all bug reports to this list. You may subscribe to the list by sending a message with 'subscribe zlibc @ www.tux.org' in its body to majordomo @ www.tux.org . (N.B. Please remove the spaces around the "@" both times. I left them there in order to fool spambots.) Announcements of new zlibc versions will also be sent to the list, in addition to the linux announce newsgroups. The mailing list is archived at http://www.tux.org/hypermail/zlibc/latest


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

2. Installing zlibc

  1. If you install zlibc on Linux, make sure that your shared loader (ld-linux.so.1/ld.so) understands LD_PRELOAD. (Best if ld.so-1.8.5 or more recent)

  2. Type ./configure. This runs the GNU autoconfigure script which configures the `Makefile' and the `config.h' file. You may compile time configuration options to ./configure, see for details.

  3. Type make to compile zlibc.

  4. Type make install to install zlibc and associated programs to its final target.

  5. To use this module, set the environment variable LD_PRELOAD to point to the object. Example (sh syntax):

     
          LD_PRELOAD=/usr/local/lib/uncompress.so
          export LD_PRELOAD
    

    or (csh syntax):

     
          setenv LD_PRELOAD /usr/local/lib/uncompress.so
    
    On linux, use /lib/uncompress.so instead of /usr/local/lib/uncompress.so .

    You might want to put these lines in your `.profile' or `.cshrc' in order to have the uncompressing functions available all the time.

  6. Compress your files using gzip and enjoy

For security reasons, the dynamic loader disregards environmental variables such as LD_PRELOAD when executing set uid programs.

However, on Linux, you can use zlibc with set uid programs too, by using one of the two methods described below:

  1. You may ing the path to `uncompress.so' into `/etc/ld.so.preload' instead of using LD_PRELOAD.

    WARNING: If you use `/etc/ld.so.preload', be sure to install `uncompress.so' on your root filesystem, for instance in /lib, as is done by the default configuration. Using a directory which is not available at boot time, such as /usr/local/lib will cause trouble at the next reboot!

    It is also careful to remove zlibc from `/etc/ld.so.preload' when installing a new version. First test it out using LD_PRELOAD, and only if everything is ok, put it back into `/etc/ld.so.preload'.

  2. If you have a version of ld.so which is more recent than 1.9.0, you can set LD_PRELOAD to just contain the basename of `uncompress.so' without the directory. In that case, the file is found as long as it is in the shared library path (which usually contains `/lib' and `/usr/lib')). Because the search is restricted to the library search path, this also works for set-uid programs.

    Example (sh syntax):

     
          LD_PRELOAD=uncompress.so
          export LD_PRELOAD
    

    or (csh syntax):

     
          setenv LD_PRELOAD uncompress.so
    

    The advantage of this approach over `ld.so.preload' is that zlibc can more easily be switched off in case something goes wrong.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

3. Using zlibc

Once zlibc is installed, simply compress your biggest datafiles using gzip. Your programs are now able to uncompress these files on the fly whenever they need them.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

3.1 Zlibc and links


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

3.1.1 Symbolic links

After compressing your datafiles, you also need to change any potential symbolic links pointing to them. Let's suppose that `x' is a symlink to `tstfil':

 
> echo 'this is a test' >tstfil
> ln -s tstfil x
> ls -l
total 1
-rw-r--r--   1 alknaff  sirac          15 Feb 25 19:40 tstfil
lrwxrwxrwx   1 alknaff  sirac           8 Feb 25 19:40 x -> tstfil

After compressing it, you'll see the following listing:

 
> gzip tstfil
> ls -l
total 1
pr--r--r--   1 alknaff  sirac          15 Feb 25 19:40 tstfil
lrwxrwxrwx   1 alknaff  sirac           8 Feb 25 19:40 x -> tstfil

`Tstfil' is now shown as a pipe by zlibc in order to warn programs that they cannot seek in it. Zlibc still shows it with its old name, and you can directly look at its contents:
 
> cat tstfil
this is a test

However, `tstfil' is not yet accessible using the symbolic link:
 
> cat x
cat: x: No such file or directory

In order to make `tstfil' accessible using the link, you have to destroy the link, and remake it:
 
> rm x
/bin/rm: remove `x'? y
> ln -s tstfil x
> ls -l
total 1
pr--r--r--   1 alknaff  sirac          15 Feb 25 19:40 tstfil
lrwxrwxrwx   1 alknaff  sirac           8 Feb 25 19:44 x -> tstfil
> cat x
this is a test


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

3.1.2 Hard links

If you compress datafiles with hard links pointing to them, gzip refuses to compress them.

 
> echo 'this is a test' >tstfil
> ln tstfil x
> ls -li
total 2
    166 -rw-r--r--   2 alknaff  sirac          15 Feb 25 19:46 tstfil
    166 -rw-r--r--   2 alknaff  sirac          15 Feb 25 19:46 x
> gzip tstfil
gzip: tstfil has 1 other link  -- unchanged

Thus you need to remove these hard links first, and remake them after compressing the file.

 
> rm x
/bin/rm: remove `x'? y
> gzip tstfil
> ln tstfil x
> ls -li
total 2
    167 pr--r--r--   2 alknaff  sirac          15 Feb 25 19:46 tstfil
    167 pr--r--r--   2 alknaff  sirac          15 Feb 25 19:46 x
> cat x
this is a test


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

4. How it works

Usually, programs don't make system calls directly, but instead call a library function which performs the actual system calls. For instance, to open a file, the program first calls the open library function, and then this function makes the actual syscall. Zlibc overrides the open function and other related functions in order to do the uncompression on the fly.

If the open system call fails because the file doesn't exist, zlibc constructs the filename of a compressed file by appending .gz to the filename supplied by the user program. If this compressed file exists, it is opened and piped trough gunzip, and the descriptor of the read end of this pipe is returned to the caller.

In some cases, the compressed file is first uncompressed into a temporary file, and a read descriptor for this file is passed to the caller. This is necessary if the caller wants to call lseek on the file or mmap it. A description of data files for which using temporary is necessary can be given in the configuration files `/usr/local/etc/zlibc.conf' (`/etc/zlibc.conf' on Linux)(1) and `~/.zlibrc'. See section 5.2 Configuration files, for a detailed description of their syntax.

Many user programs try to check the existence of a given file by other system calls before actually opening it. That's why zlibc also overrides these system calls. If for example the user program tries to stat a file, this call is also intercepted.

The compressed file, which exists physically on the disk, is also called 'the real file', and the uncompressed file, whose existence is only simulated by zlibc is called 'the virtual file'.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

5. Customization

The behavior of zlibc can be tailored using configuration files or environment variables. This customization should normally not be needed, as the compiled-in defaults are already pretty complete.

5.1 Environmental variables  
5.2 Configuration files  
5.3 Compiled-in defaults  
5.4 Compile-time configuration via GNU autoconf  


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

5.1 Environmental variables

Environmental variables come in two kinds: switch variables have a boolean value and can only be turned on or off, whereas string variables can have arbitrary strings as values.

5.1.1 Switch variables  
5.1.2 String variables  


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

5.1.1 Switch variables

These variables represent a flag which can be turned on or off. If their value is on or 1 they are turned on, if their value is off or 0 they are turned off. All other values are ignored. If the same flag can be turned on or off using config files, the environmental variable always has the priority.

LD_ZLIB_VERBOSE
If this variable is turned on, informational messages are printed on many operations of zlibc. Moreover, error messages are printed in order to point out errors in the configuration files, if any. If this variable is turned off, errors are silently ignored.

LD_ZLIB_UNLINK
If this variable is turned on, and if the user program tries to unlink a virtual (uncompressed) file, zlibc translates this call into unlinking the real file. If this variable is turned off, unlink calls on virtual files are ignored.

LD_ZLIB_DISABLE
If this variable is turned on, zlibc is switched off.

LD_ZLIB_READDIR_COMPR
If this variable is turned on, the readdir function shows the real (compressed) files instead of the virtual (uncompressed) files.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

5.1.2 String variables

These variables have a string value, which represent a file, a directory or a command.

LD_ZLIB_TMP
This is the name of the directory where the temporary uncompressed files are put. The default is /tmp.

LD_ZLIB_EXT
This is the extension which is appended to a virtual file name in order to obtain the real (compressed) file name. The default is .gz.

LD_ZLIB_UNCOMPRESSOR
This is the name of the program to be invoked to uncompress the data. Default is gzip -dc.

LD_ZLIB_CONFFILE
This is the name of an additional configuration file. If this variable is defined and if the corresponding file exists, the configuration described in this file overrides the configurations in `~/.zlibrc' and in `/usr/local/etc/zlibc.conf' (`/etc/zlibc.conf' on Linux).


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

5.2 Configuration files

To get its configuration, zlibc first looks into the file described by the environment variable LD_ZLIB_CONFFILE (if any), then in `~/.zlibrc' and finally in `/usr/local/etc/zlibc.conf' (`/etc/zlibc.conf' on Linux (2)). If the desired information is found in neither of these files, the compiled-in defaults are used. It is possible to supply only part of the needed information in the configuration files. In that case, the missing information is retrieved from the compiled-in defaults. This allows you to have really small runtime configuration files, which only list the differences between the desired configuration and the compiled-in configuration.

If an error occurs while parsing one of the configuration files, the offending file is skipped, and the search continues with the next file. However, no error message is printed unless the environmental variable LD_ZLIB_VERBOSE is turned on (i.e. set to 1 or to on ).

If two files contain contradictory information, the information in the file which is scanned first is retained (usually `~/.zlibrc'). If any flags have been set or unset using environmental variables, these settings override the flags specified in the configuration files.

The configuration files are read by each process. For each process, they are read at most once, at the time when zlibc is first used (attempt to access a compressed file). Afterwards they are cached in the process's virtual memory. Thus, changing zlibc configuration files doesn't generally have any effect on already running processes.

5.2.1 Overall structure  
5.2.2 Commands section  
5.2.3 Class section  


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

5.2.1 Overall structure

The zlibrc files consist of two sections: A commands section (5.2.2.2 Available commands line flags) and a class definition section (5.2.3 Class section).

The commands section describes how zlibc should behave depending on the executable that it was called from. Several commands are grouped together into command classe.

The class definition section describes how zlibc should behave depending on the class of the command and the datafile opened.

The configuration file may contain comments in both sections: a comment starts with a hash (#) and stops at the end of the line.

Dashes (-) and underbars (_) may be used indifferently in all keywords.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

5.2.2 Commands section

5.2.2.1 General syntax of the commands section  What goes into the commands section
5.2.2.2 Available commands line flags  Which flags are available


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

5.2.2.1 General syntax of the commands section

The format for a line in the "commands" section is as follows:

 
commands "cmd1" [ ... "cmdn" ] use flags "class"

In this line, the cmd1 ... cmdn are the basenames of the programs (commands) for which this line should apply. The basename is the name without the path, i.e. ls instead of /usr/bin/ls. The command names should be enclosed in quotes. You may also use the keyword default (without quotes) to match all commands.

The flags describe those aspects of zlibc's behaviour that are independant from the datafile which is being accessed. These flags come in pairs. The flags need not to be listed on a single commands line, they may occur in several places, even in several different configuration files (for instance one in /etc/zlibc.conf and another one in ~/.zlibrc.

If two contradictory flags are found in the configuration files, the one which is seen first is taken.

If on the other hand a certain flag is not found at all in the configuration files, the compiled-in default for this flag is used. This is usually the second flag of each pair, described below (see section 5.2.2.2 Available commands line flags).

These flags can all be overridden by environmental variables. When the corresponding environmental variable is set to 1 or to on, the first flag is used, when it is set to 0 or to off. If the environmental variables is set to neither of these 4 values, it is ignored).

The class names the commands class that these commands belong to. If, for a given command, two commands lines give different classes, the one which is seen first is taken. No union of classes is made, the classes are always treated as a whole. Thus, if you want to make a change to a command class, you need to describe it in its entirety.

The following example says that the tar, cpio, pax, cp and mv show compressed files in a directory listing (readdir_compr flag), and are of class generic_safe.
 
commands "tar" "cpio" "pax" "cp" "mv" use readdir_compr "generic_safe"

The class generic_safe would then need to be described further in the class section.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

5.2.2.2 Available commands line flags

This section describes the flags which can be used on a commands line. All these flags come in pairs.

The table below describes each of these pairs. The first word in the header of each item is the non-default flag, the second word is the default flag, and the third word is the environmental variable by which you can override the settings from the configuration files. If this environmental variable is turned on (set to 1), the non-default (first) flag is taken, if it is turned off (set to 0), the default (second) flag is taken.

disable / enable / LD_ZLIB_DISAB
The disable flag disables zlibc for the programs on this commands line. This is useful for compression and uncompression utilities. Without this flag, gunzip would not work anymore, because it would think that the uncompressed file exists already, and it would refuse to overwrite this file.

disable_child / enable_child / LD_ZLIB_DISAB_CHILD
The disable_child flag disables zlibc for the programs started by programs on this commands line. This is implemented by removeing all occurrences of uncompress.so from the LD_PRELOAD environment variable. This function is useful for programs such as xemacs, in order to make sure that all launched subprograms return results consistent with emacs itself (directory listings, etc)

readdir_compr / readdir_uncompr / LD_ZLIB_READDIR
The readdir function shows the uncompressed files (i.e. with their trailing .gz extension) when the readdir_compr flag is set, and the compressed files otherwise.

verbose / silent / LD_ZLIB_VERBOSE
When verbose is set, zlibc prints informational messages.

unlink / no_unlink / LD_ZLIB_UNLINK
If the unlink flag is set, and if the user program tries to unlink a virtual (uncompressed) file, the package translates this call into unlinking the real file. If the no_unlink flag is set, requests to unlink virtual (uncompressed) files are silently ignored.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

5.2.3 Class section

A command class(3) definition describes those aspects of zlibc's behavior that depend on the name of the datafile which is being accessed. Command classes are identified by a name which is matched against the class parameter from the commands. The class name should be enclosed between quotes both in the commands line and in the class definition.

The class section contains descriptions of different command classes (i.e descriptions how datafiles should be uncompressed). Each class definition begins with a line of the following format:

 
class "id"

The class id is the same string as the one used in the commands line. The remaining lines of a class definition are as follows:
 
[ [ criterion ] [ "name" ]] mode

The following example shows the definition of the class used for X-servers.

 
# X uses tmp files in its own directories.
class "X"
subdir "/usr/X11R6" usetmpfile
subdir "/usr/X386" usetmpfile
subdir "/usr/lib/X11" usetmpfile
showpipe

This says that all compressed files who are in a subdirectory below `/usr/X11R6', `/usr/X386' or `/usr/lib/X11' are decompressed using temporary files (usetmpfile), and that files from other directories are decompressed using pipes (showpipe)

The following examples illustrates a command class, named nopipe, which always uses temporary files for decompression
 
# generic class which uses temp files for all files.
class "nopipe"
usetmpfile

5.2.3.1 File selection Criteria  To which data files does this line apply
5.2.3.2 Mode  How zlibc behaves for the files matching the criterion


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

5.2.3.1 File selection Criteria

The criterion describes what parts of the filename should match:

filename
The entire filename of the target data file must match name.

basename
The basename (filename without directory) of the target data file must match name.

directory
The data file must live in the directory name. If the user program opens the file with an absolute pathname, that filename is used as is. If on the other hand the user program uses a relative pathname, zlibc uses the most direct path to the file (i.e. without symlinks).

subdirectory
The data file must live in the directory name or in one of its subdirectories. If the user program opens the file with an absolute pathname, that filename is used as is. If on the other hand the user program uses a relative pathname, zlibc uses the most direct path to the file (i.e. without symlinks).

suffix
The data file's name must end in name. This is useful for selecting files according to their extension.

filesystem
The data file must live on the same filesystem as name. This criterion can for example be used for example to disable zlibc on a doublespaced filesystem (where zlibc would be redundant), or to switch off uncompressed size reporting on an ftpfs filesystems (4).

default
All data files match. All class definitions must have a default line, and this default line must be the last line of the definition. The default criterion needs no 'name' parameter.

all
All data files match. Unlike default, this line needs not to be the last line of the class definitions. Thus it is possible to specify several all lines for different aspects of zlibc behaviour.

The lines of each class definition are scanned in sequence, and, for each aspect(5), the first matching line is adopted. The class may be defined in another initialisation file, if this file is scanned later. The criterion parameter may be omitted if it can be deduced from the name. In that case, the following heuristics are used:

filesystem criteria, because these tend to be more predictable.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

5.2.3.2 Mode

The mode describes the behavior of zlibc in certain situations. These are:

  1. What to do when a file is accessed in readonly mode.
  2. What to do when a user program tries to append data to an existing file.
  3. What to do when a user program tries to create a new file.
  4. What to do when a user program tries to access an existing file and none of the preceding situations apply (see section Other write requests).
  5. Whether to show the size of the uncompressed file or the size of the compressed file as a response to stat (see section Size shown for compressed files).

The mode for each off these 5 situations has to be described separately. For each of these 5 situations, the scan through the class description is done until the reaction of zlibc for this situation is defined.

Reaction to a readonly request  Behavior for programs wanting to only read files
Reaction to an append request  - for programs wanting to add more data to existing files
Reaction to a file creation request  - for programs wanting to create new files
Other write requests  - for programs wanting to apply other modifications to files
Size shown for compressed files  - for programs wanting to know the size of a file


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

Reaction to a readonly request

The reaction to a readonly request may be one of the following:

show-pipe
The uncompressed data file is sent to the user program using a pipe. This consumes very few resources, and it allows the decompression to run in paralell with the user process, but it has the disadvantage that the user program cannot use lseek. To warn the user program of this, the data file is shown as a named pipe (FIFO) when it is stat'ed.

use-tmp-file
This is the default setting. The data is uncompressed and put into a temporary file. The user programs then reads its data from the temporary file. This has the advantage that the user program may lseek, and the disadvantage that more disk space is consumed, while the programming is accessing the file.

hide-pipe
The data is sent through a pipe, but the file is shown as a regular file ("hidden") to the user program when it stats it. This might be needed for programs which are picky about a file type, but who actually don't need lseek.

leave-compressed
The virtual (uncompressed) file is shown as non-existent to stat, and readdir shows the physical (compressed) file. For certain programs, this is enough to disable zlibc on a file per file basis. This is useful, for example, to make emacs use its own compression support (crypt.el) instead of zlibc. Crypt.el is able to compress files when writing them back, whereas zlibc isn't able to do so. The leave-compressed doesn't work correctly with the directory and subdirectory criteria. Use the filesystem criterion instead.

dir-leave-compressed
The virtual (uncompressed) file is shown to stat, but readdir shows the physical (compressed) file. This is useful to tell zlibc that we prefer working on the physical file, but without making access to the virtual file impossible. The dir-leave-compressed doesn't work with the dir and subdir criteria. Use the filesystem criterion instead.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

Reaction to an append request

The reaction to an append request may be one of the following:

append-compressed
When a user program tries to append data to a non-existant file, but the corresponding compressed file exists, zlibc translates this request into appending the compressed data to the compressed file.

WARNING-1:

This works with gzip, and might not work with other (un)compressors! This relies on gzip's feature to consider a concatenation of compressed files as a compression of concatenated files.

WARNING-2:

This is only reliable if you can guarantee that the file is not accessed by some other program while the first one has it still open, or that it won't be opened again (even by the same program) shortly after it has been closed. Delays longer than a second should be ok. Don't enable append-compressed if you expect the file to be written to by several programs at once.

no-append-compressed
Don't append to a compressed file (the user program will get a file not found error). This is the default behavior.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

Reaction to a file creation request

The reaction to a create request may be one of the following:

create-compressed
When a user program tries to create a new file whose name matches the pattern, zlibc translates this request into creating a compressed file. A file is considered to be created, if the O_TRUNC flag is set, or if both O_CREAT and O_EXCL are set. This only applies to files opened write-only.

WARNING-1:

This is only reliable if you can guarantee that the newly created file is not accessed by some other program while the first one has it still open, or that it won't be opened again (even by the same program) shortly after it has been closed.

WARNING-2:

This should not be used if you expect the program to seek in this file.

no-create-compressed
Newly created files are created uncompressed. This is the default.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

Other write requests

The reaction to other write requests may be one of the following:

uncompress-before-write
When a user program tries to write to a non-existing file, and when a compressed file with a corresponding name exists, this file is uncompressed in place (i.e. having the same name, but without the .gz extension)

WARNING-1:

This is only safe when you can guarantee that the file is not opened by several programs at once. However, once the call to the open function has returned, other programs may open this file safely. If a second program tries to open the file during the open call of the first, this second program gets a permission error.

WARNING-2:

When using this option, be careful when opening files belonging to another user, or files living in a directory where you have no write access to. Using this option in a directory without write access will result in a permission error. Using this option in a directory where you do have write access will change the ownership of the file to you, even if it belonged to another user initially.

no-uncompress-before-write
Compressed files are not uncompressed before writing to them, and zlibc returns a "file not found" error. This is the default.

If several of these options apply for the same file, create-compressed has priority over append-compressed which has priority over uncompress-before-write.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

Size shown for compressed files

When an application calls stat to fetch the attributes of a file (such as its permissions, size, type, etc.), zlibc stats the corresponding compressed file instead. After doing so, zlibc has to adjust some of the values returned by stat, such as the type and the size. The type has to be adjusted for those files that should be shown as pipes. The size has to be adjusted because user programs are usually interested in the amount of data that they can actually read from the file (i.e. the size of the uncompressed file) rather than the size of the physical file (i.e. the size of the compressed file). However, in order to find out the size of the uncompressed file, zlibc has to read some data of the file, which may impact performance in situations where many files are stat'ed. This is for instance the case for find, or for ls on an ftpfs filesystem. The following two behaviors of the stat call may be specified:

show-compressed-size
Stat returns the size of the compressed file. This is less useful for the application, but more efficient. If you mount any ftpfs filesystems, you may switch on show-compressed-size just for that filesystem by using the filesystem criterion (see section 5.2.3.1 File selection Criteria).

show-uncompressed-size
Stat returns the size of the uncompressed file. This is more useful for the application, but less efficient. This is the default behavior.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

5.3 Compiled-in defaults

It is possible to operate zlibc entirely without configuration files. In this case, it uses the compiled-in defaults. These are generated at compile-time from the `zlibrc.sample' file. This file has the same syntax as the configuration files described above (see section 5.2 Configuration files). If you want to change the compiled-in defaults of zlibc, edit that file, and remake.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

5.4 Compile-time configuration via GNU autoconf

Before it can be compiled, zlibc must be configured using the GNU autoconf script ./configure. In most circumstances, running ./configure without any parameters is enough. However, you may customize zlibc using various options to ./configure. The following options are supported:

--prefix directory
Prefix used for any directories used by zlibc. By default, this is `/usr/local'. Zlibc is installed in `$prefix/lib', looks for its system wide configuration file in `$prefix/etc'. Man pages are installed in `$prefix/man', info pages in `$prefix/info' etc. On Linux, if you use zlibc via `/etc/ld.so.preload', you should use `/' as the prefix instead of the default `$prefix/lib'.

--sysconfdir directory
Directory containing the system-wide configuration file `zlibc.conf'. By default, this is derived from prefix (see above).

--disable-runtime-conf
Disables run time configuration via environmental variables and via the configuration files. This may be needed in hyper secure environments.

--disable-env-conf
Disables run time configuration via environmental variables

--disable-have-proc
Tells zlibc not to use the /proc filesystem to find out the commandline of the programs for which it runs, even if a working /proc is detected.

--disable-have-proc
Tells zlibc to use the /proc filesystem to find out the commandline of the programs for which it runs, even if no working /proc is detected.

--with-compr-ext=extension
Uses extension as the filename extension of compressed files. By default, is .gz

--with-extlen=length
Allows to configure compressed filename extensions with at most length character via runtime configuration. By default is 5.

--with-tmpdir=directory
Uses directory to store the uncompressed files. By default is /tmp.

--with-uncompressor=uncompressor-command-line
Defines how the program for uncompressing files should be invoked. This command should read the compressed file from stdin, and output the uncompressed data to stdout By default is gzip -dc.

In addition to the above-listed options, the standard GNU autoconf options apply. Type ./configure --help to get a complete list of these.


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

6. Porting zlibc

Zlibc has been tested on three variants of Unix so far: Linux, SunOs, Solaris. On all three platforms, zlibc has been compiled using GNU make and gcc. However, porting it to other platforms should be straightforward as long as they verify both of the following conditions:

  1. The target platform should have shared library support. The default Makefile links zlibc by supplying the flags -nostdlib -shared to gcc in addition to the usual CFLAGS. If your target platform needs different flags to make shared libraries, change the definition of the variable SHAREDCFLAGS in the Makefile.
  2. The target platform should supply a way to perform system calls directly without going through the library functions. Indeed, this is needed because we are redefining the library functions and thus the default functions become unavailable.

    Usually direct syscalls done by calling the syscall function. For instance, in order to perform open("test", O_RDONLY) directly, sycall(SYS_open, "test", O_RDONLY) is called.

    If your target platform uses a different way to perform direct system calls, you need to supply different definitions for the real_xxx functions in direct_sys.h. This is the case for AIX.

  3. The target platform should provide a way to instruct the dynamic linker to preload programs with extra object files not present in the shared library. Any symbols defined in that object file override equivalent symbols from libc. Usually this is done by pointing the LD_PRELOAD variable to the object file to be used. Unfortunately, I have found no way to achieve this on AIX. Any ideas about how to do this are welcome.

Many platforms, including Solaris, provide several aliases for their syscall stubs. It would be interesting to leave one of them alone, and use it as a "direct syscall" stub, but unfortunately, it turns out that in sometimes both stubs are used by the library! Thus, what would be an advantage, becomes actually a disadvantage, because we will need to override these aliases in addition to the canonical names. This is done in altnames.c. Change it as needed. In order to know the names of these aliases, try the following command: nm /lib/libc.so | grep lstat. This lists any library symbols with unlink in their name, which could be aliases.

If you have successfully ported zlibc to a new platform, could you drop me a note, so that I can include support for that platform in my next release. Thanks


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

Variable index

Jump to:   D   E   L   R   S   V  

Index Entry Section

D
disable5.2.2.2 Available commands line flags

E
enable5.2.2.2 Available commands line flags

L
LD_PRELOAD2. Installing zlibc
LD_ZLIB_CONFFILE5.1.2 String variables
LD_ZLIB_DISABLE5.1.1 Switch variables
LD_ZLIB_EXT5.1.2 String variables
LD_ZLIB_READDIR_COMPR5.1.1 Switch variables
LD_ZLIB_TMP5.1.2 String variables
LD_ZLIB_UNCOMPRESSOR5.1.2 String variables
LD_ZLIB_UNLINK5.1.1 Switch variables
LD_ZLIB_VERBOSE5.1.1 Switch variables

R
readdir_compr5.2.2.2 Available commands line flags
readdir_uncompr5.2.2.2 Available commands line flags

S
show-compressed-sizeSize shown for compressed files
silent5.2.2.2 Available commands line flags

V
verbose5.2.2.2 Available commands line flags

Jump to:   D   E   L   R   S   V  


[ < ] [ > ]   [ << ] [ Up ] [ >> ]         [Top] [Contents] [Index] [ ? ]

Concept index

Jump to:   A   B   C   D   E   F   H   I   L   M   P   R   S   T  

Index Entry Section

A
ALPHA patchesIntroduction

B
basename (selection criterion)5.2.3.1 File selection Criteria
Boot problems2. Installing zlibc
bugsIntroduction

C
class section5.2.1 Overall structure
command line for the uncompressing program (env.var.)5.1.2 String variables
commands section5.2.1 Overall structure
commands section5.2.2.1 General syntax of the commands section
compile-time configuration5.4 Compile-time configuration via GNU autoconf
configure options5.4 Compile-time configuration via GNU autoconf
Customization5. Customization

D
debugging zlibc5.1.1 Switch variables
debugging zlibc5.2.2.2 Available commands line flags
diffsIntroduction
directory (selection criterion)5.2.3.1 File selection Criteria
directory for temporary files5.4 Compile-time configuration via GNU autoconf
directory for temporary files (env.var.)5.1.2 String variables
disabling zlibc (conf. file)5.2.2.2 Available commands line flags
doublespaced DOS filesystem5.2.3.1 File selection Criteria

E
extension (selection criterion)5.2.3.1 File selection Criteria

F
filename (selection criterion)5.2.3.1 File selection Criteria
filename extension for compressed files (env.var.)5.1.2 String variables
findSize shown for compressed files
ftpfsSize shown for compressed files

H
hanging (of ls, on an ftpfs filesystem)Size shown for compressed files

I
Installation2. Installing zlibc

L
location of the configuration file (env.var.)5.1.2 String variables
ls slowness (on an ftpfs filesystem)Size shown for compressed files

M
mailing listIntroduction

P
Parsing order of configuration files.5.2.1 Overall structure
patchesIntroduction
preventing accidental removal of compressed files (env.var.)5.1.1 Switch variables
Processing order of configuration file.5.2 Configuration files

R
removing files5.1.1 Switch variables

S
Set-uid programs2. Installing zlibc
showing compressed files in a directory listing (env.var.)5.1.1 Switch variables
showing compressed files on a directory listing (conf. file)5.2.2.2 Available commands line flags
size of compressed filesSize shown for compressed files
slowness (of find due to reading the size)Size shown for compressed files
slowness (of ls, on an ftpfs filesystem)Size shown for compressed files
suffix (selection criterion)5.2.3.1 File selection Criteria
Symbolic links3. Using zlibc

T
temporary file directory5.4 Compile-time configuration via GNU autoconf

Jump to:   A   B   C   D   E   F   H   I   L   M   P   R   S   T  


[Top] [Contents] [Index] [ ? ]

Footnotes

(1)

Actually the location of the system-wide include file depends on the settings of sysconfdir and prefix during ./configure (see section 5.4 Compile-time configuration via GNU autoconf).

(2)

Actually the location of the system-wide include file depends on the settings of sysconfdir and prefix during ./configure (see section 5.4 Compile-time configuration via GNU autoconf).

(3)

These command classes are unrelated to the classes in object oriented programming.

(4)

uncompressed size reporting (see section Size shown for compressed files would make ls painfully slow on an ftpfs filesystem, as all files would have to be downloaded before their size could be reported.

(5)

See see section 5.2.3.2 Mode


[Top] [Contents] [Index] [ ? ]

Table of Contents


[Top] [Contents] [Index] [ ? ]

Short Table of Contents

Introduction
1. Where to get zlibc
2. Installing zlibc
3. Using zlibc
4. How it works
5. Customization
6. Porting zlibc
Variable index
Concept index

[Top] [Contents] [Index] [ ? ]

About this document

This document was generated by Alain Knaff on February, 28 2005 using texi2html

The buttons in the navigation panels have the following meaning:

Button Name Go to From 1.2.3 go to
[ < ] Back previous section in reading order 1.2.2
[ > ] Forward next section in reading order 1.2.4
[ << ] FastBack previous or up-and-previous section 1.1
[ Up ] Up up section 1.2
[ >> ] FastForward next or up-and-next section 1.3
[Top] Top cover (top) of document  
[Contents] Contents table of contents  
[Index] Index concept index  
[ ? ] About this page  

where the Example assumes that the current position is at Subsubsection One-Two-Three of a document of the following structure:

This document was generated by Alain Knaff on February, 28 2005 using texi2html