Oracle Database Export and Import to Gzip Through a Named Pipe

Oracle Database Export and Import to Gzip Through a Named Pipe

When you need to export a large Oracle database but do not have enough disk space for a full uncompressed dump file, piping the export directly through gzip using a Unix named pipe is a proven technique. This method compresses data on the fly, producing a .dmp.gz file that is typically only 20% of the size of the original uncompressed dump.

What is a Named Pipe?

A named pipe (also called a FIFO β€” First In, First Out) is a special file on Unix/Linux that acts as a data channel between two processes. Unlike a regular file, it does not store data on disk β€” it passes data directly from one program to another in memory. This makes it the perfect bridge between Oracle's exp/imp utilities and a compression tool like gzip.


The Code

Export to Gzip

1# Step 1: Create a named pipe
2mknod apipe p
3
4# Step 2: Start gzip in the background β€” reads from the pipe, writes to a compressed file
5gzip < apipe > apipefilename.dmp.gz &
6
7# Step 3: Run Oracle export, pointing the dump file at the named pipe
8exp ... file=apipe ...

Import from Gzip

1# Step 1: Create a named pipe
2mknod apipe p
3
4# Step 2: Start gunzip in the background β€” reads compressed file, writes to the pipe
5gunzip -c apipefilename.dmp.gz > apipe &
6
7# Step 3: Run Oracle import, reading the dump file from the named pipe
8imp ... file=apipe ...

Line-by-Line Breakdown

Export Breakdown

LineCommandPurpose
1mknod apipe pCreates a named pipe (FIFO special file) called apipe
2gzip < apipe > apipefilename.dmp.gz &Reads raw data from the pipe, compresses it, writes to .dmp.gz; runs in background with &
3exp ... file=apipe ...Runs Oracle export, writing dump output to the pipe instead of a regular file

How data flows during export: exp writes β†’ apipe β†’ gzip reads and compresses β†’ .dmp.gz file on disk.

Import Breakdown

LineCommandPurpose
1mknod apipe pCreates a new named pipe for the import session
2gunzip -c apipefilename.dmp.gz > apipe &Decompresses the .dmp.gz file and writes raw data into the pipe; runs in background with &
3imp ... file=apipe ...Runs Oracle import, reading the dump data from the pipe as if it were a normal file

How data flows during import: .dmp.gz file β†’ gunzip decompresses β†’ apipe β†’ imp reads and loads.


Key Points

  • mknod apipe p β€” The p flag tells mknod to create a FIFO (named pipe), not a regular file. The pipe has no size on disk; it only passes data between processes.[^4]
  • Background & operator β€” Both gzip and gunzip must be started before the Oracle exp/imp command and run in the background. If they are not backgrounded, the terminal will block waiting for input and the commands will never execute.[^6]
  • file=apipe β€” Oracle's exp and imp utilities treat the pipe exactly like a normal dump file. They do not know compression is happening; the pipe is transparent to them.[^3]
  • gunzip -c β€” The -c (or --stdout) flag sends decompressed output to standard output (the pipe), rather than creating a new file. Without -c, gunzip would try to write to a .dmp file on disk.[^6]
  • No temporary uncompressed file β€” The entire process is done in memory/stream. There is never a full uncompressed .dmp file on disk, which is the main advantage of this approach.[^1]

Important Insights

Disk Space Savings

Compression through gzip typically reduces Oracle dump file size by 70–80%, meaning a 100 GB dump can become as small as 20–30 GB. This is especially critical on servers with limited storage.

Low-Performance System Warning

On slow systems, imp may try to read from the pipe faster than gunzip can decompress data. In this case, add the RECORDLENGTH parameter to slow down imp's read speed.

1imp system/pwd@sid RECORDLENGTH=4096 file=apipe log=imp.log ...

Cleanup After Use

After the export or import completes, the pipe file should be removed to avoid errors on the next run.

1rm -f apipe

Not Compatible with Data Pump (expdp/impdp)

This technique works only with the original exp/imp utilities (Oracle's legacy export/import tools). Oracle Data Pump (expdp/impdp) checks whether the file already exists before writing and will throw an error if a pipe file is present. For Data Pump compression, use the COMPRESSION parameter or a wrapper script instead.

Permissions on the Pipe

If you encounter permission errors, set open permissions on the pipe file.

1chmod 777 apipe

Multiple Pipes for Large Exports

For very large databases, you can create multiple named pipes and pass all of them to exp using a comma-separated file list, running a separate gzip process in the background for each pipe. This speeds up the export and splits the output into multiple compressed files.


Full Working Example

Export a Schema to Gzip

1cd /home/oracle
2mknod exp_scott.pipe p
3gzip < exp_scott.pipe > exp_scott.dmp.gz &
4exp userid=scott/tiger file=exp_scott.pipe log=exp_scott.log owner=scott
5rm -f exp_scott.pipe

Import a Schema from Gzip

1cd /home/oracle
2mknod imp_scott.pipe p
3gunzip -c exp_scott.dmp.gz > imp_scott.pipe &
4imp userid=scott/tiger file=imp_scott.pipe log=imp_scott.log fromuser=scott touser=scott
5rm -f imp_scott.pipe

References

Posts in this series