Oracle Database Export and Import to Gzip Through a Named Pipe
Oracle Database Export and Import to Gzip Through a Named Pipe
When you need to export a large Oracle database but do not have enough disk space for a full uncompressed dump file, piping the export directly through gzip using a Unix named pipe is a proven technique. This method compresses data on the fly, producing a .dmp.gz file that is typically only 20% of the size of the original uncompressed dump.
What is a Named Pipe?
A named pipe (also called a FIFO β First In, First Out) is a special file on Unix/Linux that acts as a data channel between two processes. Unlike a regular file, it does not store data on disk β it passes data directly from one program to another in memory. This makes it the perfect bridge between Oracle's exp/imp utilities and a compression tool like gzip.
The Code
Export to Gzip
1# Step 1: Create a named pipe
2mknod apipe p
3
4# Step 2: Start gzip in the background β reads from the pipe, writes to a compressed file
5gzip < apipe > apipefilename.dmp.gz &
6
7# Step 3: Run Oracle export, pointing the dump file at the named pipe
8exp ... file=apipe ...
Import from Gzip
1# Step 1: Create a named pipe
2mknod apipe p
3
4# Step 2: Start gunzip in the background β reads compressed file, writes to the pipe
5gunzip -c apipefilename.dmp.gz > apipe &
6
7# Step 3: Run Oracle import, reading the dump file from the named pipe
8imp ... file=apipe ...
Line-by-Line Breakdown
Export Breakdown
| Line | Command | Purpose |
|---|---|---|
| 1 | mknod apipe p | Creates a named pipe (FIFO special file) called apipe |
| 2 | gzip < apipe > apipefilename.dmp.gz & | Reads raw data from the pipe, compresses it, writes to .dmp.gz; runs in background with & |
| 3 | exp ... file=apipe ... | Runs Oracle export, writing dump output to the pipe instead of a regular file |
How data flows during export: exp writes β apipe β gzip reads and compresses β .dmp.gz file on disk.
Import Breakdown
| Line | Command | Purpose |
|---|---|---|
| 1 | mknod apipe p | Creates a new named pipe for the import session |
| 2 | gunzip -c apipefilename.dmp.gz > apipe & | Decompresses the .dmp.gz file and writes raw data into the pipe; runs in background with & |
| 3 | imp ... file=apipe ... | Runs Oracle import, reading the dump data from the pipe as if it were a normal file |
How data flows during import: .dmp.gz file β gunzip decompresses β apipe β imp reads and loads.
Key Points
mknod apipe pβ Thepflag tellsmknodto create a FIFO (named pipe), not a regular file. The pipe has no size on disk; it only passes data between processes.[^4]- Background
&operator β Bothgzipandgunzipmust be started before the Oracleexp/impcommand and run in the background. If they are not backgrounded, the terminal will block waiting for input and the commands will never execute.[^6] file=apipeβ Oracle'sexpandimputilities treat the pipe exactly like a normal dump file. They do not know compression is happening; the pipe is transparent to them.[^3]gunzip -cβ The-c(or--stdout) flag sends decompressed output to standard output (the pipe), rather than creating a new file. Without-c,gunzipwould try to write to a.dmpfile on disk.[^6]- No temporary uncompressed file β The entire process is done in memory/stream. There is never a full uncompressed
.dmpfile on disk, which is the main advantage of this approach.[^1]
Important Insights
Disk Space Savings
Compression through gzip typically reduces Oracle dump file size by 70β80%, meaning a 100 GB dump can become as small as 20β30 GB. This is especially critical on servers with limited storage.
Low-Performance System Warning
On slow systems, imp may try to read from the pipe faster than gunzip can decompress data. In this case, add the RECORDLENGTH parameter to slow down imp's read speed.
1imp system/pwd@sid RECORDLENGTH=4096 file=apipe log=imp.log ...
Cleanup After Use
After the export or import completes, the pipe file should be removed to avoid errors on the next run.
1rm -f apipe
Not Compatible with Data Pump (expdp/impdp)
This technique works only with the original exp/imp utilities (Oracle's legacy export/import tools). Oracle Data Pump (expdp/impdp) checks whether the file already exists before writing and will throw an error if a pipe file is present. For Data Pump compression, use the COMPRESSION parameter or a wrapper script instead.
Permissions on the Pipe
If you encounter permission errors, set open permissions on the pipe file.
1chmod 777 apipe
Multiple Pipes for Large Exports
For very large databases, you can create multiple named pipes and pass all of them to exp using a comma-separated file list, running a separate gzip process in the background for each pipe. This speeds up the export and splits the output into multiple compressed files.
Full Working Example
Export a Schema to Gzip
1cd /home/oracle
2mknod exp_scott.pipe p
3gzip < exp_scott.pipe > exp_scott.dmp.gz &
4exp userid=scott/tiger file=exp_scott.pipe log=exp_scott.log owner=scott
5rm -f exp_scott.pipe
Import a Schema from Gzip
1cd /home/oracle
2mknod imp_scott.pipe p
3gunzip -c exp_scott.dmp.gz > imp_scott.pipe &
4imp userid=scott/tiger file=imp_scott.pipe log=imp_scott.log fromuser=scott touser=scott
5rm -f imp_scott.pipe
References
- Oracle FAQ β Import/Export FAQ (Named Pipe technique for exp/imp with gzip): https://www.orafaq.com/wiki/Import_Export_FAQ
- Alex Lima β Export/Import to/from a Compressed File (step-by-step gzip pipe examples): https://alexlima.com/2008/10/18/exportimport-tofrom-a-compressed-file/
- Dave OUK β Oracle Export/Import Tips and Tricks (pipe sizing and multiple pipe technique): https://daveouk.wordpress.com/2013/08/13/oracle-exportimport-tips-and-tricks-2-2/
- Aditya Nath Oracle DBA β Compression of Dumpfiles with EXP/IMP/EXPDP (pipe-based compression methodology): https://adityanathoracledba.com/2015/09/06/compression-of-dumpfiles-with-expimpexpdp/
- Smahamed Blog β Compress EXP/IMP Using Pipes (full shell script examples with sysdba): http://smahamed.blogspot.com/2011/01/compress-expimp-using-pipes.html
- Gavin Soorma β Export and Import Using Unix Pipes and Compression (parfile-based approach): https://gavinsoorma.com.au/knowledge-base/export-import-using-unix-pipes-and-compression/
- Spiceworks Community β Compressing Data While Exporting (RECORDLENGTH and permission tips): https://community.spiceworks.com/t/compressing-the-data-while-exporting-data-urgent/922724
- Oracle Forums β Export and Import Using Named Pipes (RAC and AIX discussion): https://forums.oracle.com/ords/apexds/post/export-and-import-using-named-pipes-8334
- DBA RJ β Run expdp with Data Compression in Oracle 10g and 11g (Data Pump alternative): https://www.dbarj.com.br/en/2013/05/run-expdp-compression-oracle-10g-11g/
- Rudnik Inc β Unix Named Pipes and Export/Import Utilities (foundational concept): https://www.rudnikinc.com/samples/sample8.html