Oracle Export Through a Named Pipe with mknod for Space-Constrained Hosts

Oracle Export Through a Named Pipe with mknod for Space-Constrained Hosts

Purpose

Oracle's original Export utility (exp) writes a binary dump file. On production hosts where disk space is tight — where staging a full-database dump and its compressed copy simultaneously would exhaust the filesystem — the classic workaround is a named pipe (FIFO). A named pipe looks like a file on the filesystem but has no storage: bytes written to one end are immediately readable from the other. By piping the Export directly through gzip or netcat, the dump never touches disk as an uncompressed file.

This technique, documented in the shutdownabort.com DBA Quick Guides (Andrew Barry, 2007–2013, preserved via the Wayback Machine), predates Oracle Data Pump. Data Pump (expdp) has superseded exp for most purposes since Oracle 10g — it runs in parallel, is faster, and supports compression natively. But the named-pipe Export remains relevant for Oracle 9i and early 10g environments still in production, in air-gapped shops where Data Pump destination paths have security controls, and as a general UNIX pattern for streaming any large Oracle-generated file off-host without staging.

Code

 1# Step 1: Create the named pipe (FIFO)
 2mknod /tmp/oracle_pipe p
 3
 4# Step 2: Start Export writing to the pipe (run in background)
 5exp userid=system/password full=y file=/tmp/oracle_pipe &
 6
 7# Step 3: Read from the pipe and compress to a real file (run in foreground)
 8gzip < /tmp/oracle_pipe > /backup/full_exp.dmp.gz
 9
10# Clean up the named pipe after completion
11rm /tmp/oracle_pipe
12
13# Variant: stream directly to another host with netcat — no local file at all
14mknod /tmp/oracle_pipe p
15exp userid=system/password full=y file=/tmp/oracle_pipe &
16nc remotehost 12345 < /tmp/oracle_pipe
17# On remotehost: nc -l 12345 > /backup/full_exp.dmp.gz

Code Breakdown

mknod /tmp/oracle_pipe p

Creates a named pipe (FIFO — First In, First Out) at /tmp/oracle_pipe. The p argument specifies type: pipe. Unlike a regular file, a FIFO has zero storage. Any process that opens it for writing blocks until another process opens it for reading, and vice versa. The FIFO acts as a rendezvous point between the writer (exp) and the reader (gzip).

Create the pipe in /tmp (a local filesystem) even if the target compressed file is on NFS — some versions of exp cannot write to a FIFO on NFS because the NFS client does not support FIFO creation.

exp userid=system/password full=y file=/tmp/oracle_pipe &

Runs the classic Oracle Export utility targeting the named pipe as the output file. From exp's perspective, /tmp/oracle_pipe is just a file path — it has no awareness that it is writing to a pipe. The & runs the command in the background because exp will block on the open() system call until the reader (step 3) also opens the pipe.

The full=y flag exports the entire database. Substitute owner=<schema> or tables=<tablename> for a partial export.

gzip < /tmp/oracle_pipe > /backup/full_exp.dmp.gz

Reads from the named pipe and compresses the stream into the output file. This command runs in the foreground and blocks until exp finishes writing and closes its end of the pipe. At that point, gzip sees EOF and closes the output file.

Ordering constraint

exp and gzip must be started in the correct order or either can deadlock:

  1. Start exp first in the background (&) so it blocks waiting for a reader.
  2. Start gzip second in the foreground so it opens the pipe for reading, unblocking exp.
  3. Both processes then run concurrently until exp finishes.

Reversing the order — starting gzip before exp — also works, but gzip will block waiting for a writer instead of exp blocking waiting for a reader. Either ordering is valid; the background & is what allows both sides to rendezvous.

nc remotehost 12345 < /tmp/oracle_pipe

Replaces gzip with a network transfer. The receiving host must be listening first: nc -l 12345 > /backup/full_exp.dmp.gz. This eliminates local disk entirely — the Export dump travels over the network in real time. Combine with gzip on the sending side for compressed network transfer: gzip -c < /tmp/oracle_pipe | nc remotehost 12345.

Key Points

  • exp is deprecated since Oracle 11.1. Oracle Database 23ai does not include exp/imp. For current systems use expdp with COMPRESSION=ALL instead — it produces a compressed dump without the named-pipe workaround. The named-pipe technique is valuable primarily for Oracle 9i and 10g environments still in production.
  • The named pipe must be on a local filesystem. Some versions of exp cannot write to a FIFO on NFS. Create the pipe in /tmp (local) even if the target compressed file is on NFS.
  • exp does not support parallelism. The single-stream output is a serial bottleneck. A full database Export via pipe will be as slow as the database's single-threaded read speed. For large databases (> 100 GB), this is a practical constraint — Data Pump parallel export is the right tool.
  • Clean up the named pipe after each run. rm /tmp/oracle_pipe removes it. Leaving it in place does not consume disk space, but a subsequent exp targeting the same path will block indefinitely if there is no reader ready.
  • The file= parameter accepts only one pipe. exp supports file=(f1,f2) for multi-file dumps but all targets must be regular files; you cannot point multiple file targets at different pipes simultaneously.

Insights and Best Practices

Estimate export size before starting

Before starting the Export, estimate the compressed size so you know whether the target filesystem has room:

1SELECT ROUND(SUM(bytes)/1024/1024/1024, 1) AS "Estimated GB"
2FROM   dba_segments;

Typical gzip compression on Oracle dump files is 3:1 to 5:1. A 100 GB database usually compresses to 20–30 GB. Confirm the target path has enough space before starting an Export that may take hours.

Data Pump replacement pattern (Oracle 10g+)

The modern equivalent of the named-pipe Export is:

1expdp system/password full=y \
2  dumpfile=full_exp.dmp \
3  logfile=expdp_full.log \
4  compression=all \
5  directory=DATA_PUMP_DIR

COMPRESSION=ALL compresses metadata and data inline without staging an uncompressed file. It requires the Advanced Compression option on most editions; COMPRESSION=METADATA_ONLY is included in Enterprise Edition without extra licensing on Oracle 12c+.

Test the pipe before a production run

Before running a multi-hour full-database Export through a named pipe, validate the pattern with a small table export:

1mknod /tmp/test_pipe p
2exp userid=system/password tables=small_test_table file=/tmp/test_pipe &
3gzip < /tmp/test_pipe > /tmp/test.dmp.gz
4rm /tmp/test_pipe
5ls -lh /tmp/test.dmp.gz
6gzip -t /tmp/test.dmp.gz && echo "OK"

If test.dmp.gz is non-zero and passes the gzip integrity test, the pipe is working correctly and the full Export can proceed with confidence.

When to Run This

  • Exporting a full database on a host with limited staging space (disk < 2× uncompressed dump size)
  • Streaming a schema or full Export directly to a remote host without intermediate files
  • Pre-Oracle-10g environments where Data Pump is not available
  • Disaster-recovery drills on legacy Oracle instances to validate Export still works
  • Space-constrained Oracle XE or SE2 environments where licensing prevents full Data Pump compression

Troubleshooting Common Issues

If exp hangs indefinitely after launching in background, the reader has not opened the pipe. Confirm the gzip command is running in a parallel terminal. Use lsof /tmp/oracle_pipe to verify both processes have the pipe open — you should see one process with w (write) access and one with r (read) access. If only one side is open, the other command failed to start.

If the compressed output file is 0 bytes, exp likely failed immediately — bad password, insufficient privileges, or missing EXPORT FULL DATABASE privilege. Redirect exp's stderr to a log: exp ... file=/tmp/oracle_pipe 2>exp_err.log &, then check exp_err.log for the error.

If gzip completes but the import (imp file=full_exp.dmp.gz) fails with "not a valid export file," the gzip decompression must happen before imp reads the file: gunzip full_exp.dmp.gz first, then imp file=full_exp.dmp.

References

Posts in this series