Oracle Full Database Export Using a Named Pipe and Gzip Compression in KSH

Oracle Full Database Export Using a Named Pipe and Gzip Compression in KSH

Exporting a full Oracle database can produce very large dump files. This script solves that problem by streaming the export directly through a Linux named pipe into gzip, compressing the output on the fly — without ever writing an uncompressed file to disk.

What This Script Does

This KSH shell script performs a full Oracle database export (exp) and compresses the output simultaneously using gzip and a Linux named pipe (FIFO). It is designed to run as a background or nohup job on a Unix/Linux server where Oracle is installed.

The Script

1#!/usr/bin/ksh
2export ORAENV_ASK=NO
3export ORACLE_SID=scr10
4. /usr/local/bin/oraenv
5rm andypipe 2>/dev/null
6mknod andypipe p
7gzip < andypipe > full_scr10.dmp.gz &
8exp / full=y indexes=N grants=N statistics=none file=andypipe log=full_scr10.log
9rm andypipe 2>/dev/null

To make the script executable:

1chmod 700 <file>

To run it in the background so it survives logout:

1nohup ./<file> &

Line-by-Line Breakdown

Shebang — Set the Shell

1#!/usr/bin/ksh

This tells the operating system to run the script using the Korn Shell (KSH). KSH is a standard shell used in many Unix/Linux Oracle environments and supports the features needed for Oracle DBA scripting.

Set the Oracle Environment

1export ORAENV_ASK=NO
2export ORACLE_SID=scr10
3. /usr/local/bin/oraenv

The dot (.) before /usr/local/bin/oraenv is critical — it sources the script in the current shell session, so the environment variables are available to all commands that follow.

Clean Up Any Old Named Pipe

1rm andypipe 2>/dev/null

This removes any leftover named pipe file from a previous run. The 2>/dev/null part silences the error message if the file does not exist — this is clean, defensive scripting that prevents the script from failing on a fresh run.

Create the Named Pipe (FIFO)

1mknod andypipe p

mknod creates a named pipe (also called a FIFO — First In, First Out). The p argument tells the system this is a pipe, not a regular file. A named pipe is a special file in the filesystem that lets two separate processes communicate in real time — one writes data in, the other reads data out — without storing anything on disk in between.

Start gzip Reading From the Pipe

1gzip < andypipe > full_scr10.dmp.gz &

This line is started first and sent to the background because the named pipe blocks until both a reader and a writer are connected. gzip waits here as the reader, ready to compress whatever comes through andypipe.

Run the Oracle Export

1exp / full=y indexes=N grants=N statistics=none file=andypipe log=full_scr10.log

The exp utility writes the dump data into andypipe. At the same time, the background gzip process reads that data from andypipe and compresses it into full_scr10.dmp.gz. The two processes run in parallel — data flows through the pipe, never fully landing on disk uncompressed.

Clean Up the Named Pipe

1rm andypipe 2>/dev/null

Once the export finishes, this removes the named pipe. Leaving it behind could cause confusion on the next run (which is why we also remove it at the top of the script). Again, 2>/dev/null safely suppresses any error.

Key Concepts and Insights

Why Use a Named Pipe?

A named pipe allows two programs to be piped together by name without a real intermediate file. This is essential here because the Oracle exp utility writes to a file= parameter — it does not natively support stdout piping. By pointing file= at a named pipe, we trick exp into writing through the pipe, where gzip can pick it up in real time.

Disk Space Savings

Because the dump is compressed on the fly, you only need disk space for the .gz output — not the full uncompressed dump. Oracle full exports can easily be 10x–50x larger before compression. This approach makes exporting large databases practical on systems with limited storage.

OS Authentication With /

The exp / syntax uses OS-level authentication (SYSDBA privilege via the dba OS group). This means no password is stored in the script, which is important for security in automated jobs.

The & Background Operator Is Critical

The gzip < andypipe ... & line must run first and must be backgrounded. A named pipe blocks until both ends are open. If you ran exp first, the script would hang indefinitely waiting for a reader to open the pipe.

indexes=N, grants=N, statistics=none — Why?

These are common best practices for migration or refresh exports:

• Indexes are faster to rebuild at import time via indexes=Y on imp, or rebuilt automatically — avoids bloating the dump.
• Grants may not apply in the target environment.
• Statistics imported from a different database are often misleading — it is better to gather fresh statistics after import.

Legacy exp vs. Data Pump

The exp utility is the original Oracle export tool, available since early Oracle versions. Oracle recommends using Data Pump (expdp) for Oracle 10g and newer databases. However, exp is still used in legacy environments and remains the only option for exporting to older Oracle versions. Named-pipe compression with exp is a well-proven DBA technique where Data Pump's built-in compression or parallelism is not available.

How to Run This Script

1. Save the script to a file, for example exp_full_scr10.ksh
2. Make it executable:

1chmod 700 exp_full_scr10.ksh

3. Run it in the background so it survives terminal logout:

1nohup ./exp_full_scr10.ksh > nohup_exp.out 2>&1 &

4. Monitor the export log:

1tail -f full_scr10.log

5. After completion, verify the compressed dump exists:

1ls -lh full_scr10.dmp.gz

Quick Reference Checklist

• Replace scr10 with your actual ORACLE_SID
• Confirm /usr/local/bin/oraenv is the correct path on your system
• Ensure the OS user running the script is in the dba OS group (for / authentication)
• Confirm you have write permission in the directory where the script runs
• Always check full_scr10.log after the run to confirm no export errors

References

• Oracle Original Export Utility (exp) — Oracle 19c Documentation — Official Oracle documentation covering the legacy exp utility, its parameters, and usage guidelines.
• Oracle Export Utility — Oracle 9.2 Classic Reference — Classic Oracle 9i export utility reference covering all exp parameters including full, indexes, grants, and statistics.
• Oracle Data Pump Export (expdp) — Oracle 19c Documentation — Official documentation for the modern replacement to exp, recommended for Oracle 10g and later.
• Named Pipe — Wikipedia — Explains how named pipes (FIFOs) work in Unix/Linux, including how mknod creates them and how two separate processes communicate through them.
• Correct Use of gzip With Named Pipes — Stack Overflow — Community discussion on using gzip with named pipes on Linux, including common pitfalls.
• How to Use nohup with a Shell Script — Oracle Forums — Oracle community thread covering how to run shell scripts including export scripts with nohup to keep them running after logout.
• FIFO / Named Pipe in Linux — Engineers Garage — Practical guide to creating and using FIFO named pipes with mknod and mkfifo on Linux, including a compression example.
• How to Full DB Export/Import — Ask TOM (Oracle) — Tom Kyte's discussion on full database export and import best practices, covering exp full=y and related tips.