☕ Java

Buffered Streams

Buffered streams wrap an underlying unbuffered stream with an in-memory buffer, dramatically reducing the number of native I/O system calls by batching reads and writes. Without buffering, each read() or write() call typically results in one OS system call, which transitions the CPU between user mode and kernel mode — an operation that costs thousands of CPU cycles. With buffering, data is read from or written to the OS in large chunks (typically 8192 bytes by default), and individual application read() and write() calls are served from memory, requiring no system call unless the buffer fills or empties. Java provides four buffered stream classes: BufferedInputStream and BufferedOutputStream for byte-level I/O, and BufferedReader and BufferedWriter for character-level I/O. All four wrap an existing stream, are transparent to the application (the same read/write API), and dramatically improve performance for I/O-intensive code. This entry covers the internal buffer mechanics, buffer sizing guidance, the flush contract (when buffered data is actually written), the decorator pattern that makes buffering composable, mark/reset functionality in BufferedInputStream, and the readLine() convenience in BufferedReader.

Buffer Mechanics — How Buffering Reduces System Calls

Every Java I/O stream ultimately communicates with the OS through system calls. A read() on an unbuffered FileInputStream triggers a read() system call that transitions to kernel mode, retrieves data from disk or network, and returns to user mode — a round trip that costs thousands of nanoseconds even for a single byte. Writing one byte at a time to an unbuffered FileOutputStream produces one system call per byte, turning a 1MB write into one million system calls — completely impractical for any performance-sensitive application. Buffered streams interpose an in-memory byte array between the application and the OS. On a BufferedInputStream.read(), if the buffer is non-empty, the byte is taken from the buffer at memory speed with no system call. Only when the buffer is empty does the stream make one native read() call to refill the entire buffer from the underlying stream. The default buffer size of 8192 bytes means that for sequentially reading a file, the number of system calls is reduced by a factor of 8192 compared to unbuffered reading — one system call per 8KB instead of one per byte. On the write side, BufferedOutputStream.write(byte) adds the byte to the buffer. Only when the buffer fills, or when flush() is called, or when the stream is closed, does the stream make a native write() system call with the entire buffer contents. For writing many small values (like writing JSON character by character), this batching can reduce system calls by orders of magnitude. The buffer size can be customized: BufferedInputStream(InputStream in, int size) and BufferedOutputStream(OutputStream out, int size). The default 8192 bytes is appropriate for most use cases. Larger buffers (65536, 1MB) benefit sequential I/O of large files where the OS can perform efficient large sequential reads. Smaller buffers waste less memory when many streams are open simultaneously. The optimal size depends on the OS filesystem block size and the access pattern. For most networked I/O, the default is appropriate; for disk-bound large file processing, 64KB to 256KB buffers often improve throughput.
Java
// ── Performance comparison: unbuffered vs buffered ────────────────────
import java.io.*;
import java.nio.file.*;

// Writing 1MB one byte at a time WITHOUT buffering:
long start = System.nanoTime();
try (FileOutputStream fos = new FileOutputStream("test.bin")) {
    for (int i = 0; i < 1_000_000; i++) {
        fos.write(i & 0xFF);   // 1,000,000 system calls — catastrophic
    }
}
System.out.printf("Unbuffered: %.0fms%n", (System.nanoTime() - start) / 1e6);
// Typical: ~5000ms on SSD, much longer on HDD

// Writing 1MB one byte at a time WITH buffering:
start = System.nanoTime();
try (BufferedOutputStream bos = new BufferedOutputStream(
        new FileOutputStream("test_buffered.bin"))) {
    for (int i = 0; i < 1_000_000; i++) {
        bos.write(i & 0xFF);   // bytes go to 8192-byte buffer → ~122 system calls total
    }
}   // close() flushes remaining buffer — final system call
System.out.printf("Buffered:   %.0fms%n", (System.nanoTime() - start) / 1e6);
// Typical: ~5ms — ~1000x faster

// ── Default buffer size ───────────────────────────────────────────────
BufferedInputStream defaultBIS = new BufferedInputStream(new FileInputStream("test.bin"));
// Buffer is 8192 bytes — not exposed by public API, but documented in source

// ── Custom buffer size ────────────────────────────────────────────────
// 64KB buffer for large sequential file reads:
int BUFFER_SIZE = 65_536;
try (BufferedInputStream bigBuffer = new BufferedInputStream(
        new FileInputStream("large.bin"), BUFFER_SIZE)) {
    byte[] chunk = new byte[BUFFER_SIZE];
    int bytesRead;
    while ((bytesRead = bigBuffer.read(chunk)) != -1) {
        process(chunk, bytesRead);
    }
}

// Small buffer (1KB) for many small streams where memory is constrained:
try (BufferedInputStream small = new BufferedInputStream(
        new FileInputStream("small.txt"), 1024)) {
    // works correctly — just refills buffer more often
}

// ── System call visualization ─────────────────────────────────────────
// File: 100,000 bytes
// Unbuffered read():  100,000 system calls (read 1 byte each)
// Buffered read():    ~13 system calls    (read 8192 bytes each = ceil(100000/8192))
// Manual bulk read(): 1 system call       (read(byte[100000]) — one OS call)

// ── The decorator pattern: stacking buffering onto any stream ──────────
// BufferedInputStream wraps ANY InputStream:
InputStream network = socket.getInputStream();
BufferedInputStream bufferedNetwork = new BufferedInputStream(network);

// BufferedOutputStream wraps ANY OutputStream:
OutputStream compressor = new GZIPOutputStream(new FileOutputStream("out.gz"));
BufferedOutputStream bufferedCompressor = new BufferedOutputStream(compressor);
// Writing to bufferedCompressor: buffered → GZIP compressed → file

The Flush Contract and Stream Closing

The flush contract defines when buffered data actually reaches the underlying stream. A BufferedOutputStream (or BufferedWriter) holds data in memory until: the buffer fills (at which point it is automatically flushed to make room for new data), flush() is called explicitly, or the stream is closed via close() (which always flushes first). Between these events, the buffered data is invisible to the underlying stream, the file on disk, or the remote host on a network connection. This creates a potential data loss scenario: if a process terminates, crashes, or the stream is not properly closed, buffered data that was not flushed is lost. The mandatory pattern for avoiding this is try-with-resources: wrapping the outermost stream in try (BufferedOutputStream bos = ...) { ... } ensures that close() is called even if an exception is thrown, which flushes the buffer before closing. If try-with-resources is not used, the finally { bos.close(); } pattern provides the same guarantee. Never rely on garbage collection to close and flush streams — GC does not run predictably enough and stream finalization is deprecated. The flush()-then-continue pattern applies when the application needs to ensure that data is visible to a reader on the other end of a network connection (or another process reading a pipe) without closing the stream. After writing a complete protocol message, calling flush() ensures the OS sends it. For file I/O, flush() ensures the data has been handed to the OS (in the OS buffer); calling the underlying FileOutputStream.getFD().sync() additionally ensures the OS has written to the physical disk. When multiple buffered streams are stacked — for example, BufferedOutputStream wrapping GZIPOutputStream wrapping FileOutputStream — calling flush() or close() on the outermost stream propagates the operation through the chain: BufferedOutputStream flushes its buffer to GZIPOutputStream, which flushes its compressed data to FileOutputStream, which writes to the file. The innermost flush/close always happens last, which is the correct order.
Java
// ── Flush contract: data is NOT written until flush or close ─────────
File file = new File("output.txt");
BufferedOutputStream bos = new BufferedOutputStream(new FileOutputStream(file));
bos.write("Hello".getBytes());
// At this point, "Hello" is in the 8192-byte buffer — NOT on disk
// If the process crashes here, the file is empty

bos.flush();   // now "Hello" is handed to the OS — visible to other processes
bos.write(" World".getBytes());
bos.close();   // flush() + close on underlying stream — " World" now on disk

// ── try-with-resources: guaranteed flush and close ────────────────────
// CORRECT — always use try-with-resources for streams:
try (BufferedOutputStream safe = new BufferedOutputStream(
        new FileOutputStream("safe.bin"))) {
    for (int i = 0; i < 100_000; i++) {
        safe.write(i & 0xFF);
    }
}   // close() called automatically — flushes buffer — all 100,000 bytes written

// WRONG — exception before close() loses data:
BufferedOutputStream unsafe = new BufferedOutputStream(
        new FileOutputStream("unsafe.bin"));
for (int i = 0; i < 100_000; i++) {
    unsafe.write(i & 0xFF);
    if (i == 50_000) throw new RuntimeException("Crash!");   // buffer not flushed
}
unsafe.close();   // never reached — last ~50,000 bytes in buffer are lost

// ── Stacked streams: close/flush propagates through the chain ─────────
try (BufferedOutputStream bos2 = new BufferedOutputStream(
        new GZIPOutputStream(
            new FileOutputStream("data.gz")))) {

    bos2.write(largeData);
}
// close() chain: BufferedOutputStream.close()
//                → flush buffer to GZIPOutputStream
//                → GZIPOutputStream.close() (writes GZIP trailer)
//                → FileOutputStream.close() (closes file descriptor)

// ── flush() for network protocols ─────────────────────────────────────
try (Socket socket = new Socket("api.example.com", 80);
     BufferedOutputStream out = new BufferedOutputStream(socket.getOutputStream())) {

    String request = "GET / HTTP/1.0
Host: api.example.com

";
    out.write(request.getBytes());
    out.flush();   // critical: without this, the server never receives the request
                   // — it's still in the 8192-byte buffer on our end

    // Now read the response...
}

// ── sync() for durability guarantees ─────────────────────────────────
try (FileOutputStream fos = new FileOutputStream("critical.dat");
     BufferedOutputStream bos3 = new BufferedOutputStream(fos)) {

    bos3.write(criticalData);
    bos3.flush();          // hand data to OS kernel buffer
    fos.getFD().sync();    // force OS to write kernel buffer to physical disk
    // After sync(): data survives a power failure
}

mark/reset in BufferedInputStream and Composition Patterns

BufferedInputStream supports the mark/reset API that its parent InputStream declares. mark(int readlimit) marks the current position in the stream; reset() returns the stream position to the marked position. The readlimit argument specifies how many bytes can be read after mark() before the mark becomes invalid. BufferedInputStream maintains the mark by keeping the buffered data from the marked position in memory even after it would normally be discarded. If more than readlimit bytes are read, the buffer may be released and reset() will throw IOException ("Resetting to invalid mark"). The mark/reset capability is used for look-ahead parsing: read bytes to determine the format or content, then reset and re-read with the appropriate parser. It is also used by format detectors (determine if a stream is a PNG, JPEG, or GIF by reading the first few bytes, then reset and handle accordingly). BufferedInputStream is one of the few InputStream implementations where markSupported() returns true — raw FileInputStream and network streams do not support mark/reset. Composing streams effectively: the standard layering for high-performance file processing is BufferedInputStream(FileInputStream) for reading and BufferedOutputStream(FileOutputStream) for writing. For character I/O, BufferedReader(new InputStreamReader(new FileInputStream(path), charset)) correctly handles encoding while buffering. The outermost stream is always the one the application code interacts with; the innermost is the actual data source or sink; intermediate streams add transformations (decompression, encoding conversion, buffering, encryption). A critical composition mistake: wrapping a BufferedOutputStream inside another BufferedOutputStream creates double buffering — data is buffered twice, the inner buffer effectively acting as a large collection buffer that is flushed to the outer buffer which then flushes to the OS. This is wasteful but not incorrect. More harmful is wrapping a DataOutputStream or PrintWriter inside a BufferedOutputStream — these classes have their own internal write() calls that go to their underlying stream, and if that stream is already buffered, the layering is correct; if not, every write is unbuffered.
Java
// ── mark/reset in BufferedInputStream ────────────────────────────────
byte[] data = {0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A};  // PNG header
try (BufferedInputStream bis = new BufferedInputStream(
        new ByteArrayInputStream(data))) {

    // Check if markSupported() before using:
    System.out.println("Mark supported: " + bis.markSupported());  // true

    bis.mark(8);   // mark position 0, can read 8 bytes before mark invalid
    byte[] header = new byte[4];
    bis.read(header);   // read first 4 bytes: [0x89, 0x50, 0x4E, 0x47]

    String format = detectFormat(header);   // e.g., "PNG"
    System.out.println("Detected format: " + format);

    bis.reset();   // return to mark position — position is 0 again

    // Now re-read with proper parser for the detected format:
    if ("PNG".equals(format)) {
        parsePng(bis);   // reads from position 0 again
    }
}

// ── Format detection with mark/reset ─────────────────────────────────
public static String detectFormat(BufferedInputStream bis) throws IOException {
    if (!bis.markSupported()) throw new IllegalArgumentException("Stream must support mark");

    bis.mark(16);   // allow reading 16 bytes for detection
    byte[] magic = new byte[4];
    int read = bis.read(magic);
    bis.reset();    // always reset after detection

    if (read < 4) return "UNKNOWN";
    // PNG: 89 50 4E 47
    if (magic[0] == (byte)0x89 && magic[1] == 'P' && magic[2] == 'N' && magic[3] == 'G')
        return "PNG";
    // JPEG: FF D8 FF
    if (magic[0] == (byte)0xFF && magic[1] == (byte)0xD8 && magic[2] == (byte)0xFF)
        return "JPEG";
    // PDF: %PDF
    if (magic[0] == '%' && magic[1] == 'P' && magic[2] == 'D' && magic[3] == 'F')
        return "PDF";
    return "UNKNOWN";
}

// ── Standard composition patterns ────────────────────────────────────
// Byte file I/O — the standard pattern:
try (BufferedInputStream in   = new BufferedInputStream(new FileInputStream("in.bin"));
     BufferedOutputStream out = new BufferedOutputStream(new FileOutputStream("out.bin"))) {
    byte[] buffer = new byte[8192];
    int n;
    while ((n = in.read(buffer)) != -1) {
        out.write(buffer, 0, n);
    }
}

// Character file I/O with explicit encoding — the standard pattern:
try (BufferedReader reader = new BufferedReader(
        new InputStreamReader(new FileInputStream("text.txt"), StandardCharsets.UTF_8));
     BufferedWriter writer = new BufferedWriter(
        new OutputStreamWriter(new FileOutputStream("out.txt"), StandardCharsets.UTF_8))) {
    String line;
    while ((line = reader.readLine()) != null) {
        writer.write(line);
        writer.newLine();
    }
}

// With compression:
try (BufferedInputStream in = new BufferedInputStream(
        new GZIPInputStream(new FileInputStream("data.gz")));
     BufferedOutputStream out = new BufferedOutputStream(
        new FileOutputStream("decompressed.bin"))) {
    byte[] buf = new byte[65536];
    int n;
    while ((n = in.read(buf)) != -1) out.write(buf, 0, n);
}

Related Topics in Java I/O

BufferedReader
BufferedReader wraps any Reader with an in-memory character buffer, dramatically reducing system calls for character-by-character or line-by-line reading. Its defining method is readLine(), which reads a complete line of text terminated by \n, \r, or \r\n and returns it without the terminator, or returns null at end-of-file. Beyond buffering, BufferedReader also exposes a lines() method (Java 8+) that returns a lazy Stream<String> of lines, enabling the full Stream API for file processing without loading the entire file into memory. BufferedReader supports mark/reset with a caller-specified readAheadLimit. It is obtained either by wrapping a Reader (new BufferedReader(new FileReader(...))) or directly from Files.newBufferedReader(path, charset), which is the preferred idiom in modern Java. This entry covers construction and buffer sizing, all read methods and their contracts, readLine() edge cases (empty lines, last line without terminator), the lines() stream and its relationship to reader lifecycle, mark/reset semantics with readAheadLimit, and the use of BufferedReader as a base for protocol parsing.
BufferedWriter
BufferedWriter wraps any Writer with an in-memory character buffer, reducing system calls by accumulating characters until the buffer fills, flush() is called, or close() is called. It adds two capabilities not present in Writer: newLine(), which writes the platform-specific line separator, and an optimized write(String, int, int) that avoids creating a char[] copy by writing directly from the String. BufferedWriter is the standard output partner to BufferedReader — together they provide efficient line-by-line text file processing. It is constructed either by wrapping a Writer (new BufferedWriter(new FileWriter(...))) or via Files.newBufferedWriter(path, charset, options), the modern idiomatic alternative. Like all buffered streams, correct usage requires try-with-resources to guarantee that buffered data is flushed and the file is closed even when exceptions occur. This entry covers construction and buffer sizing, all write methods and their interaction with the buffer, newLine() and its platform behavior, flush semantics including when explicit flush is necessary, the difference between close() and flush(), and performance patterns for high-throughput text writing.
PrintWriter
PrintWriter is a character-based output class that wraps any Writer or OutputStream and adds convenience methods for printing formatted representations of all Java primitive types, strings, and objects. Its defining characteristic is that none of its print(), println(), and printf() methods throw checked IOException — errors are silently swallowed and can only be detected after the fact by calling checkError(). This makes PrintWriter easy to use interactively and in situations where I/O failure is genuinely unrecoverable (writing to System.out, generating diagnostic output), but makes it dangerous for critical data writing where exceptions must be caught and handled. PrintWriter can auto-flush on println(), printf(), and format() calls when constructed with autoFlush=true, which is useful for interactive console output and network protocol streams. Its printf() and format() methods delegate to java.util.Formatter, enabling C-style formatted output with full locale awareness. This entry covers all constructor variants and their autoFlush and buffering behavior, every print/println/printf method, the checkError() error detection model, the difference between PrintWriter and PrintStream, charset handling, and when PrintWriter is the right choice versus BufferedWriter.
DataInputStream
DataInputStream wraps any InputStream and adds methods for reading Java primitive types in a machine-independent binary format. It reads boolean, byte, short, int, long, float, double, and char values from the underlying stream using fixed byte widths and big-endian byte order. The big-endian, fixed-width encoding is identical to the format written by DataOutputStream, making the two classes the natural pair for serializing and deserializing primitive data across files, network connections, or inter-process pipes. DataInputStream also provides readFully(), which blocks until exactly the specified number of bytes have been read — filling a buffer completely rather than returning a partial read as InputStream.read() may do. readUTF() reads a string encoded in a modified UTF-8 format (a two-byte length prefix followed by the encoded string bytes) that was written by DataOutputStream.writeUTF(). DataInputStream is unbuffered, so it should always be wrapped inside a BufferedInputStream for performance. This entry covers the full method API, the big-endian byte order contract, readFully() vs read() semantics, the modified UTF-8 format and its limitations, end-of-file detection, and composition patterns for binary protocol parsing.