☕ JavaJava NIO

Buffers

A Buffer in Java NIO is a fixed-capacity, typed container for data that serves as the intermediary between application code and channels. Every I/O operation in NIO reads data into a buffer or writes data from a buffer — there is no byte-at-a-time API. The Buffer class hierarchy provides typed buffers for every Java primitive: ByteBuffer, CharBuffer, ShortBuffer, IntBuffer, LongBuffer, FloatBuffer, and DoubleBuffer. ByteBuffer is the foundation type because channels always operate in bytes; the other types are views over a ByteBuffer or are used for in-memory data processing. Every buffer has three state variables — position, limit, and capacity — that control where reads and writes occur and how much data is available. The flip(), clear(), compact(), rewind(), and mark()/reset() operations manipulate these state variables to implement the write-then-read and read-then-write patterns that buffer-based I/O requires. ByteBuffer can be backed by a Java heap array (allocate()) or by off-heap native memory (allocateDirect()), with significant performance implications for I/O operations. This entry covers all buffer state variables and their transitions, every buffer method in depth, heap vs direct buffer performance model, view buffers and byte order, the slice() and duplicate() operations, and the common patterns for buffer-based channel I/O.

Buffer State — position, limit, capacity, and the Core Operations

Every Buffer has exactly three state variables that fully determine what can be read from it and what can be written to it. capacity is the total number of elements the buffer can hold — it is set at creation and never changes. limit is the index of the first element that cannot be read or written — in write mode (just after clear() or at creation), limit equals capacity; in read mode (after flip()), limit equals the number of elements written. position is the index of the next element to be read or written — it advances automatically with each get() or put() call. The invariant is always: 0 ≤ position ≤ limit ≤ capacity. Violating this invariant (by setting position > limit, or limit > capacity) throws IllegalArgumentException from the setter methods. The remaining() method returns limit - position — the number of elements that can be read (after flip()) or written (before flip()) without exceeding the limit. hasRemaining() returns remaining() > 0. flip() is the operation that transitions a buffer from write mode to read mode. It sets limit = position and position = 0. After flip(), position points to the first element that was written, and limit points one past the last element written. Everything between position and limit is the data that was just written and can now be read. clear() is the operation that transitions a buffer from read mode back to write mode for overwriting. It sets position = 0 and limit = capacity, discarding any information about what was previously in the buffer. The data is not erased — it is still in the array — but it is considered undefined because any subsequent writes will overwrite it. compact() is the operation that transitions a buffer from read mode back to write mode without discarding unread data. It copies the bytes between position and limit to the start of the buffer, sets position = (old limit - old position), and limit = capacity. This preserves partial reads — if a channel read produced a partial record that spans two reads, compact() keeps the unprocessed part and allows the next channel read to append more data immediately after it. rewind() sets position = 0 without changing limit — it prepares for re-reading data that was already read (or for writing from the start of the same data). mark() saves the current position; reset() restores position to the saved mark.

Java

// ── Buffer construction ───────────────────────────────────────────────
ByteBuffer heapBuf   = ByteBuffer.allocate(1024);         // heap, zeroed
ByteBuffer directBuf = ByteBuffer.allocateDirect(1024);   // off-heap, zeroed
ByteBuffer wrappedBuf = ByteBuffer.wrap(new byte[]{1, 2, 3, 4, 5}); // existing array

System.out.println("Capacity: " + heapBuf.capacity());   // 1024
System.out.println("Limit:    " + heapBuf.limit());      // 1024 (= capacity in write mode)
System.out.println("Position: " + heapBuf.position());   // 0
System.out.println("Remaining:" + heapBuf.remaining());  // 1024 (= limit - position)

// ── Write mode: position advances with each put() ────────────────────
ByteBuffer buf = ByteBuffer.allocate(16);

buf.putInt(42);          // writes 4 bytes; position = 4
buf.putLong(1234567L);   // writes 8 bytes; position = 12
buf.putInt(99);          // writes 4 bytes; position = 16

System.out.println("After writes — position=" + buf.position() +
                   " limit=" + buf.limit());  // position=16, limit=16

// ── flip() — switch from write mode to read mode ──────────────────────
buf.flip();
// BEFORE flip: position=16, limit=16
// AFTER flip:  position=0,  limit=16 (limit was set to old position)
System.out.println("After flip — position=" + buf.position() +
                   " limit=" + buf.limit());   // position=0, limit=16

// Read back in same order:
int value1 = buf.getInt();    // 42;        position = 4
long value2 = buf.getLong();  // 1234567L;  position = 12
int value3 = buf.getInt();    // 99;        position = 16
System.out.println(value1 + " " + value2 + " " + value3);
System.out.println("Remaining: " + buf.remaining());  // 0 — all consumed

// ── clear() — reset for overwriting ──────────────────────────────────
buf.clear();
// position=0, limit=16 (= capacity) — ready to write again
// Data is NOT erased — still in the backing array — just logically "undefined"
System.out.println("After clear — position=" + buf.position() +
                   " limit=" + buf.limit());

// ── compact() — preserve unread data, write more ─────────────────────
ByteBuffer partial = ByteBuffer.allocate(8);
partial.put(new byte[]{1, 2, 3, 4, 5, 6});  // write 6 bytes; position=6
partial.flip();                               // flip: position=0, limit=6
partial.getInt();                             // read 4 bytes; position=4

// Now: position=4, limit=6 — bytes [4,5] not yet read
partial.compact();
// compact copies remaining 2 bytes ([4,5] at indices 4,5) to indices 0,1
// position = 2 (number of bytes compacted)
// limit = capacity = 8
// Can now write 6 more bytes starting at position 2
System.out.println("After compact — position=" + partial.position() +
                   " limit=" + partial.limit());  // position=2, limit=8

// ── rewind() — re-read from start ────────────────────────────────────
ByteBuffer rewindBuf = ByteBuffer.allocate(8);
rewindBuf.putInt(111).putInt(222);
rewindBuf.flip();
System.out.println(rewindBuf.getInt());  // 111
rewindBuf.rewind();                      // position=0, limit unchanged (4 wait — limit is 8)
// Actually flip() set limit=8 (position was 8 after two putInts)
// rewind() sets position=0
System.out.println(rewindBuf.getInt());  // 111 again

// ── mark() and reset() ────────────────────────────────────────────────
ByteBuffer markBuf = ByteBuffer.allocate(16);
markBuf.put(new byte[]{1,2,3,4,5,6,7,8}).flip();

markBuf.getInt();    // read 4 bytes; position=4
markBuf.mark();      // mark position 4
markBuf.getInt();    // read 4 more; position=8
markBuf.reset();     // restore position to 4
System.out.println(markBuf.getInt());   // 5 — re-reads the marked position

ByteBuffer put/get Methods, Byte Order, and View Buffers

ByteBuffer has two families of put() and get() methods: relative methods that use and advance the current position, and absolute methods that access a specific index without changing position. The relative family: get() returns the byte at position and increments position; get(byte[] dst) reads dst.length bytes into dst; get(byte[] dst, int offset, int length) reads length bytes into dst starting at offset. get(int index) returns the byte at index without changing position. The same pattern applies to getInt(), getLong(), getDouble(), etc. for reading multi-byte primitive types. All relative methods throw BufferUnderflowException if remaining() < sizeof(type); absolute methods throw IndexOutOfBoundsException if index is out of bounds. Byte order determines how multi-byte values are interpreted. ByteBuffer.order(ByteOrder) sets the byte order for all subsequent multi-byte operations. The two byte orders are ByteOrder.BIG_ENDIAN (most-significant byte first — the Java default, also called "network byte order") and ByteOrder.LITTLE_ENDIAN (least-significant byte first — the x86 native byte order, used by most binary file formats). Setting byte order before reading or writing multi-byte types is essential for interoperability with C programs, binary file formats, and network protocols that specify their byte order. The default ByteOrder is BIG_ENDIAN regardless of the host architecture — this is correct for Java-to-Java communication but wrong for most binary formats (which are typically little-endian for Windows and x86-native formats). View buffers are typed buffers that share the same memory as a ByteBuffer but access it as a different type. ByteBuffer.asIntBuffer() returns an IntBuffer whose get() and put() read and write int values from the underlying ByteBuffer's memory, using the ByteBuffer's current byte order. asLongBuffer(), asDoubleBuffer(), asFloatBuffer(), asShortBuffer(), asCharBuffer(), and asReadOnlyBuffer() create the corresponding views. A view buffer's position and limit are in units of its element type (integers, longs, etc.), not bytes. The view buffer shares the ByteBuffer's backing memory: writes to the view are immediately visible in the ByteBuffer and vice versa. The view buffer's position maps to the ByteBuffer's current position at the time the view was created. slice() creates a new ByteBuffer that shares the current ByteBuffer's backing array from position to limit. The new buffer's position is 0 and its capacity and limit equal the original's remaining(). Modifications to either buffer are reflected in the other. slice() is useful for partitioning a buffer into independent regions. duplicate() creates a new ByteBuffer with its own independent position, limit, and mark, but sharing the backing array. It is used when multiple readers need to traverse the same data independently. asReadOnlyBuffer() creates a read-only view that shares the backing data but throws ReadOnlyBufferException on any put().

Java

// ── Relative vs absolute get/put ─────────────────────────────────────
ByteBuffer buf = ByteBuffer.allocate(16);

// Relative put — advances position:
buf.put((byte) 10);       // position: 0→1
buf.putShort((short) 256); // position: 1→3
buf.putInt(70000);         // position: 3→7
buf.putLong(Long.MAX_VALUE); // position: 7→15
buf.put((byte) 42);        // position: 15→16

buf.flip();

// Relative get — advances position:
byte   b = buf.get();       // 10;   position: 0→1
short  s = buf.getShort();  // 256;  position: 1→3
int    i = buf.getInt();    // 70000; position: 3→7

// Absolute get — does NOT change position:
long lAtSeven = buf.getLong(7);   // Long.MAX_VALUE; position stays at 7
long lRelative = buf.getLong();   // Long.MAX_VALUE; position: 7→15

System.out.printf("b=%d s=%d i=%d l=%d%n", b, s, i, lRelative);

// ── Byte order — critical for interoperability ────────────────────────
ByteBuffer data = ByteBuffer.allocate(8);

// Default: BIG_ENDIAN (Java standard, network byte order):
data.putInt(0x01020304);   // bytes: [01, 02, 03, 04]
data.flip();
System.out.printf("Big-endian bytes: %02X %02X %02X %02X%n",
    data.get(0), data.get(1), data.get(2), data.get(3));
// 01 02 03 04

data.clear();
// LITTLE_ENDIAN (x86 native, most Windows/Intel binary formats):
data.order(ByteOrder.LITTLE_ENDIAN);
data.putInt(0x01020304);   // bytes: [04, 03, 02, 01]
data.flip();
System.out.printf("Little-endian bytes: %02X %02X %02X %02X%n",
    data.get(0), data.get(1), data.get(2), data.get(3));
// 04 03 02 01

// Read a little-endian binary file:
try (FileChannel fc = FileChannel.open(Path.of("windows-format.bin"))) {
    ByteBuffer le = ByteBuffer.allocateDirect(4096)
                              .order(ByteOrder.LITTLE_ENDIAN);
    fc.read(le);
    le.flip();
    int windowsInt = le.getInt();   // correctly reads as little-endian
    System.out.println("Windows int: " + windowsInt);
}

// ── View buffers ──────────────────────────────────────────────────────
ByteBuffer bytes = ByteBuffer.allocate(16).order(ByteOrder.BIG_ENDIAN);
bytes.putLong(0x0102030405060708L);
bytes.putLong(0x090A0B0C0D0E0F10L);
bytes.flip();

// IntBuffer view — reads 4-byte ints from the same memory:
IntBuffer ints = bytes.asIntBuffer();   // view of current position to limit
System.out.printf("As ints: %08X %08X %08X %08X%n",
    ints.get(), ints.get(), ints.get(), ints.get());
// 01020304, 05060708, 090A0B0C, 0D0E0F10

// LongBuffer view:
bytes.position(0);
LongBuffer longs = bytes.asLongBuffer();
System.out.printf("As longs: %016X %016X%n", longs.get(), longs.get());
// 0102030405060708, 090A0B0C0D0E0F10

// ── slice() — independent sub-buffer of the same memory ───────────────
ByteBuffer original = ByteBuffer.allocate(16);
for (int k = 0; k < 16; k++) original.put((byte) k);
original.flip();

original.position(4);
original.limit(12);
ByteBuffer sliced = original.slice();   // covers bytes 4..11 of original
System.out.println("Sliced capacity: " + sliced.capacity());  // 8
System.out.println("Sliced[0]: " + sliced.get(0));            // 4 (offset in original)
sliced.put(0, (byte) 99);   // modifies original too
System.out.println("Original[4]: " + original.get(4));        // 99

// ── duplicate() — independent position, shared data ───────────────────
ByteBuffer original2 = ByteBuffer.wrap(new byte[]{1,2,3,4,5,6,7,8});
ByteBuffer dup = original2.duplicate();  // same data, own position/limit

original2.getInt();   // advances original2's position to 4
System.out.println("Original pos: " + original2.position()); // 4
System.out.println("Dup pos:      " + dup.position());       // 0 (independent)

dup.put(0, (byte) 99);  // modifies original2's backing array too!
System.out.println("Original2[0]: " + original2.get(0));     // 99

// ── asReadOnlyBuffer() ────────────────────────────────────────────────
ByteBuffer writable = ByteBuffer.wrap(new byte[]{1,2,3,4});
ByteBuffer readOnly = writable.asReadOnlyBuffer();

readOnly.getInt();      // OK — reads 4 bytes
try {
    readOnly.putInt(42);   // ReadOnlyBufferException
} catch (ReadOnlyBufferException e) {
    System.out.println("Cannot write to read-only buffer");
}

Direct Buffers, Channel I/O Patterns, and Bulk Operations

Direct ByteBuffers (ByteBuffer.allocateDirect(n)) are allocated in native OS memory outside the Java heap. Their primary purpose is to enable I/O without data copying: when a heap ByteBuffer is used with a channel's read() or write(), the JVM internally copies the data to a temporary direct buffer before making the system call (because the OS cannot safely I/O directly to garbage-collected heap memory, which may move during GC). A direct buffer eliminates this copy — the channel can I/O directly to/from the buffer's native memory. For high-throughput I/O paths (network servers transferring gigabytes, file processing pipelines), direct buffers can reduce CPU usage by 30-50% by eliminating data copies. Direct buffer trade-offs: allocation is significantly slower than heap allocation (hundreds of microseconds vs nanoseconds), direct buffers don't benefit from JVM garbage collection (they are released when the ByteBuffer object is GCed and the Cleaner fires, which is unpredictable), and they consume native memory rather than heap memory, so they don't appear in the heap dump or standard memory profiling. Direct buffer memory pressure is not subject to the -Xmx limit; it uses the native process address space up to -XX:MaxDirectMemorySize (default equals -Xmx). Running out of direct memory throws OutOfMemoryError: Direct buffer memory, which is difficult to diagnose. The standard channel I/O pattern using ByteBuffer follows: allocate the buffer; call channel.read(buffer) which fills the buffer and returns the count (or -1 at end of stream); call buffer.flip() to switch to read mode; process the data between buffer.position() and buffer.limit(); call buffer.clear() or buffer.compact() for the next read. For writing: prepare the data by putting values into the buffer in write mode; call buffer.flip() to switch to read mode; call channel.write(buffer) which reads from the buffer; after write, buffer.compact() or buffer.clear() for the next preparation. A write loop must check buffer.hasRemaining() because channel.write() may write fewer bytes than available (for non-blocking channels or when the socket send buffer is full). hasArray() and array() allow access to the backing byte[] for heap buffers — this is safe and efficient when you need to pass the buffer's data to an API expecting a byte[]. For direct buffers, hasArray() returns false and array() throws UnsupportedOperationException. arrayOffset() gives the offset of position 0 within the backing array (relevant for wrapped buffers and slices of wrapped buffers).

Java

// ── Direct buffer: when and why ──────────────────────────────────────
// Long-lived, reusable buffers for high-throughput I/O — use direct:
ByteBuffer ioBuffer = ByteBuffer.allocateDirect(65536);  // 64KB direct buffer
// Allocate once, reuse for many read/write cycles

// Short-lived buffers or low-throughput code — heap is fine:
ByteBuffer parseBuffer = ByteBuffer.allocate(4096);  // heap — fast allocation, GC-managed

// Checking buffer type:
System.out.println(ioBuffer.isDirect());      // true
System.out.println(parseBuffer.isDirect());   // false

// Direct buffer in channel I/O (no internal copy):
try (FileChannel fc = FileChannel.open(Path.of("data.bin"))) {
    ByteBuffer direct = ByteBuffer.allocateDirect(65536);
    int n = fc.read(direct);   // JVM reads directly to native memory — no heap copy
    direct.flip();
    // Process direct.limit() bytes
}

// ── Standard read loop — the canonical NIO I/O pattern ───────────────
try (FileChannel fc = FileChannel.open(Path.of("data.bin"))) {
    ByteBuffer buf = ByteBuffer.allocateDirect(8192);

    while (true) {
        buf.clear();           // 1. Switch to write mode (position=0, limit=capacity)
        int n = fc.read(buf); // 2. Fill buffer from channel
        if (n == -1) break;   // 3. End of stream

        buf.flip();            // 4. Switch to read mode (limit=n, position=0)
        while (buf.hasRemaining()) {  // 5. Process all bytes
            byte b = buf.get();
            process(b);
        }
    }
}

// ── compact() for partial records spanning multiple reads ─────────────
try (SocketChannel sc = SocketChannel.open(new InetSocketAddress("host", 8080))) {
    ByteBuffer buf = ByteBuffer.allocateDirect(4096);
    int recordSize = 16;   // each record is 16 bytes

    while (true) {
        int n = sc.read(buf);   // fill buffer
        if (n == -1) break;     // connection closed

        buf.flip();
        // Process as many complete records as available:
        while (buf.remaining() >= recordSize) {
            processRecord(buf);   // reads exactly recordSize bytes
        }
        // compact() moves any partial record (< recordSize bytes) to buffer start:
        buf.compact();   // position = bytes remaining (0 to recordSize-1)
                         // limit = capacity (ready for more data from channel)
    }
}

// ── Standard write loop — ensure all bytes are written ────────────────
try (SocketChannel sc = SocketChannel.open(new InetSocketAddress("host", 80))) {
    ByteBuffer request = ByteBuffer.wrap(
        "GET / HTTP/1.0

".getBytes(StandardCharsets.US_ASCII));

    // Write loop: channel.write() may not write all bytes in one call
    while (request.hasRemaining()) {
        sc.write(request);   // writes from position to limit, advances position
    }
    // All bytes written when !request.hasRemaining() (position == limit)
}

// ── Bulk transfers between buffers ────────────────────────────────────
ByteBuffer src  = ByteBuffer.wrap(new byte[]{1,2,3,4,5,6,7,8});
ByteBuffer dst  = ByteBuffer.allocate(16);

// put(ByteBuffer) copies from src.position to src.limit into dst at dst.position:
dst.put(src);   // copies 8 bytes; src.position=8, dst.position=8

// get into byte array:
src.flip();
byte[] array = new byte[src.remaining()];
src.get(array);   // copies src[position..limit] into array
System.out.println(Arrays.toString(array));  // [1,2,3,4,5,6,7,8]

// ── hasArray() and array() — accessing backing array for heap buffers ──
ByteBuffer heapBuf2 = ByteBuffer.wrap(new byte[]{10, 20, 30, 40});
if (heapBuf2.hasArray()) {
    byte[] backing = heapBuf2.array();
    int offset      = heapBuf2.arrayOffset();  // offset of position 0 in backing
    // Safe to pass to API expecting byte[]:
    legacyMethod(backing, offset, heapBuf2.limit());
}
// Direct buffer: hasArray() = false, array() throws UnsupportedOperationException

Buffers

Buffer State — position, limit, capacity, and the Core Operations

ByteBuffer put/get Methods, Byte Order, and View Buffers

Direct Buffers, Channel I/O Patterns, and Bulk Operations

Related Topics in Java NIO