☕ Java
Batch Processing
JDBC batch processing groups multiple SQL statements together so they can be sent to the database and executed in a single network round trip, rather than incurring the latency cost of one round trip per statement. Both Statement and PreparedStatement expose the same addBatch()/executeBatch() mechanism, making batching applicable both to varying SQL strings and to repeated execution of one parameterized statement with different bound values. This entry covers the batching API itself, combining batching with transactions for bulk operations, and practical considerations around batch sizing, error handling, and driver-specific rewriting behavior.
The addBatch/executeBatch Mechanism
Without batching, executing N similar statements means N separate network round trips between the application and the database, each carrying its own protocol overhead even when the statements are nearly identical — for bulk operations involving thousands of rows, this overhead dominates the total time far more than the actual work the database does per statement. addBatch() queues a statement (for Statement, a full SQL string; for PreparedStatement, the currently-bound parameter values against the SQL that was already supplied when the statement was prepared) into an in-memory batch held by the driver, without sending anything to the database yet. Calling executeBatch() sends every queued entry to the database together and returns an int[] where each element reports the update count for the corresponding queued statement, in the same order they were added.
After executeBatch() returns, the batch is automatically considered consumed by most drivers, but calling clearBatch() explicitly is good practice whenever a Statement or PreparedStatement object will be reused for an unrelated subsequent batch, since otherwise there's a risk of statements lingering and being unexpectedly re-executed. Each element of the returned int[] is ordinarily a non-negative row count, but JDBC also defines two special values a driver may return instead: Statement.SUCCESS_NO_INFO, meaning the statement succeeded but the driver can't or won't report how many rows were affected, and Statement.EXECUTE_FAILED, meaning that specific statement failed — relevant mainly for drivers that continue executing the remaining batch entries after one fails rather than aborting the whole batch immediately, which is itself a point of vendor-specific behavior worth verifying rather than assuming.
Java
import java.sql.*;
public class BatchMechanism {
public static void batchWithPreparedStatement(Connection conn, java.util.List<String> names)
throws SQLException {
String sql = "INSERT INTO tags (name) VALUES (?)";
try (PreparedStatement ps = conn.prepareStatement(sql)) {
for (String name : names) {
ps.setString(1, name);
ps.addBatch(); // queues the currently-bound value; SQL text stays fixed
}
int[] results = ps.executeBatch();
for (int i = 0; i < results.length; i++) {
if (results[i] == Statement.EXECUTE_FAILED) {
System.err.println("Statement " + i + " failed");
} else if (results[i] == Statement.SUCCESS_NO_INFO) {
System.out.println("Statement " + i + " succeeded (row count unknown)");
} else {
System.out.println("Statement " + i + " affected " + results[i] + " row(s)");
}
}
ps.clearBatch(); // defensive — avoids re-queuing if this ps is reused
}
}
public static void batchWithPlainStatement(Connection conn) throws SQLException {
try (Statement stmt = conn.createStatement()) {
stmt.addBatch("UPDATE inventory SET status = 'reviewed' WHERE category = 'electronics'");
stmt.addBatch("UPDATE inventory SET status = 'reviewed' WHERE category = 'books'");
stmt.addBatch("DELETE FROM stale_flags WHERE created_at < NOW() - INTERVAL '30 days'");
// Plain Statement batching is fine for distinct, static statements —
// but note each entry here is a full SQL string, unlike PreparedStatement's
// single-SQL-text-plus-many-parameter-sets model.
int[] results = stmt.executeBatch();
}
}
}Combining Batching with Transactions for Bulk Operations
Batching and transactions solve related but distinct problems: batching reduces network round trips, while a transaction ensures a group of statements succeeds or fails together atomically. For genuine bulk operations — importing a large dataset, syncing records from another system — the two are almost always used together: auto-commit is disabled, statements are queued via addBatch() and periodically flushed via executeBatch(), and the whole operation is committed once at the end (or rolled back entirely if any part fails), rather than committing after every individual flush, which would partially defeat the purpose of batching by reintroducing a transactional commit's overhead at high frequency.
A practical detail that matters at scale is batch sizing: queuing an extremely large number of statements before calling executeBatch() (say, hundreds of thousands at once) can consume significant client-side and driver-side memory and, on some drivers, doesn't necessarily perform better than periodically flushing smaller chunks — a common pattern flushes every few hundred to a few thousand queued statements rather than accumulating an entire dataset's worth before the first executeBatch() call. If a BatchUpdateException is thrown partway through a flush, the transaction (if one is open) should generally be rolled back rather than partially committed, since the application typically cannot easily determine from getUpdateCounts() alone whether it's safe to consider the batch partially successful — though this depends on whether the goal is all-or-nothing correctness or best-effort processing with per-row error tracking, which calls for a different design (e.g. catching and logging per-row failures by falling back to individual execution for a failed chunk) rather than a single large all-or-nothing batch.
Java
import java.sql.*;
import java.util.List;
public class BulkImportWithBatchAndTransaction {
private static final int FLUSH_EVERY = 500;
public static void importRecords(Connection conn, List<String[]> rows) throws SQLException {
String sql = "INSERT INTO imported_records (external_id, payload) VALUES (?, ?)";
boolean originalAutoCommit = conn.getAutoCommit();
conn.setAutoCommit(false);
try (PreparedStatement ps = conn.prepareStatement(sql)) {
int queued = 0;
for (String[] row : rows) {
ps.setString(1, row[0]);
ps.setString(2, row[1]);
ps.addBatch();
queued++;
if (queued % FLUSH_EVERY == 0) {
ps.executeBatch();
ps.clearBatch();
// NOTE: no commit() here — the whole import is one transaction;
// flushing batches periodically only manages client-side memory
}
}
ps.executeBatch(); // flush any remainder under the last FLUSH_EVERY boundary
conn.commit(); // entire import becomes permanent atomically
} catch (SQLException e) {
conn.rollback(); // undo the entire import, including already-flushed batches
throw e;
} finally {
conn.setAutoCommit(originalAutoCommit);
}
}
}Driver-Specific Rewriting and Real-World Performance Considerations
Whether batching actually delivers a large performance improvement, and how large, depends substantially on the specific driver and database involved, because the JDBC batch API itself only guarantees a single logical call from the application's point of view — what the driver does underneath is left as an implementation detail. Some drivers take advantage of this by rewriting a batch of structurally identical INSERT statements into a single multi-row INSERT statement before sending it to the database, which can produce dramatic speedups over sending each insert as a separate statement even within one network round trip; this behavior is often opt-in rather than automatic, configured via a driver-specific connection property (for example, MySQL Connector/J's rewriteBatchedStatements=true, or PostgreSQL's pgjdbc reWriteBatchedInserts=true).
Because this kind of rewriting is a driver-specific optimization rather than something the JDBC specification mandates, relying on a particular performance characteristic of batching should be validated against the actual driver and database version in use rather than assumed from general JDBC knowledge — what produces a large speedup on one driver/database pairing may produce a much smaller one, or even none, on another. Beyond raw insert/update throughput, it's also worth remembering that batching changes nothing about the SQL injection considerations covered under PreparedStatement: batching with Statement and string-concatenated SQL is exactly as vulnerable as non-batched Statement usage, so batching should not be treated as a substitute for using bound parameters when any part of the batched SQL varies based on untrusted input.
Java
// ── Enabling driver-specific batch rewriting via the JDBC URL ──────────────
// MySQL Connector/J: rewrites a batch of INSERTs into one multi-row INSERT
String mysqlUrl = "jdbc:mysql://localhost:3306/storedb?rewriteBatchedStatements=true";
// PostgreSQL pgjdbc: similar optimization, opt-in via its own property name
String pgUrl = "jdbc:postgresql://localhost:5432/storedb?reWriteBatchedInserts=true";
// ── Without rewriting, illustratively, this might be N round trips: ───────
// INSERT INTO t (a, b) VALUES (1, 'x');
// INSERT INTO t (a, b) VALUES (2, 'y');
// INSERT INTO t (a, b) VALUES (3, 'z');
// ── With rewriting enabled, the driver may send something closer to: ──────
// INSERT INTO t (a, b) VALUES (1, 'x'), (2, 'y'), (3, 'z');
// ── as a single statement — but this is a driver implementation detail,
// not something the JDBC specification guarantees, and behavior should be
// verified against the specific driver version in use rather than assumed.
import java.sql.*;
public class BatchRewriteAwareInsert {
public static void insertMany(Connection conn, java.util.List<Object[]> rows) throws SQLException {
// Application code is unchanged regardless of whether the driver rewrites
// the batch under the hood — the rewriting is transparent to this method.
String sql = "INSERT INTO t (a, b) VALUES (?, ?)";
try (PreparedStatement ps = conn.prepareStatement(sql)) {
for (Object[] row : rows) {
ps.setObject(1, row[0]);
ps.setObject(2, row[1]);
ps.addBatch();
}
ps.executeBatch();
}
}
}Related Topics in Java Database Connectivity (JDBC)
JDBC Architecture
JDBC (Java Database Connectivity) is the standard Java API for connecting to and executing operations against relational databases, defined as a set of interfaces in java.sql and javax.sql that database vendors implement with concrete driver classes. Its architecture is deliberately layered to decouple application code from any specific database product: applications code against interfaces like Connection, Statement, and ResultSet, while a pluggable driver underneath translates those calls into the vendor-specific network protocol the database actually understands. This entry covers the two-layer (API/driver) architecture, the four historical driver types, the role of the JDBC driver manager and connection pooling layers, and how the pieces fit together at runtime.
JDBC Drivers
A JDBC driver is the vendor-specific library that implements the java.sql/javax.sql interfaces for a particular database product, translating standard JDBC calls into that database's native network protocol. Choosing, loading, registering, and configuring the correct driver is a prerequisite for any JDBC code to run, and the mechanics of how a driver gets discovered and instantiated have evolved significantly since JDBC's introduction. This entry covers driver registration via the Service Provider Interface (SPI) versus the older explicit Class.forName() approach, JDBC URL syntax and structure, driver-specific configuration properties, and how to choose and manage driver dependencies in a real project.
DriverManager
java.sql.DriverManager is the original, classic entry point into JDBC: a static factory class that maintains a registry of available Driver implementations and hands back a live Connection when given a JDBC URL and credentials. It predates the javax.sql.DataSource abstraction and connection pooling, and while it remains fully functional and is still commonly used for simple applications, scripts, and tests, production server-side code generally prefers DataSource-based connection pooling for the reasons covered below. This entry covers how DriverManager locates and selects a driver, its core methods, login timeout and logging configuration, and where it fits (and doesn't fit) in modern application architecture.
Connection
java.sql.Connection represents a single session with a specific database — the live link through which all statements, transactions, and metadata queries flow, and the factory from which Statement, PreparedStatement, and CallableStatement objects are created. A Connection is a relatively expensive, stateful resource that wraps an underlying network socket and server-side session, which is why its lifecycle, transaction settings, and proper closing matter so much in practice. This entry covers obtaining and closing connections, transaction control (auto-commit, commit, rollback, savepoints, isolation levels), creating statements from a connection, and connection metadata and health checks.