Opening Handles and Managing Connections

Purpose and Scope

This page explains how to think about database/sql handles and the underlying connections they manage. A database handle is the value application code normally keeps for the lifetime of a process, while a driver connection is the lower-level resource that communicates with a specific database server. The important design point is that application code should usually share a single sql.DB handle and let the package manage a pool of driver connections behind it. The handle is concurrency-safe, and operations such as queries, executions, prepares, and transactions borrow a connection only for the work that needs one.

The official database guide describes the default pool as suitable for most programs: when all current connections are busy, sql.DB can create another connection, and when an operation finishes, the connection can be returned to the idle pool. Tuning is therefore an advanced activity rather than the first step in using the package. The public knobs are DB.SetMaxOpenConns, DB.SetMaxIdleConns, connection lifetime and idle-time controls, and DB.Stats for observing the result. The selected repository sources show the contracts that make that behavior possible at the driver boundary: drivers open individual connections, optionally provide context-aware operations, validate pooled connections, reset sessions before reuse, and convert query arguments to driver values.

Sources: src/database/sql/driver/driver.go, src/database/sql/ctxutil.go, src/database/sql/convert.go

Relevant Source Files

• src/database/sql/closemu.go - Defines the internal closingMutex, a read-mostly synchronization primitive used for safe close coordination when operations may be active concurrently.
• src/database/sql/convert.go - Implements argument conversion from caller values to driver.NamedValue and driver.Value, including named-value checking, column conversion, and default parameter conversion.
• src/database/sql/ctxutil.go - Provides context-aware helper paths for prepare, exec, query, statement execution, statement query, and transaction begin operations, including fallback behavior for older drivers.
• src/database/sql/driver/driver.go - Defines the public driver interfaces that database/sql relies on for opening connections, connecting through connectors, preparing statements, executing queries, beginning transactions, pinging, resetting sessions, and validating pooled connections.
• src/database/sql/driver/types.go - Defines driver-side value conversion contracts such as Value, NamedValue, ValueConverter, and Valuer, plus standard converters used to normalize values crossing the package boundary.
• src/database/sql/internal/sql.go - Contains an internal ScanContext bridge shared by database/sql and database/sql/driver without exposing the underlying value directly to users.

Core Concepts

A sql.DB should be read as a handle to a database, not as a single network connection. That distinction matters because many database libraries in other ecosystems expose a connection object that can perform one operation at a time, while Go presents a safe shared handle that multiplexes operations over a pool. The driver contract reinforces this model: Driver.Open returns a new Conn, and its documentation says that the returned connection is used by only one goroutine at a time. The package-level pool, not the driver, is responsible for efficient idle reuse, so drivers do not need to maintain their own cached closed connections for normal performance.

Drivers can participate in handle creation through two related paths. The basic Driver interface opens a connection from a driver-specific string. The newer DriverContext path lets database/sql.DB call OpenConnector once, obtain a Connector, and then call that connector's Connect method when the pool needs a new connection. This distinction is useful because it lets a driver parse or validate the data source name once for a pool rather than repeatedly for each physical connection. The driver documentation also states that Connector.Connect and Driver.Open should not return ErrBadConn; that error is reserved for cases where an existing connection has become invalid during validation, reset, or query execution.

Sources: src/database/sql/driver/driver.go

Opening a Handle and Letting the Pool Work

In application code, opening a handle is usually a small setup step: import a driver for its registration side effect, call sql.Open or a connector-based open function with the driver name and data source name, then verify reachability with Ping or PingContext if the program needs startup validation. The handle itself does not imply that every future operation will use one fixed connection. Instead, each operation can acquire an available connection or cause the pool to create one, subject to configured limits. This means the handle can be stored in application state and reused by HTTP handlers, workers, or command handlers without wrapping it in another mutex.

The driver interface documentation explains why the pool is placed in database/sql instead of pushed into every driver. Driver.Open may return a cached connection, but the comment says that doing so is unnecessary because the SQL package maintains a pool of idle connections for efficient reuse. A driver connection returned from Open is only used by one goroutine at a time, so the package can enforce exclusive use while still allowing many goroutines to operate through the shared handle. That division keeps drivers focused on protocol behavior and lets the standard package provide common pooling semantics across databases.

Sources: src/database/sql/driver/driver.go

Connection Pool Controls

Most programs should begin with the defaults and add limits only after observing application behavior. DB.SetMaxOpenConns limits the total number of open connections; after the limit is reached, additional operations wait for another operation to release a connection. This can protect a database server, but it also makes database access behave like a semaphore, so a too-small limit can create queueing or even deadlock when code holds one connection while waiting for work that needs another. DB.SetMaxIdleConns controls how many unused connections the pool keeps available for future operations, which trades faster reuse against server-side connection cost.

The remaining public pool knobs address time rather than count. Connection maximum lifetime controls how long a connection may exist before it is retired, which can help with load balancers, server restarts, or database policies that expire sessions. Maximum idle time controls how long an idle connection remains available before being closed. DB.Stats is the companion API for these settings: it lets code observe wait counts, wait duration, open connection count, in-use count, idle count, and close counts attributable to lifetime or idle-time limits. Tuning should therefore be a feedback loop, not a guess made before the program has traffic.

Although the public pool methods are on sql.DB, the selected driver source shows how a connection is prepared for reuse. The driver package recommends that all Conn implementations provide Pinger, SessionResetter, and Validator. If a connection implements Validator, its IsValid method is called before the connection is returned to the pool. If an idle pooled connection implements SessionResetter, ResetSession is called before it is reused for another query. The documentation also notes a subtle case: if a connection is never returned to the pool but is immediately reused, reset still occurs before reuse, while validation on return does not.

Sources: src/database/sql/driver/driver.go

Context-Aware Operations and Driver Fallbacks

The package supports context-aware database work even when not every driver implements the newest optional interfaces. The helper functions in ctxutil.go first check for context-capable driver interfaces such as ConnPrepareContext, ExecerContext, QueryerContext, StmtExecContext, StmtQueryContext, and ConnBeginTx. When the driver provides one, the helper delegates directly to the context-aware method. That is the best path because the driver can interrupt network I/O, cancel a server-side operation, or translate context cancellation into the database protocol's cancellation mechanism.

When the driver only supports older interfaces, the helpers convert named arguments to positional driver values, check the context before invoking the non-context method, and perform defensive cleanup when cancellation is detected after setup. For example, prepare falls back to Conn.Prepare, but if the context is done immediately after a successful prepare, the statement is closed and the context error is returned. Transaction begin has a similar fallback: if the driver does not implement ConnBeginTx, unsupported non-default isolation levels or read-only requests produce errors, and if the context is canceled after a legacy begin succeeds, the transaction is rolled back before returning the cancellation error.

Sources: src/database/sql/ctxutil.go

Arguments, Named Parameters, and Value Conversion

Opening and pooling connections are only part of connection management; each borrowed connection also needs safely converted operation inputs. The conversion path in convert.go turns caller arguments for statement execution and queries into driver.NamedValue slices. It records ordinal positions, validates named parameter syntax, consults driver.NamedValueChecker when available on a statement or connection, falls back through column converters when present, and finally uses the default parameter converter. This sequence gives drivers a chance to accept custom values, remove per-query option arguments, or reject unsupported data before the call crosses the driver boundary.

The driver-side type definitions make the conversion contract explicit. A driver Value is limited to nil, driver-supported custom values, or standard values such as int64, float64, bool, []byte, string, and time.Time; cursor-like result values can also implement Rows. A NamedValue carries an optional name, a one-based ordinal, and the value. A Valuer lets a Go type convert itself into a driver value, and the standard converters in types.go provide consistent conversions such as boolean parsing and integer range checking. This keeps driver calls predictable even though application arguments may be ordinary Go values.

Sources: src/database/sql/convert.go, src/database/sql/driver/types.go

Close Coordination and Internal Boundaries

Connection handles are commonly used by many goroutines, so closing requires coordination with active operations. The selected closingMutex implementation is an internal read-write style mutex specialized for close synchronization. Its comment explains two key differences from sync.RWMutex: RLock takes priority over Lock, and reads are safely reentrant. The state field encodes the reader count and whether a writer is waiting, while condition variables are allocated lazily only in contended cases. This design favors common active-operation paths and still lets close wait for readers to drain when it must acquire the write side.

The same subsystem also has a small internal bridge between database/sql and database/sql/driver. internal/sql.go defines ScanContext and helper functions that wrap and unwrap a value while keeping the type opaque to ordinary users of the driver package. That boundary is a pattern used inside the Go repository when two standard-library packages need to coordinate behavior without expanding the public API. For handle and connection users, the practical lesson is that the exported API remains the stable contract; internal helpers exist to preserve package layering while the implementation evolves.

Sources: src/database/sql/closemu.go, src/database/sql/internal/sql.go

Compact Reference

Area	Public or driver-facing names	Behavior to remember
Handle lifecycle	sql.Open, connector-based opening, DB.Close	Open a reusable handle, share it broadly, and close it during application shutdown rather than per operation.
Pool sizing	DB.SetMaxOpenConns, DB.SetMaxIdleConns	Limit total open connections or retained idle connections; too-small open limits can make database use wait like a semaphore.
Pool timing	connection max lifetime, connection max idle time	Retire old or idle connections to match server, proxy, or operational requirements.
Pool observation	DB.Stats	Inspect pool behavior before and after tuning.
Driver creation	driver.Driver.Open, driver.DriverContext, driver.Connector.Connect	Drivers provide physical connections; database/sql owns ordinary idle pooling.
Reuse hooks	driver.Validator, driver.SessionResetter, driver.Pinger	Validate connections before return, reset sessions before reuse, and support health checks.
Context operations	driver.ExecerContext, driver.QueryerContext, driver.ConnPrepareContext, driver.ConnBeginTx	Implement these in drivers to make cancellation and transaction options precise.
Argument conversion	driver.NamedValueChecker, driver.ColumnConverter, driver.Valuer, driver.ValueConverter	Convert or reject application values before executing through a driver connection.

Practical Workflow

For a new service, start with one long-lived handle per database configuration. Import the driver, open the handle once during startup, call PingContext with a bounded context if startup should fail fast when the database is unavailable, and inject the handle into the rest of the program. Avoid opening a handle per request, because that defeats pooling and can exhaust both application and database resources. Also avoid assuming one handle equals one transaction or one session; use DB.Conn or transactions only when you intentionally need connection affinity.

Tune only after you know the workload. If the database server rejects excessive sessions, set SetMaxOpenConns to a safe upper bound and watch DB.Stats for waits. If latency is dominated by reconnecting, allow enough idle connections with SetMaxIdleConns, but keep server cost in mind. If infrastructure closes long-lived sessions or load balancing requires churn, add lifetime or idle-time limits. When writing or choosing a driver, prefer the modern interfaces listed above so cancellation, session reset, validation, named values, and custom values work consistently with the standard package. Next, read the pages on querying and changing data, prepared statements and transactions, and cancellation and SQL injection to connect pool behavior to day-to-day database operations.

Sources: src/database/sql/driver/driver.go, src/database/sql/ctxutil.go, src/database/sql/convert.go, src/database/sql/driver/types.go