Memory Leak in Echo Go with Dynamodb

A memory leak in an Echo Go service that uses the AWS SDK for Go v2 with DynamoDB typically arises when responses from DynamoDB are not fully drained or closed, and when the application retains references to request-scoped objects across requests. In Go, memory pressure often surfaces as continually rising RSS memory under load, even when the service handles only read operations.

With DynamoDB, common patterns that contribute to leaks include: not closing the response body from GetItem, Query, or Scan; holding references to large attribute values (e.g., base64-encoded blobs) in global or long-lived caches; and reusing contexts in ways that prevent cancellation signals from reaching the SDK, thereby keeping internal buffers alive. The Echo framework ties each request to a context; if a handler passes a request-scoped context to a DynamoDB call and then stores a reference to the response payload in a global variable or a long-lived cache keyed by user ID, the garbage collector cannot reclaim that memory until the reference is removed.

For example, consider a handler that calls GetItem and stores the raw attribute map in a module-level map to avoid repeated database calls. If the stored value includes large fields and the cache eviction policy is absent or misconfigured, memory usage grows unbounded as traffic increases. This pattern becomes more pronounced when the SDK is configured with a custom HTTP client that buffers responses, and the application does not read the entire response body or close it promptly.

In the context of middleBrick scans, this combination would surface findings in the Input Validation and Data Exposure checks, highlighting improper resource handling and potential exposure of sensitive data in memory. The scan would note the absence of response-body closure and context timeouts, which can contribute to increased memory retention under sustained load.

To mitigate memory leaks when using DynamoDB in Echo Go, ensure that every AWS SDK response is fully consumed and closed, use context with timeouts for each request, avoid caching large raw payloads in global variables, and stream or paginate large results instead of loading entire datasets into memory.

1) Properly close DynamoDB responses and drain bodies

Always close the response body from DynamoDB operations. With the AWS SDK for Go v2, the response includes an io.ReadCloser that must be closed to release underlying buffers.

import (
	"context"
	"fmt"
	"io"

	"github.com/aws/aws-sdk-go-v2/aws"
	"github.com/aws/aws-sdk-go-v2/config"
	"github.com/aws/aws-sdk-go-v2/service/dynamodb"
	"github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
	"github.com/labstack/echo/v4"
)

func getItemHandler(c echo.Context) error {
	db, err := dynamodb.NewFromConfig(aws.Config{})
	if err != nil {
		return fmt.Errorf("unable to create client: %w", err)
	}

	resp, err := db.GetItem(c.Request().Context(), &dynamodb.GetItemInput{
		TableName: aws.String("Users"),
		Key: map[string]types.AttributeValue{
			"ID": &types.AttributeValueMemberS{Value: c.Param("id")},
		},
	})
	if err != nil {
		return fmt.Errorf("getitem failed: %w", err)
	}
	// Always close the response body to release buffers
	defer func() { _ = resp.Body.Close() }()

	// Drain and close; read the body if you need to inspect raw payloads
	_, _ = io.ReadAll(resp.Body)

	if len(resp.Item) == 0 {
		return c.NoContent(404)
	}
	// Process only required fields; avoid storing the entire Item globally
	return c.JSON(map[string]interface{}{
		"id": resp.Item["ID"],
	})
}

2) Use context timeouts and avoid long-lived references to large payloads

Create a scoped context with timeout per request. Do not store large items in a global cache; if caching is required, cache only identifiers or small summaries, and set TTLs that prevent unbounded growth.

import (
	"context"
	"time"

	"github.com/aws/aws-sdk-go-v2/aws"
	"github.com/aws/aws-sdk-go-v2/config"
	"github.com/aws/aws-sdk-go-v2/service/dynamodb"
	"github.com/labstack/echo/v4"
)

var (
	// Avoid caching large raw attribute maps; use this for metadata only
	summaryCache = make(map[string]string)
)

func queryHandler(c echo.Context) error {
	timeoutCtx, cancel := context.WithTimeout(c.Request().Context(), 8*time.Second)
	defer cancel()

	db, err := dynamodb.NewFromConfig(aws.Config{}, func(o *dynamodb.Options) {
		of
	})
	if err != nil {
		return fmt.Errorf("unable to create client: %w", err)
	}

	out, err := db.Query(timeoutCtx, &dynamodb.QueryInput{
		TableName:                 aws.String("Events"),
		KeyConditionExpression:    aws.String("PK = :v"),
		ExpressionAttributeValues: map[string]types.AttributeValue{":v": &types.AttributeValueMemberS{Value: c.Param("pk")}},
	})
	if err != nil {
		return fmt.Errorf("query failed: %w", err)
	}
	defer out.Items.Close()

	var results []map[string]interface{}{}}
	for out.Items.HasMore() {
		item, err := out.Items.Next()
		if err != nil {
			return fmt.Errorf("failed to get item: %w", err)
		}
		// Extract only necessary fields to avoid holding large blobs
		results = append(results, map[string]interface{}{
			"id":   item["ID"],
			"name": item["Name"],
		})
	}
	// Do not assign results to a global variable; return directly
	return c.JSON(results)
}

3) Paginate Scan/Query results and limit field projection

For operations that require scanning or querying large datasets, use pagination and request only the attributes you need. This reduces memory pressure per page and avoids accumulating many pages in memory.

func scanHandler(c echo.Context) error {
	db, _ := dynamodb.NewFromConfig(aws.Config{})
	paginator := dynamodb.NewScanPaginator(db, &dynamodb.ScanInput{
		TableName: aws.String("LargeTable"),
		// Select only required attributes to reduce memory usage
		ProjectionExpression: aws.String("ID,Name,Status"),
	})

	var items []map[string]interface{}{}
	for paginator.HasMorePages() {
		page, err := paginator.NextPage(c.Request().Context())
		if err != nil {
			return fmt.Errorf("scan page failed: %w", err)
		}
		for _, item := range page.Items {
			// Store only projected fields
			items = append(items, map[string]interface{}{
				"ID":     item["ID"],
				"Name":   item["Name"],
				"Status": item["Status"],
			})
		}
	}
	return c.JSON(items)
}