Optimizing For MongoDB Atlas And AWS Lambda

If you want to use MongoDB for your AWS Lambda serverless app, MongoDB Atlas may be a good option for you to start with. MongoDB Atlas has been generously offering 500MB storage for free and it is very easy to setup with just a few clicks.

Recently, I have migrated my databases to MongoDB Atlas and connected them from my AWS Lambda serverless apps. While i doing some test, however, I found that the AWS Lambda functions always timeout when connecting to MongoDB Atlas. After some studies, I have applied following changes to optimize the connection of AWS Lambda functions with MongoDB Atlas.

#1 Create Shared MongoDB Client Connection

I have moved the MongoDB connection logic out of the AWS Lambda function's handler so that the connection can be reuse between the invocations of the AWS Lambda function as long as the function is alive.

"use strict";
const MongoClient = require('mongodb').MongoClient;
const MONGODB_URI = 'YOUR_MONGODB_ATLAS_CONNECTION_STRING';

// Shared connection between invocations
let cachedDb = null;

function connectToDatabase (uri) {
  console.log('Connect to database');

  if (cachedDb) {
    console.log('Using cached database instance');
    return Promise.resolve(cachedDb);
  }

  return MongoClient.connect(uri, { useNewUrlParser: true })
    .then(client => {
      cachedDb = client.db(database);
      return cachedDb;
    });
}

module.exports.handler = (event, context, callback) => {
  connectToDatabase(MONGODB_URI)
    .then(db => {
        // Do some queries
    })
    .catch(err => {
        // Catch error
    });
};

#2 Set context.callbackWaitsForEmptyEventLoop = false

The second changes that I have made is to set the context.callbackWaitsForEmptyEventLoop to false as the first thing in the AWS Lambda function's handler. By setting the value to false, AWS Lambda will freeze the process as soon as the callback is invoked.

By default, the callback waits until the runtime event loop is empty before freezing the process and returning the results to the caller. Setting this property to false requests that AWS Lambda freeze the process soon after the callback is invoked, even if there are events in the event loop. AWS Lambda will freeze the process, any state data, and the events in the event loop. Any remaining events in the event loop are processed when the Lambda function is next invoked, if AWS Lambda chooses to use the frozen process.

https://docs.atlas.mongodb.com/best-practices-connecting-to-aws-lambda/

As a result, the AWS Lambda function's code will look like following:

"use strict";
const MongoClient = require('mongodb').MongoClient;
const MONGODB_URI = 'YOUR_MONGODB_ATLAS_CONNECTION_STRING';

// Shared connection between invocations
let cachedDb = null;

function connectToDatabase (uri) {
  console.log('Connect to database');

  if (cachedDb) {
    console.log('Using cached database instance');
    return Promise.resolve(cachedDb);
  }

  return MongoClient.connect(uri, { useNewUrlParser: true })
    .then(client => {
      cachedDb = client.db(database);
      return cachedDb;
    });
}

module.exports.handler = (event, context, callback) => {
  context.callbackWaitsForEmptyEventLoop = false;

  connectToDatabase(MONGODB_URI)
    .then(db => {
        // Do some queries
    })
    .catch(err => {
        // Catch error
    });
};

Once i applied the changes above, the AWS Lambda functions' invocation duration has been reduced from 6000ms to 100ms. I'm happy with the results and I hope these changes can help you too.

Bonus: MongoDB Atlas Free Tier Limitations

Although MongoDB Atlas has a great free tier plan, but there are some limitations for production usage:

1. If you are using AWS Lambda like me, then you may have a problem to whitelist the IP addresses to your MongoDB because AWS Lambda does not have a fixed IP address. You will need to create assign a VPC with Elastic IP to the functions which it will be charged.
2. There are only maximum of 100 connections available in the free tier MongoDB Atlas. If your application has high traffic, you may need to consider upgrade your plan.

To understand more, you may read the MongoDB Atlas documentation