How to Process Thousands Of Records on Salesforce

Sharing is Caring

As any Salesforce Developer, Consultant or Admin will tell you there’s a lot of governor limits on the Force.com platform. Most of the time, these limits make perfect sense because they allow Salesforce to have a stable platform for all of its customers.

On Salesforce, if you want to process thousands of records the only way to truly achieve it is through asynchronous apex.

What is Asynchronous Apex?

Asynchronous Apex allows us to run processes in a separate thread at a later time without having the user sit around waiting for a task to finish. The benefits of asynchronous apex are User Efficiency, Scalability and higher limits.

User Efficiency

Users are able to be more efficient because they won’t be sitting around for the screen to finish loading and are able to do something else. Maybe they could send another email or make another call and close more revenue.

Scalability

As a single Transaction in Salesforce is pretty limited, it becomes a huge problem when the data we need to process is over the governor limits. Asynchronous Apex removes a lot of the limits because it processes it when more resources are available to use.

Higher Limits

Asynchronous Apex generally gets much higher limits than that what the synchronous methods (Triggers, Controllers, Controller Extensions) get. For example, the Total Heap Size doubles from 6mb to 12mb. The Maximum CPU time on Salesforce Servers goes from 10seconds to 60 seconds. Even the number of SOQL queries goes from 100 to 200.

In Salesforce, there’s a few different ways to implement Asynchronous Apex:

  1. Future Methods,
  2. Batch Apex,
  3. Queueable Apex, and
  4. Scheduled Apex.

Future Methods

A Future Method is an asynchronous method that runs in the background on another thread. We should think of a future method as a quick way to do a relatively small task on a small number of records.

See my blog post What are Salesforce Future Methods? for more details.

Batch Apex

Batch apex runs asynchronously based on available resources and can chunk up the data per batch OR we can provide our own batch sizes.

Using a Batch Job with the Query Locator, it’s possible to process up to 50 million records (50,000,000). For example, we could implement a batch job that looks like this:

We can then start the batch processing by calling it through Execute Anonymous or maybe through other Apex Code.

If you go to Environment -> Jobs -> Apex Flex Queue you can see the jobs and if they’ve executed yet and what the result was.

Queueable Apex

Queueable Apex allows us to add jobs to the Queue and to be able to monitor them. Queueable Jobs are very similar to Future Methods but have a future other benefits.

  • Calling System.enqueueJob puts the job into the queue and gives us an ID that allows us to potentially identify and monitor the job.
  • Non-primitive variables can be passed (ie sObjects, or lists of sObjects).
  • If a transaction fails any queued jobs are cancelled which gives us better protection.
  • Queueable Apex can be chained. Ie one job starts, which can that start another job. This is a really great way of running some pretty sophisticated processes.

Creating a Queueable class is pretty easy. We need to implement the Queueable interface and provide a void execute method which takes in a QueueableContext object. Here’s a fairly simplistic example that would create a renewal opportunity for accounts that are about to renew.

When we go to create the job we need to pass in a list of accounts.

If we need to do callouts from a Queueable Job we need to make sure that we implement the Database.AllowsCallouts interface.

Scheduled Apex

Scheduled Apex is Apex that is set to run at a certain time – it could be set to run every hour, every day, every week or etc. The main point is that Scheduled Apex enters the queue at the scheduled time and executes at some point after that.

Scheduled Apex classes implement the Schedulable interface and must contain an execute method that accepts a SchedulableContext class as it’s parameter. A good potential use case for Scheduled Apex is something like a billing run or an Accounts Receivable process.

Jobs can be scheduled through Apex and Execute Anonymous by using the following syntax.

System.schedule() is the method that needs to be called. It takes a name for the scheduled task, and then a string that is in the following format: “Seconds Minutes Hours Day_of_month Month Day_of_week optional_year”. If you don’t need to include something you can replace it with a ?.

And finally, a class that implements the schedulable interface.

Wrapping it Up

In this blog post, we’ve covered the various ways that we can execute asynchronous apex which has a lot less limits then synchronous apex. The future method is a good first step, but there’s a lot more scalable ways to build out your system.

Sharing is Caring

Brian is a software architect and technology leader living in Niagara Falls with 13+ years of development experience. He is passionate about automation, business process re-engineering, and building a better tomorrow.

Brian is a proud father of four: two boys, and two girls and has been happily married to Crystal for more than ten years. From time to time, Brian may post about his faith, his family, and definitely about technology.