Comment on page
Recursive workflows
Recursive workflows are an important tool in building apps that can scale.
A recursive workflow is one that schedules itself to run again, and in the wider programming world this is known as a loop. We use these when we want to perform actions on lists, and while other methods for processing lists are available, recursive workflows have an important advantage.
This article will provide a deeper understanding as to why recursive workflows are necessary, when to use them, and how to set them up.
It's best practice to use recursive workflows any time you want to make changes to a list of things. Some examples might be:
- Iterating over a list of products and marking them as inactive
- Sending a text message to all users who have opted in
- Making new relationships between your data types as you re-configure your database architecture
While the actions: Make changes to a list of things and Schedule workflow on a list can achieve this with fewer steps, they are limited in the following ways:
- Their performance decreases over time as the lists grow
- When the time it takes to search for and schedule the list takes longer than 10 seconds, bubble terminates the process leaving data unprocessed
- They can cause large spikes in capacity usage, causing other processes to have to wait, resulting in general slowness
Recursive workflows are immune to these issues as we process the data iteratively and can control the frequency at which the workflow is run. They also provide the ability for more complex processing, such as:
- Nesting loops within one another (trigger one recursive loop from another)
- You can define dynamic exit conditions (to tell the workflow when to halt)
- You can run an action right at the end of the workflow
It's important to consider the data type and how it will grow over the life of the app when selecting how to process lists, and unless the data set will remain very small, using a recursive workflow is the right choice.
Recursive workflows aren't a term used by Bubble (you won't find a run recursive workflow action) - rather, it describes a particular structure of connected events.
The only events which can trigger themselves (the critical piece for a recursive workflow) are backend workflows, so go there to get started.
In order to use backend workflows, you need to:
- Be on a paid Bubble plan
- Make sure that Enable Workflow API and backend workflows is checked in your app's API settings:
Settings > API
Now, let's make a really simple recursive workflow, one that iterates over every user in our database, and updates their email address.
This exact example can come in handy if you copy your live database data into your dev version, but want to sanitize any personal contact information (so that you don't accidentally send emails to real users while playing around in your development app).
- 1.Under Backend workflows create a new API workflow and add the following parameters:
Here we've defined 4 parameters:
Index - This will keep track of what user we're making a change to (we can only make changes to one item at a time)
Count - This stores the total number of users we want to iterate over, so we can calculate when to stop the workflow
Users - This is our list of things to iterate over
Delay - This is used to set how far in the future the workflow reschedules itself, typically in seconds
- 2.Now we add an action to this backend workflow that schedules itself, and set the parameters in the following way:
- Index: Incrementing the index by 1. The index is keeping track of how many times we've looped through our list of users. It should start at 1, and then each time the API workflow is triggered anew, it will increase by one
- Count: Passing through the count and users values without making any changes to them (we need them to stay the same for this setup to work)
- Checking if the current index is less than the count. This is our exit condition. The count is just how many cycles of the workflow we want to go through before quitting (i.e, how many users we want to run the workflow on)
- Scheduled date: In order to use capacity in a friendly manner, you should schedule the next run in the future. Setting the scheduled date to "Current date/time" would schedule the workflow to run again immediately, and for long lists and heavy operations this could cause the capacity to spike. Here, we're using the delay parameter to determine how many seconds in the future to schedule the next run.
We now have a recursive workflow that, once started, will keep going until the index is equal to the number of users in the list.
Somewhere in your app (usually on another page), you'll just need to trigger this workflow for the first time, setting the initial values of index, count, and users. After that, it will continue running until the exit condition is met.
Here's how it might look where you kick off the workflow:
The search criteria for count and users should be identical so that the process runs once, and only once for each item in the list
We're sourcing the delay parameter from our App variables. Most recursive workflows will use a similar delay, and storing these in a data type allows us to adjust the frequency without having to alter the logic and deploy to live. This way we can make adjustments quickly if we start seeing capacity issues from any of our recursive workflows.
We're using 3 number fields in our App variables data type to hold different delay values:
- Recursive Delay Short (sec): 0.5
- Recursive Delay Medium (sec): 2
- Recursive Delay Long (sec): 5
We could also use an Option set to store these delay values, and that would prevent us from having to alter logic to make a change, but we would still need to deploy after updating in order for the change to take effect.
Consider how many items will need to be processed and the urgency of completion to determine how long of a delay to use. For example, we often send all App admins an email when a new user signs up or performs a certain action. They need to be notified soon, but it's not crucial that this completes instantly. For this we would select the Long delay.
- 3.At this point our workflow isn't actually doing anything of value. So let's add the part that actually does something.
Here, we're using the index (which increments every iteration) to specify a user from the list, and updating their email.
The reason for the matt+index is because Bubble won't let you have more than one user with the same email in the database. So by appending +index, we get [email protected], [email protected] .... We're using arbitrary text because this allows us to build a text string made up of static and dynamic values.
The last thing we might want to do is send ourselves a notification when the recursive workflow has finished running. For this, we just need to append an action at the end of our workflow that will only run when we've iterated through every user in the list, which is the same thing as saying when index = count.
(username = username | password = password)