Recursive workflows
Recursive workflows are an important tool in building apps that can scale.
A recursive workflow is one that schedules itself to run again, and in the wider programming world this is known as a loop. We use these when we want to perform actions on lists, and while other methods for processing lists are available, recursive workflows have an important advantage.
This article will provide a deeper understanding as to why recursive workflows are necessary, when to use them, and how to set them up.
It's best practice to use recursive workflows any time you want to make changes to a list of things. Some examples might be:
- Iterating over a list of products and marking them as inactive
- Sending a text message to all users who have opted in
- Making new relationships between your data types as you re-configure your database architecture
While the actions: Make changes to a list of things and Schedule workflow on a list can achieve this with fewer steps, they are limited in the following ways:
- Their performance decreases over time as the lists grow
- When the time it takes to search for and schedule the list takes longer than 10 seconds, bubble terminates the process leaving data unprocessed
- They can cause large spikes in capacity usage, causing other processes to have to wait, resulting in general slowness
Recursive workflows are immune to these issues as we process the data iteratively and can control the frequency at which the workflow is run. They also provide the ability for more complex processing, such as:
- Nesting loops within one another (trigger one recursive loop from another)
- You can define dynamic exit conditions (to tell the workflow when to halt)
- You can run an action right at the end of the workflow
It's important to consider the data type and how it will grow over the life of the app when selecting how to process lists, and unless the data set will remain very small, using a recursive workflow is the right choice.
Recursive workflows aren't a term used by Bubble (you won't find a run recursive workflow action) - rather, it describes a particular structure of connected events.
The only events which can trigger themselves (the critical piece for a recursive workflow) are backend workflows, so go there to get started.
In order to use backend workflows, you need to:
- 1.Be on a paid Bubble plan
- 2.Make sure you've got Enable Workflow API and backend workflows enabled
3. Under Backend workflows simply create a new event, and as one of the actions, choose Schedule API workflow, picking the event you just created.
That's it! You just created your first recursive workflow! Only it doesn't do anything....yet.
Let's make a really simple recursive workflow, one that simply iterates over every user in our database, and updates their email address.
This exact example can come in handy if you copy your live database data into your dev version, but want to sanitize any personal contact information (so that you don't accidentally send emails to real users while playing around in your development app).
To start with, we'll define some event parameters that will make it easy for us to control the behavior of the recursive workflow.
Here we've defined three parameters:
Index - This will keep track of what user we're making a change to (we can only make changes to one item at a time)
Count - This stores the total number of users we want to iterate over, so we can calculate when to stop the workflow
Users - This is our list of things which we're going to make changes to
Now that we have these, we can configure our Schedule action a little better:
When we schedule the same workflow from within itself like this, we're doing a few crucial things:
- Incrementing the index by 1. The index is keeping track of how many times we've looped through our list of users. It should start at 1, and then each time the API workflow is triggered anew, it will increase by one
- Passing through the count and users values without making any changes to them (we need them to stay the same for this setup to work)
- Checking if the current index is less than the count. This is our exit condition. The count is just how many cycles of the workflow we want to go through before quitting (i.e, how many users we want to run the workflow on)
We now have a recursive workflow that, once started, will keep going until the index is equal to the number of users in the list.
Somewhere in your app (usually on another page), you'll just need to trigger this workflow for the first time, setting the initial values of index, count, and users. After that, it will continue running until the exit condition is met.
Here's how it might look where you kick off the workflow:
Of course, at this point our workflow isn't actually doing anything of value. So let's add the part that actually does something.
Here we are just grabbing our list of users, and pulling out the Nth entry in the list, where N = index. In other words, every time we cycle through the workflow, we move one entry down in the list, find the user who's there, and update the email for that user.
The reason for the matt+index is because Bubble won't let you have more than one user with the same email in the database. So by appending +index, we get [email protected], [email protected] .... We're using arbitrary text because this allows us to build a text string made up of static and dynamic values.
The last thing we might want to do is send ourselves a notification when the recursive workflow has finished running. For this, we just need to append an action at the end of our workflow that will only fire when we've iterated through every user in the list, which is the same thing as saying when index = count.
(username = username | password = password)
Here's the full overview of the workflow