node-red-contrib-cloud-scheduler
This module provides a Scheduler Request node in Node-RED for executing any flow at a given time interval using Google Cloud Scheduler. It allows you to run the flows at the same time each week, day, or hour with guaranteed execution and retries in case of failures.
Scheduler Request Start node will retry to execute the flow in the case of error or failure, with a minimum backoff value of 5s to maximum backoff value of 1h.
Pre-requisites
- The scheduler node requires Node-RED to be installed.
- The Node-RED should be accessible from internet.
- Select or create a Cloud Platform project.
- Enable the Google Cloud Scheduler API.
- Set up authentication with a service account so you can access the API from your local workstation.
Setup Google Cloud Scheduler and Google Credentials
Cloud Scheduler node uses Google Cloud Scheduler to execute the flow. GCP credentials are required to securely create a scheduler task in Google Cloud Scheduler. Use the scheduler node's configuration to provide GCP credentials.
If Node-RED is running under a GCP environment such as a Compute Engine, Google Kubernetes Engine or Cloud Run then there is an implicit identity presented to GCP and the flow developer need not provide any authentication configuration. However, if your Node-RED runtime is not running under GCP or you wish to call a service with a distinct identity, then you will need to use explicit credentials.
Credentials can be provided either as a path to a named key file or by creating a Node-RED managed name credentials secret. Each credential defined as a Node-RED secret has the following properties:
| Property | Type | Description |
|---|---|---|
| name | string |
Label for easy identification, essentially a comment. |
| account | string |
Credentials in the form of a JSON key. |
The credentials for a service account can be acquired from the APIs & Services menu. After you finish creating a service account key, it will be downloaded in JSON format and saved to a local file.
Install
To install the stable version use the Menu - Manage palette option and search for node-red-contrib-cloud-scheduler, or run the following command in your Node-RED user directory - typically ~/.node-red:
npm install node-red-contrib-cloud-scheduler
Or else install the package directly from Manage Palette
Restart your Node-RED instance and you should have a Schedule Request node node available in the palette.
Get Started
After installing the mapper node, follow the below steps:
- Drag the Scheduler Request node from the palette to the workspace.
- Double click to open the Config node.
- To add Click on the edit icon to Add Google Cloud Credentials in the config node. (Copy and paste the contents of the Google service account credential JSON file created above, directly into the Key field.)
- Click Add to save the credentials.
- The URL field will be auto-populated based on the browser URL. It is important to note that this URL should be accessible via internet for scheduler to execute the flow.
- Attach Scheduler Request node infront of any new or already existing flow.
For example, if your existing flow looks like below
Attach Scheduler Request node in front of this flow and attach Scheduer Response node witha status code of 200 as below.
Usage
With Scheduler node you can set up your flows to be executed at defined times or regular intervals. The scheduler node work as a cron job for your flows.
Run your batch and big data jobs on a recurring schedule to make them more reliable and reduce manual toil. Instead of using poorly written scripts or human intervention to run large jobs, Cloud Scheduler allows you to run them at the same time each week, day, or hour with guaranteed execution and retries in case of failures.
Important Note: Each scheduler node requires a publicly accessible URL where the flow will be deployed and will be used by the Google Cloud scheduler to send to a target according to a specified schedule.
It is also important to note that the target must be HTTP/S endpoints.
To test the scheduler node in a local environment, developer can either
- Utilize tools like ngrok to accept HTTP traffic
- Export their flows & import them to Krysp platform (built on Node-RED).
The Repeat field, defines the schedule for the Google Cloud Scheduler to invoke and execute the flow. The schedule can be as follows:
- Every minute
- Hourly
- Daily (at a specific time or at a certain frequency between time interval)
- Weekly (at a specific time or at a certain frequency between time interval)
- Monthly (on a specific day, for a date range or all days, at a specific time or at a certain frequency between time interval)
The node also provides a way to know the selected schedule in a readable format for better understanding.
FAQs
- How does cloud-scheduler work?
- Do I have to create a Google Cloud account?
- How to get google-cloud-credentials?
- How to add google-cloud-credentials?
- What should be the method (POST/PUT)?
- What is the URL?
- Is it mandatory to have a publicly accessible URL?
- If I am having problems and unable to troubleshoot what should I do?
How does cloud-scheduler work?
Cloud Scheduler is a fully manged cron job scheduler. It allows you to scheduler any job for executing the flow.
Do I have to create a Google Cloud account?
Yes, you've to create a Google Cloud account and create a new project for Google Cloud Scheduler to work. Please follow the guide here to continue.
How to get google-cloud-credentials?
Please follow the guide here to create Google Cloud credentials.
How to add google-cloud-credentials?
Drag and Open the scheduler node. Click on the edit icon to Add Google Cloud Credentials in the config node. (Copy and paste the contents of the Google service account credential JSON file created above, directly into the Key field.)
What should be the method (Post/Put)?
Cloud Scheduler will use POST method by default. PUT method can be used for updating any job.
What is the URL?
Cloud Scheduler will create the URL from the URL entered in the field in combination with node id.
Is it mandatory to have a publicly accessible URL?
Yes. It is important that the URL is publicly accessible for Google Cloud Scheduler to execute the flow.
If I am having problems and unable to troubleshoot what should I do?
If you're still not able to execute your flow using the scheduler, please send an email with a subject line Cloud Scheduler Issue at developer@krysp.io. Please describe your issue and attach your flow, logs (optional), and screenshots if possible.
Node-RED On Cloud
If you're looking for a Node-RED solution on cloud, try https://www.krysp.io/.
- Krysp platform is built on Node-RED
- Quick Start Templates for different use cases.
- Flow execution metrics for analysis
- Enhanced log viewer that shows application and system logs
- Multi-user capabilities
Contributions
Development of Cloud Scheduler Node happens in the open on GitHub, and we are grateful to the community for contributing bugfixes and improvements. Read below to learn how you can take part in improving cloud scheduler node.
Contributing Guide
Read our contributing guide to learn about our development process, how to propose bugfixes and improvements.
Discussions and suggestions
Use the Krysp Forum: https://www.krysp.io/forum to ask questions or to discuss new features.