@itentialopensource/archive-job-data

Deprecation Notice

This Pre-Built has been deprecated as of 04-30-2024 and will be end of life on 04-30-2025. The capabilities of this Pre-Built have been replaced by the IAP - Workflow Utilities

Archive Job Data

Table of Contents

• Archive Job Data
  • Table of Contents
  • Overview
  • Installation Prerequisites
  • How to Install
  • How to Run
  • Inputs
  • Additional Information

Overview

This pre-built allows IAP users to archive the jobs and tasks collections to keep IAP performant. It also allows the users to restore the archived jobs and tasks. Users have the ability to choose:

1. The number of days older than which, all qualifying data will be archived.
2. Custom names for the archive collections to store the matching jobs and tasks documents.
3. To keep or delete the matching documents from the original jobs and tasks collections after backing them up.
4. Restore the archived jobs and tasks documents and delete the archived collections.

Users can use the included Operations Manager item to schedule the pre-built to run at regular intervals.

Notes:

1. For every run, new jobs and tasks collections would be created with the current timestamps appended to the end of the jobs and tasks collection names put in by the user.
2. Only jobs and tasks related to jobs that are either in complete or canceled state will be considered for archiving. Jobs that are in errored or incomplete state will be left untouched.
3. Getting an RangeError [ERR_OUT_OF_RANGE] typically indicates that your bundle size is too large. Try setting it lower within the automation catalog.
4. Very large databases with a lot of jobs and tasks to archive will typically result in multiple archive collections within the database.

Installation Prerequisites

Users must satisfy the following prerequisites:

• Itential Automation Platform
  • ^2021.2
• Adapter DB Mongo
  • ^0.5.7

How to Install

To install Archive Job Data, verify you are running a supported version of Itential Automation Platform (IAP) as listed above in the Installation Prerequisites section. If you do not currently have App-Artifacts installed as a service on your node, the .tgz file or "tarball" can be obtained from the Nexus repository. Please refer to the instructions included in the App-Artifacts README to install it.

Archive Job Data can be installed from within Admin Essentials. Search for archive-job-data and click the Install button.

How to Run

Use the following steps to run the pre-built:

1. Once the pre-built is installed as outlined in the How to Install section above, go to Operations Manager and run the Archive Jobs and Tasks Collections automation.
2. In the Copy/Move Jobs data to Collection and the Copy/Move Tasks data to Collection fields, enter the names for collections you want the matching jobs and tasks collection documents to be copied to.
3. In the Copy/Move data from before # days field, enter the number of days (from the current date) oolder than which you want data to be archived. Note: Please enter a positive number
4. In the MongoDB Adapter dropdown menu, select the adapter that is connected the MongoDB that contains the Tasks and Jobs Collections that you want to create an archive of.
5. In the Batch Size field, select how many jobs you want to archive at a time.
6. Check the Create Archive Collections checkbox if you want to create an archive collection that will be stored in the MongoDB. Unchecking this box will mean no archive will be created.
7. Check the Delete copied data from collections checkbox if you want to discard the matching documents from the original jobs and tasks collections after they have been backed up.
8. Check the Verbose checkbox if you want to see the results of the pre-built at the end of its run and click on START and then on VIEW JOB.

To restore the archived jobs and tasks, run the Restore Archived Jobs and Tasks workflow.

Note: The adapter ID for mongoDB in all the respective tasks of the workflow must be changed accordingly.

Inputs

The inputs for the pre-built are outlined in the following table. To run the workflow directly from IAP/API call (and not through Operations Manager), please encapsulate all of these inputs into a root object with the key formData.

Input	Description	Type
createArchiveCollections	Set as true or false to either create or skip creating a new collection for jobs and tasks with archived data.	boolean
deleteCopiedDataFromCollections	Set as true or false to either discard or keep the matching documents from the original jobs and tasks collections after they have been backed up.	boolean
verbose	Set as true or false to either see or not see the results of the pre-built at the end of its run.	boolean
copyMoveJobsDataToCollection	Enter the name of the desired collection (new or pre-existing) where you want the matching documents from the jobs collection to be archived.	string
mongoDbAdapter	Select the Mongo DB Adapter instance from which the documents from jobs and tasks collections data will be archived.	string
copyMoveDataFromBeforeDays	Enter the number of days (from the current date) older than which all data will be archived.	integer
copyMoveTasksDataToCollection	Enter the name of the desired collection (new or pre-existing) where you want the matching documents from the tasks collection to be archived.	string
batchSize	Enter the number of jobs that should be archived or deleted on a single iteration.	integer

Inputs for Restore Archived Jobs and Tasks workflow are defined below:

Input	Description	Type
deleteFromArchiveCollections	Set as true or false to either delete or not delete the archived jobs and tasks collections.	boolean
archiveJobsCollectionName	Enter the name of the archived jobs collection that you want to be restored.	string
archiveTasksCollectionName	Enter the name of the archived tasks collection that you want to be restored.	string
numberOfArchivedCollections	Enter the number of archived collections that should be restored.	integer

Sample Input for the overall Pre-Built:

{
  "formData": {
    "createArchiveCollections": true,
    "deleteCopiedDataFromCollections": true,
    "verbose": true,
    "copyMoveJobsDataToCollection": "jobs_archive",
    "mongoDbAdapter": "mongoPronghorn",
    "copyMoveDataOlderThanDays": 3,
    "copyMoveTasksDataToCollection": "tasks_archive",
    "batchSize": 2000,

  }
}

Sample Input for the Restore Archived Jobs and Tasks workflow:

{
	"deleteFromArchiveCollections": true,
	"archiveJobsCollectionName": "Archive_Jobs_2023-08-03T21:17:11.079Z",
	"archiveTasksCollectionName": "Archive_Tasks_2023-08-03T21:17:11.079Z",
	"numberOfArchivedCollections": 9
}

Additional Information

Please use your Itential Customer Success account if you need support when using this Pre-Built.