@itentialopensource/remove-duplicates-from-array-of-objects-or-arrays

Deprecation Notice

This Pre-Built has been deprecated as of 01-15-2024 and will be end of life on 01-15-2025. The capabilities of this Pre-Built have been replaced by the IAP - Data Manipulation

Remove Duplicates From Array of Objects/Arrays

Table of Contents

• Overview
• Installation Prerequisites
• How to Install
• How to Run
• Attributes
• Examples
• Additional Information

Overview

This JST allows IAP users to remove duplicates from an array of objects or an array of arrays. The JST uses the Node method fast-deep-equal to compare nested objects/arrays accurately to remove duplicates. The resulting array will preserve only the first occurences of each element from the original array. Please note, the worst case in time complexity for this JST is $O(n^2)$; hence the operation might be slow for sufficiently large datasets.

Installation Prerequisites

Users must satisfy the following prerequisites:

• Itential Automation Platform : ^2022.1

How to Install

To install the pre-built:

• Verify you are running a supported version of the Itential Automation Platform (IAP) as listed above in the Prerequisites section.
• The pre-built can be installed from within App-Admin_Essential. Simply search for the name of your desired pre-built and click the install button.

How to Run

Use the following to run the pre-built:

1. Once the JST is installed as outlined in the How to Install section above, navigate to the workflow where you would like to remove duplicates from an array and add a JSON Transformation task.

2. Inside the Transformation task, search for and select removeDuplicates-Objects (the name of the internal JST).

3. After selecting the task, the transformation dialog displays.

4. The inputs to the JST would be the array from which the duplicates have to be removed.

5. Save your input and the task is ready to run inside of IAP.

Attributes

Attributes for the pre-built are outlined in the following tables.

Input

Attribute	Description	Type
array	An array of objects or an array of arrays.	array[objects/arrays]

Output

Attribute	Description	Type
array	Array without duplicate elements.	array[objects/arrays]

Examples

Example 1

Input Array

[
    {
      "a": "a"
    },
    {
      "a": "b"
    },
    {
      "a": "a"
    },
    {
      "a": 
          {
            "a": [1,2,3]
          }
    },
    {
      "a": 
          {
            "a": [1,2,3]
          }
    }
]

Output Array

[
    {
      "a": "a"
    },
    {
      "a": "b"
    },
    {
      "a": 
          {
            "a": [1,2,3]
          }
     }
]

Example 2

Input Array

[
    [[1],[1],[2]],
    [1,2],
    [2,1],
    [3,4,5],
    [1,2],
    [[1],[1],[2]],
    [[1],[1],[1]]
]

Output Array

[
    [[1],[1],[2]],
    [1,2],
    [2,1],
    [3,4,5],
    [[1],[1],[1]]
]

Additional Information

Please use your Itential Customer Success account if you need support when using this Pre-Built Transformation.