Three ways of removing duplicates from an array using JavaScript

August 2, 2020

javascript

Sometimes you can receive data from a different number of sources for example an API, and it can contain some repeated data and you just want it cleaned up for whatever reason (in some cases anyway).

So, I will be providing three different ways of how this can be accomplished.

1. Using the traditional for-loop

const removeDups = (arr) => {
    const unique = [];

    for (let i = 0, len = arr.length; i < len; i += 1) {
        const elem = arr[i];

        if (unique.indexOf(elem) === -1) { // OR !unique.includes(elem)
            unique.push(elem);
        }
    }

    return unique;
};

In this function, we first create an empty array that will store all the unique elements.
We then iterate over all the elements in the array given and check if each element already exists in the array of unique elements and if it doesn't exist in the array of unique elements, then add it but if not, ignore it.

2. Using the array filter method

const removeDups = (arr) => {
    return arr.filter(function (elem, index) {
        return arr.indexOf(elem) === index;
    });
};

I got this way of doing it from one of the best vanilla JavaScript teachers out there; and that's none other than Chris Ferdinandi. Please sign up for one of his wonderful courses or his daily newsletter, you will not regret your decision.

Now back to the code and how it works, let us assume we had the following array

//                    0          1         2         3        4        5        6
const browsers = ['firefox', 'chrome', 'firefox', 'opera', 'edge', 'chrome', 'firefox'];

I have commented on the index of each element in the array above it. Let's now just focus on the firefox browser and as you can see it has the indexes of 0, 2, and 6.

So as the filter method loops through the array and first encounters the firefox browser at index 0, the check arr.indexOf(elem) === index passes because the array indexOf method returns the lowest index of an element in an array, which in this case will be 0, but for the next iterations where the element is firefox the check will fail because since we know the lowest index of firefox is 0 and neither the other indexes for firefox (2, 6) are equal to 0 they will be ignored.

The same concept will be applied for the other elements (or in this case, browsers).

For a better explanation of this concept, please check out this post from Chris.

3. Using Set

const removeDups = (arr) => {
    return Array.from(new Set(arr));
};

Here we take advantage of some of the newest features of JavaScript language i.e. Array.from static method and Set.

If you are coming from a Math, Python or Java programming background, you will probably be more familiar with the concepts of sets than most.

Essentially, a set is collection of distinct values or objects. So, when we provide an array to the constructor of a set in JavaScript, it will return a set object of distinct values.

But since we want an array to be returned from our function, pass the created set to the Array.from, which as the Mozilla documentation states "creates a new, shallow-copied Array instance from an array-like or iterable object" or in layman's terms, creates and returns an array.

Thank you for reading and God bless.