Roll out web analytics at scale pt. 5 – Checking whether Google Tag Manager is installed automatically

Background

We’re finally here – the last part in our five part series on rolling out web analytics at scale. As always, if you still need to read parts one, two, three, and four, they’re right here for you. Our final part will address how you can check multiple websites for Google Tag Manager installation automatically, and be notified when they’re installed.

Introducing the UrlFetchApp service

Google have really pushed the capability of Apps Script to the next level with their UrlFetchApp service. This simple library allows you to send http request, meaning you can communicate with the world outside the Google universe. For example, you can make calls to 3rd party APIs which are not already built into Google Apps Script. The basic syntax looks like this:

UrlFetchApp.fetch(“https://someurl.com”);

This will essentially grab the content of that URL and return the response. You can then manipulate the response using other built in Apps Script methods. For example, getContentText();

var response = UrlFetchApp.fetch("https://someurl.com");

var html = response.getContentText();

Logger.log(html); 

This allows you to print the content that is returned from this URL to the logger.

Applying this to our use case

As you can manipulate the responses, we’re going to check whether our response contains ‘GTM-’ and use that as our indicator of installed status. For example, the following code will search the response from that URL for ‘GTM-’, and if it finds it, it will send an email to someone:

 
var response = UrlFetchApp.fetch(website);
                    if (response.getContentText().search('GTM-') != -1) {
                        GmailApp.sendEmail('someEmail@email.com', 'GTM installed on ' + website, 'GTM is now installed on ' + website + '.\n\nDo what you need to do.');
                    }; 

We could come up with a regular expression for the tag manager container, but to be fair I have never encountered a situation where I recieved a false positive for installed status (and have checked over 1000 websites at once).

Scaling this up

Unfortunately, there are a lot of challenges with an synchronous execution of a script like this, where fetching URLs could take a long time, or never come back at all. This is compounded by the fact that Apps Script has execution timing limits. To work around this, I have created a script that can check 30 URLs at a time in random order, from an array of URLs of any size. It also handles errors and continues on with the script without getting stuck. It also saves the state so that it does not check ones that have already been installed again. You can schedule this to run every 5 minutes, so it will eventually get through a relatively large list of URLs.

The 30 URL limit was necessary to handle this problem in Apps Script, but if you know something like Node.js, you won’t encounter any problems with checking thousands of URLs, as you can send these requests asynchronously.

The solution

For this to work you need to update the check array with the websites you would like to check for GTM installation. You’ll also need to update the email address which receives alerts when it finds an installed one, or an error.

 
function myFunction() {

    var check = ['https://leadto.sale', 'http://www.cwar082.com', 'http://google.co.nz', 'https://stackoverflow.com'];
    var results = [];

    //check if save file exists
    function shuffle(array) {
        var currentIndex = array.length,
            temporaryValue, randomIndex;

        // While there remain elements to shuffle…
        while (0 !== currentIndex) {

            // Pick a remaining element…
            randomIndex = Math.floor(Math.random() * currentIndex);
            currentIndex -= 1;

            // And swap it with the current element.
            temporaryValue = array[currentIndex];
            array[currentIndex] = array[randomIndex];
            array[randomIndex] = temporaryValue;
        }

        return array;
    }

    function checkSave() {

        var saveFile = DriveApp.getFilesByName('gtmCheck.json');


        if (saveFile.hasNext() === false) {

            for (var j = 0; j < check.length; j++) {

                results.push({
                    'website': check[j],
                    'status': 'uninstalled'
                });

            }

            var saveFile1 = DriveApp.createFile('gtmCheck.json', JSON.stringify(results), MimeType.PLAIN_TEXT);
            return saveFile1.getAs(MimeType.PLAIN_TEXT).getDataAsString();
        } else return saveFile.next().getAs(MimeType.PLAIN_TEXT).getDataAsString();
    }


    function runOn30() {
        var save = JSON.parse(checkSave());
        DriveApp.getFilesByName('gtmCheck.json').next().setTrashed(true);
        var k = 0;
        var count = 0;
        while (k < save.length && count < 30) {

            if (save[k].status == "installed") {
                k++
            } else {
                var website = save[k].website;

                try {

                    var response = UrlFetchApp.fetch(website);
                    count = count + 1;
                    if (response.getContentText().search('GTM-') != -1) {
                        save[k].status = "installed";
                        GmailApp.sendEmail('someEmail@email.com', 'GTM installed on ' + website, 'GTM is now installed on ' + website + '.\n\nDo what you need to do.');
                        //alert someone
                        k++
                    } else k++
                } catch (e) {
                    GmailApp.sendEmail('someEmail@email.com', 'Error on website while checking for GTM', 'There was an error on ' + website + ' when checking for GTM.\n\nMight pay to check it.');
                    continue
                }
            }
        }
        return save
    }

    (function createNewSave() {

        var newSave = DriveApp.createFile('gtmCheck.json', JSON.stringify(shuffle(runOn30())), MimeType.PLAIN_TEXT);

    })();

} 

Final thoughts

This was done in Apps Script for convenience, but there are other tools out there that are much better at handling this problem. It also only checks the URLs you enter in the array, so won’t automatically check every page of the website for you. However, this solution is good enough to get started. With this blog series, you should now have the ideas or tools needed to roll out web analytics to your customer base at scale. There are absolutely no excuses for not tracking online marketing anymore – so get out there and show us how good you are!