What is Google Data Studio?

Google Data Studio is a data dashboard and reporting tool which provides easy-to-read, shareable and customisable reports from a variety of data sources.

Data Studio is now free to use and can produce reports from almost any online data source; including digital marketing analytics, CRM pipelines, YouTube stats, etc.

You can browse a gallery on Google’s site here, or read on to skip to the good stuff!

Google Data Studio history

In May 2016, Google announced Data Studio would be accessible for users who are not subscribed to the Google Analytics 360 Suite. The product initially launched with a limited number of reports per account and a limited number of data connectors. But, since February 2017’s announcement, the tool has been unlimited and free for all users. In March, Google finally launched it worldwide and the product has been improving ever since (see changelog).

Initially, Google Data Studio launched with connectors for Google-only services (including Google Cloud), as well as a few more general purpose Data Sources, such as File Upload and MySQL. Then, in September 2017, the Data Studio “Community Connectors” developer programme launched with around 200 community-driven connectors from around 10 providers.

Popular providers like Supermetrics have launched Data Studio Community Connectors under the developer program

Not only did this make the platform infinitely more useful for cross-channel marketers (popular providers like Supermetrics got in there; below) but this has also opened up the opportunities for in-house and in-agency developers to build and deploy their own connectors. And that’s exactly what we did here at Impression:

About Connectors and Google Apps Scripts

Community Connectors can be built by almost anyone with experience in JavaScript and using Google Apps Script. The documentation is reasonably thorough despite the program being in its early days; you can certainly get up and running with the existing docs.

To become a Google Data Studio Community Connector developer, you need to request to be whitelisted to see extra features in your Google Scripts account. You can send this request here. You’ll know you have access as you also get an invite into a Google Group for developers and your Scripts ‘Publish’ menu will now have this option:

Google Apps Script: Deploy from manifest

For those familiar with the Google Apps Script IDE, you’re in luck as all the usual Apps Scripts are available for use here.

Beware that whitelisting your account can take “2-4” working days. In reality, I was whitelisted in a little over a day.

Example use: pulling live keyword ranking data into Google Data Studio

Our demo below is far from complete, but it’s enough to give a good demonstration of what’s possible with Data Studio connectors. It actually only took a few hours to set up and it’s safe to say we’ll be working on this a lot more in the coming weeks.

There are a number of elements that this tutorial will not cover, including date ranges, OAuth authentication/connectivity and data caching. These topics will be covered in a future follow up post, or in the comments below.

Our enterprise search engine rank tracking data provider, STAT, provides a really open API that allows us to pull all keywords that we’re tracking for a specific site. Currently, among all Community Connectors, there are only a few entry-level packages that do the same, so we were keen to plug in the enterprise level data for our clients.

It doesn’t allow for date-specific searching, hence this tutorial not covering that requirement, but we know of a couple other agencies circumventing this by caching the data in SQL databases and then using the SQL connector to get the data that way.

How to create a Google Data Studio Community Connector

As Google’s documentation simply puts it, there are three steps to creating a community connector;

  1. Create a new Apps Script project.
  2. Write the connector code.
  3. Complete the project manifest.

I don’t disagree with this sentiment, but it’s not entirely that simple…

Setting up a new Google Apps Script for the Connector

No special settings are required in order to start scripting. However, you will need a whitelisted developer account to test deploy your work. In the interim, you can use the standard debugging tools and spoof requests to your getData() function to check that the data is properly formed. I’ve covered how we tackled this in the Testing and Deployment section below.

If you can already create a manifest document, do so now, otherwise refer back to this once you’re whitelisted. This is required to deploy the Script later on. You can view your Scripts manifest by clicking:

To edit your manifest file to include the required content, click View > Show manifest file, once whitelisted.

The full manifest file specification can be found here; otherwise please feel free to repurpose the example in my code below.

Default file layouts and function calls

The Community Connector developer program documentation gives a good amount of information on this. From this page, copy across the shell functions and the schema variable that I’ve listed below or dive straight into the sample project we’ve developed at the bottom of this post.

Functions

  • getConfig (request) {}
  • getSchema (request)  {}
  • getData (request) {}
  • getAuthType () {}

Variables

  • Schema or “fixedSchema” in the documentation

Assuming that you aren’t yet using OAuth and you’re comfortable reading code samples (Google’s or mine below) then I will focus on the getData() function here.

This function, if not implemented properly, can result in errors for a number of reasons. I was tripped up by the header field order not matching my rows and the varying row length (different queries can result in a different number of metrics being required).

The most important part of this function is to return a properly formed object:

var return_data = {
schema: headers,
rows: rows,
};

The schema argument expects an array of Field objects. If in the request, the fields array appears in a different order to your defined schema (for example metrics can be reordered), then you must refer to this array and reorder it. If any metric is not requested in the request.fields array, then it should be excluded from the reordered return array.

Similarly, the rows must be formatted correctly to prevent errors. The required format is an array of objects; one per row. Within this object, the values parameter is an array of the requested field values. A properly formatted example of this is below, along with an error page example.

rows=[{values=[ab split testing, 50, 101.0]}, {values=[ab testing, 1900, 101.0]}, {values=[bing ads management, 30, 8]}];

This is what a Data Studio Connector error looks like. Often it will be due to a poorly formatted response to getData().

Connector application design

When you’re designing your own Community Connector you’ll need to be mindful of the dimensions and metrics you’re looking to present. This will fit into your schema and will then be presented like so on the controls sidebar.

This is how Dimensions and Metrics are presented in Data Studio

I always like to think of dimensions not as metrics, but ways to segment your data (location, city name, keyword, colour, etc). Metrics are the facts which you are tying to your dimensions and these can usually be filtered. This principle of metrics and dimensions is exactly the same as within Google Analytics.

If your connector allows multiple dimensions and multiple metrics, for use across a number of the graph and chart types on offer in Data Studio, then you should be more wary of error handling within your application. Google provides more information on error handling here.

Testing

To test our getData(request) function in advance, we constructed a sample request object from the documentation and later refined this when we had access. If you consider this is based on your configuration, you can take the below to represent a sample query for yourself. The configParams object contains the data your user enters when they install the Data Connector. The fields array includes a named reference to your Schema object — the fields here (dimensions and metrics) are in order as per your user’s selection.

var request = {
configParams:{
apiKey:'12345',
siteName:'impression',
siteId:'666'
},
fields:[
{name:'keyword'},
{name:'regionalSearchVolume'},
{name:'googleRank'}
] };

Once you’ve got access to deploying your connector to Data Studio, I found the StackDriver logging invaluable. In the place of the Logger class, you’ll want to use the Console class so that the debug content is passed to StackDriver.

With Google Cloud’s StackDriver logging you can debug your connector whilst it’s in use

You can actually expand out each line item here, to view the JSON payload and dig a little deeper. Errors are also shown in here, too, and they look like so;

Deploying a custom Data Studio Connector

The process of deploying the connector is straightforward. Click Publish > Deploy and then find and click on the Data Studio URL.

When you can deploy from Apps Scripts, click on Publish > Deploy from manifest to get this option. Expand the top option and select the Data Studio URL to start adding your Data Source to a report

This will open a new window and add the data source to your Data Studio account. Here, you can enter any required information and add it to a report. You can also take this opportunity to share the Data Source with colleagues.

Much like other connectors – all settings will be visible on the Data Source setup screen. After entering these details, you add the configured Data Source to your report.

Once you’ve added the data source, you have an opportunity to confirm you’re happy with the settings for your fields. You can use this to trial the impact of any schema changes, such as the aggregation settings.

Although you define your fields in your Apps Script code, you can also make amends to the data on this screen.

Finally, once on your report canvas, you’ll be asked one final time to confirm that you’re happy to add the new Data Source to your report. Following this, you’re free to select a graph/chart and begin using the data.

When adding a newly installed Data Source, you’ll be presented with this modal window

A table is the most appropriate display of the data my example is showing. You can see here that I’ve selected two metrics, Google Rank and Regional Search Volume, and that they’re being shown in this order in the table. As the data is now read into Data Studio, the Sort and Filter settings work as expected without additional API calls.

A successful deployment and test of a new Data Source in Data Studio

Questions?

I’d be more than happy to take a look at them in the comments below! Please feel free to post code snippets or links to hosted code gists or similar. We’re interested to see what you’re achieving with Data Studio Connectors. Good luck!

Full code example

Aaron Dicks

Managing Director

Managing Director of Impression. Search engine optimisation, paid media and web analytics consultant. Also web developer and digital all-rounder. @aarondicks

8 thoughts on “How to Build a Google Data Studio Community Connector — Tutorial

  1. Balkaran Chandel says:

    https://uploads.disquscdn.com/images/2b0405775eed8700ade3672bf38ac45e6e4bafd0956a257ac8ebaeafd5dc05cc.png https://uploads.disquscdn.com/images/d9d72e739ff9cdc96900f58658380ccbcdb34b6583d80e109c67a0f1d3efa082.png https://uploads.disquscdn.com/images/64b3340b228a6214e47909f052f810a9b10c31518942979038c5118693809e99.png

    https://uploads.disquscdn.com/images/96bf1684c6654798e1c91a93ff7b69dddfc8ff3df19e18e63fb1116a9ba5b5f2.png
    https://uploads.disquscdn.com/images/7d96aac4b3a4d8a1a96dc69a75679132aa360e4302c2ca411b2338addbc3d66d.png

    Hello ,
    I am trying to build connector that will fetch the data from our API and push that data to Data Studio.
    My data format is fine as you suggested, but i am getting two errors one is same as you mentioned in this post and other is fix data source. Right now i am not using oAuth. I am also attaching the screen shots of errors and my manifest file.
    Could you please also let me know whats the use of (sources [] ) array inside manifest file? is it used for creating the data sources in Data Studio?

    Thanks

    1. Balkaran Chandel says:

      Hello,

      Could you please help me here

    2. aarondicksaarondicks says:

      This is most likely due to the incorrect connector configuration. Can you confirm you’ve entered;

      Full API key
      Correct Site ID (which you need to retrieve via the API)
      Subdomain word which appears before .getstat.com in your login dashboard?

      Thanks
      Aaron

      1. AJ says:

        Hi, I’m also having trouble with the connector working, and can confirm the input fields were correct

  2. snooplion says:

    Awesome work Aaron – I was wondering was there ever a part 2 to this? Would love to know if you found a solution to getting the date ranges in there as well. Cheers.

    1. aarondicksaarondicks says:

      Hi,
      I haven’t had a chance to write this up just yet, but this sounds like an easy thing to implement (https://developers.google.com/datastudio/connector/reference#getdata)

      In your getData() object, just reference request.dateRange.startDate or request.dateRange.endDate. They’re provided as strings so you’ll need to parse these values!

      Hope that helps!
      Aaron

  3. Mads says:

    Hi Aaron,

    Loved the post!

    What’s your take on creating a connector with fx Facebook Ads/Insights + LinkedIn Ads/Insights?

    I feel like there aren’t that many templates out there available which are fulfilling my needs, so I’m looking into creating my own. You have any suggestions on what to be aware of and how to get started in the best possible way?

    Cheers!

    1. Aaron DicksAaron Dicks says:

      Hi Mads,
      Sorry for the late reply. To be honest, I’d recommend Super Metrics for most things, just STAT wasn’t on there!
      Hope that helps
      Aaron

Leave a Reply

Your email address will not be published. Required fields are marked *