Historical Data

Overview

Store Leads maintains archives of store data in all of our weekly data updates going back to early 2019. Live searches can be run against historical data or you can download the raw data files to analyze it yourself (which can be helpful if you want to run arbitrary queries in an analytics database.)

Note: Access to Historical Data is only available on the Enterprise plan.

Search

To search against historical data, use the Dataset field to select the historical data for the search. Note that this field is only visible to customers on the Enterprise plan.

When a historical dataset is selected, the field will have an orange border. This is a visual reminder that searches are being run against historical data. Switch the field back to "Current" to run searches against the current weekly dataset.

Once a dataset has been selected, you may search as normal. Basic and Advanced searches are both supported.

Search Limitations

While searches against historical datasets support most of the same features as searches against the current dataset, there are some important differences to note.

The following features are not supported in searches against historical data:

• Sorting by the Merchant Name field.
• Filtering by integration information (ie. Salesforce, Hubspot)
• Filtering by custom matches
• Filtering by custom search filters
• Filtering by manual lists (filtering on saved search lists is supported)
• Filtering App Reviews, App Review Ratings and App Reviwe Count
• Advanced filters using regular expressions.

Finally, searches that query against text fields (e.g, query, meta description and meta keywords) may return slightly different results.

Viewing Domain Details

Searches against historical datasets show historical values directly in the search results. As normal, the columns displayed in search results can be customized.

Click on a domain in the historical search results to see additional information for each domain as they appeared in the selected historical dataset.

A notice is displayed at the top of the page as a reminder that historical data is being viewed. Switch back to the Current dataset to see the most recent data for the domain.

The CRM integration tabs are disabled when viewing data from a historical domain to prevent accidentally sending historical data, which is likely out of date, to your CRM.

Attribute Time Series

When viewing domain details, click on the Historial tab to see a time series of changes to specific domain attributes.

Export

Exporting a search against historical data works the same way that it does for regular searches. Simply run a search against historical data and click the Export button.

One thing to note, exports against historical data can take longer than exports against the current data. If you are exporting a large volume of historical data (for instance, to ingest it into an analytics database), consider accessing our raw data files instead.

Exporting Historical Counts

Sometimes, rather than exporting data for individual domains in historical exports, you just want to know how many domains matched a given search over time. This is often useful when trying to assess growth of a particular segment (for instance, the number of Shopify Plus domains outside of the US over time).

To export historical counts, simply run a search, click on the Export button and select Export Historical Counts.

Raw Data

Raw data files contain a list of all stores in each historical dataset. Files are provided in JSONL format and can be accessed through our API, downloaded to your local computer or transferred to S3 or GCS.

API Access

Refer to our Historical Data API documentation for information on how to retrieve the data files through our API.

Download

To download historical data files to your local computer, start by visiting the Historical Data page in the Dashboard. Note that the features documented below are only visible to customers on the Enterprise plan.

Click the Download button next to the dataset that you'd like to download. You'll be asked to provide an optional list of fields to include in the dataset. Leave this blank to include all of the default fields returned by the API. If you only need a subset of fields in the file, it is recommended to list the fields required since it can significantly reduce download time. See the API documentation on fields for more information.

Transfer to S3

To transfer a raw data file to Amazon S3, be sure to enable the S3 Integration. With the integration enabled, click the Transfer to S3 button to initiate a transfer.

Provide the S3 URL and an optional list of fields to include in the dataset. Leave the Fields input blank to include all of the default fields returned by the API. If you only need a subset of fields in the file, it is recommended to list the fields required since it can significantly reduce transfer time. See the API documentation on fields for more information.

Transfer to GCS

To transfer a raw data file to Google Cloud Storage, be sure to enable the Google Cloud Storage Integration. With the integration enabled, click the Transfer to GCS button to initiate a transfer.

Provide the GCS URL and an optional list of fields to include in the dataset. Leave the Fields input blank to include all of the default fields returned by the API. If you only need a subset of fields in the file, it is recommended to list the fields required since it can significantly reduce transfer time. See the API documentation on fields for more information.