Skip to main content

Preparing to run the BigQuery collector

Setting up pre-requisites for running the collector

Make sure that the machine from where you are running the collector meets the following hardware and software requirements.

Table 1.

Item

Requirement

Hardware

RAM

8 GB

CPU

2 Ghz processor

Software

Docker

Click here to get Docker.

Java Runtime Environment

OpenJDK 17 is supported and available here.

data.world specific objects

Dataset

You must have a ddw-catalogs (or other) dataset set up to hold your catalog files when you are done running the collector.



Setting up authentication for BigQuery

data.world connects to BigQuery using a Service Account associated with your project.

To set up authentication for BigQuery:

  1. Create a service account with the following roles BigQuery Data Viewer and BigQuery User. For additional information on predefined roles and permissions, see Google Cloud Platform documentation.

  2. After you create a service account, create a key for the account and download the associated JSON key file.

    create_json_key.png
  3. Place this key file on the machine from where you plan to run the collector. You will need this file while running the collector.