Skip to main content

Running the Oracle collector in Cloud

Configuring the cloud collector for Oracle

To configure the cloud collector for Oracle:

  1. On the Organization profile page, go to the Settings tab > Metadata collectors section.

  2. Click the Add a collector button.

    add_a_collector.png
  3. On the Choose metadata collector screen, select the correct metadata source. Click Next.

  4. On the Choose where the collector will run screen, in the Cloud section, select data.world. Click Next.

    select_cloud.png
  5. On the Configure a cloud Oracle Collector screen, set the following:

  6. On the next screen, set the following properties and click Next.

    Table 2.

    Field name

    Description

    Required

    Server

    The hostname of the database server to connect to.

    Yes

    Server port

    The port of the database server (if not the default).

    No

    Database

    The name of the database to connect to.

    Yes

    Username

    The username to use to make the JDBC connection.

    Yes

    Password

    Specify this option to provide the password for the database on the command line.

    Yes

    Schemas to collect

    Select from one of the following options: Collect all schema, Specify which schema to collect

    Yes

     

     

    Collect all schema

    Catalog all schemas to which the user has access.

    Specify which schema to collect

    The name of the database schema to catalog.



  7. On the next screen, set the following optional properties and click Next.

    Table 3.

    Field name

    Description

    Required?

    Database is an Oracle SID

    Indicates that the database is actually an Oracle SID.

    No

    Disable lineage collection

    Skip harvesting of intra-database lineage metadata.

    No

    Enable sample string values collection

    To enable harvesting of sample values and histograms for columns containing string data.

    No

    Enable column statistics collection

    To enable harvesting of column statistics (i.e., data profiling).

    Note: Activating the profiling feature may extend the running time of the collector. This is because the collector needs to read the table data to be able to gather metadata for profiling.

    No

    Target sample size for column statistics

    To control the number of rows sampled for computation of column statistics and string-value histograms. For example, to sample 1000 rows, set the parameter as: --target-sample-size=1000

    No

    Server Environment

    If your provided server name is localhost, use this to give a friendly name to the environment in which your database server runs to help differentiate it from other environments.

    No

    Database ID

    A unique identifier for this database - will be used to generate the ID for the database (this is optional, you only need to provide this if the database name used for the connection is not sufficiently unique to completely identify the database)

    No

    JDBC Properties

    JDBC driver properties to pass through to driver connection, as name-value pair.

    No



  8. On the next screen, provide the Collector configuration name and set the run schedule. You can also set the schedule at a later point.

  9. Click Save and View to go the collector details page.

Scheduling collector runs

Important things to note:

  • Different collectors can be scheduled to run at the same time, but one collector can only run once a day.

  • It is recommended that you schedule the runs in off-peak hours.

  • The collector runs in the timezone in which the scheduler is located. For example, if the scheduler sets the collector runs from PST timezone, the collectors will follow the PST timezone.

  • Runs may start up to one hour after the scheduled time.

To schedule collector runs:

  1. On the Configured collectors page, locate the collector you want to run on a schedule.

  2. Click the Edit configurations button.

  3. Go to the screen where you can set the schedule for the collector.

  4. Enable the Scheduled runs option.

  5. From the Frequency dropdown, select from Daily, Weekly, or Monthly.

  6. For Weekly and Monthly options, select the day when the collector should run.

  7. Select the time for running the collector.

  8. Click Save and view. The schedule and next run date and time are displayed on the collector details page.

  9. To get notifications about the collector runs, simply setup web hooks at the Organization level from the Organization profile page > Settings tab. The Webhooks will automatically start capturing the Status events (Pending, Provisioning, Running, Completed, Error, Cancelled) for the collector runs.

    org_webhooks.png

    Sample data captured by the webhook.

    hooks_notfication_collectors.png

Running collectors manually

After setting up the collector configuration, it's advisable to manually execute it once to ensure correct configuration. Even collectors that are scheduled to run automatically can be manually initiated at anytime.

To run the collectors manually:

  1. On the Configured collectors page, locate the collector you want to run.

  2. On the collector configuration details page, click the Run now button. Alternatively, on the Configured collectors page, click the Three dot menu and click Run/Sync now button.

  3. On both pages, the Status field shows the status as Running with information about time elapsed since the run was started.

    The collector starts running in the background and you can navigate away from the page at any time. For a long running collector, if the collector run does not complete in a weeks time, the collector run automatically terminates after one week. The Status section and the Status field update to an Error state.

  4. After the collector has completed the required pre-configuration steps and starts harvesting the metadata, you get an option to Cancel the harvesting process, if you want. The Status section and the Status field update to Cancelled.

  5. After the collector run has completed, the Status section of the collector configuration details page updates to show the successful status. The Last run summary page also updates to show the total number of resource collected and total number of types of resources collected. The Resources collected by type gives granular level information about the number of resources collected for each type of resource.

  6. Browse to the Collection and Dataset specified while running the collector to view the collector output.

  7. To get notifications about the collector runs, simply setup web hooks at the Organization level from the Organization profile page > Settings tab. The Webhooks will automatically start capturing the Status events (Pending, Provisioning, Running, Completed, Error, Cancelled) for the collector runs.

    org_webhooks.png

    Sample data captured by the webhook.

    hooks_notfication_collectors.png

Copying collector configurations

After you have configured a collector for a source system, you can easily create a copy of the configuration to configure another collector for the same source system but for different parameters.

To copy collector configurations:

  1. On the Configured collectors page, locate the collector configuration you want to copy.

  2. From the Three dot menu, click Duplicate configuration.

  3. In the Edit Collector window, provide a new name for the collector configuration. Optionally, set a schedule. Click Save and view.

  4. You are taken to the copied collector configuration page. Click the Edit Configuration button to adjust the details of the configuration.

Deleting configurations

Important things to note:

  • Deleting the configuration will not affect the resources that were collected from previous runs.

  • Any scheduled future runs for the collector are suspended.

To delete a configuration:

  1. On the Configured collectors page, locate the collector configuration you want to delete.

  2. From the Three dot menu, click the Delete configuration button.

  3. Confirm the deletion. The configuration is deleted and removed from the Configured collectors page.