Skip to main content

Tools for checking network connectivity for collectors

This topic provides a comprehensive guide on using various tools within a Docker container to ensure connectivity and diagnose network issues:

  • Ping Utility: Checks basic internet connectivity by sending ICMP requests to external hosts. Note that it may not work if ICMP is blocked by a firewall.

  • Traceroute Utility: Maps the network path to a host, helping to identify routing issues when accessing a destination.

  • NSLookup Tool: Ensures DNS resolution is functioning, verifying domain names correctly map to IP addresses.

  • Curl Utility: Sends HTTP/HTTPS requests to test server or API endpoint connectivity. It is useful for diagnosing API-related connectivity problems.

Using ping utility to verify internet connectivity

Use the ping utility to verify that the Docker container has internet connectivity and can reach external hosts. Use this Reference guide for common commands for the ping utility.

Important

Ping will not work if your IT team has disabled ICMP through the firewall. A failure of ping is not necessarily proof positive that there is actually a connectivity issue.

To use the ping utility:

  1. Run the following command:

    docker run -it --rm --entrypoint ping datadotworld/dwcc:{version} {ping-options}

    1. Replace {version} with the appropriate version of the Collector.

    2. Replace {ping-options} with the host and any necessary options to reach the host.

    For example, the command could look like:

    docker run -it --rm --entrypoint ping datadotworld/dwcc:2.232 data.world

  2. Review the output.

    1. Successful pings indicate that the container has internet connectivity to the destination host.

    2. Failures indicate the destination host is not accessible for the container. A network administrator may need to investigate further network configuration, network access restrictions, or DNS issues.

Using traceroute utility to check network route connectivity

Use the traceroute utility to examine the network route from the Docker container to the destination host. Use this Reference guide for common commands for the tracerout utility.

Important

Traceout utility will not work if your IT team has disabled ICMP through the firewall. A failure is not necessarily proof positive that there is actually a connectivity issue.

To use the traceout utility:

  1. Run the following command:

    docker run -it --rm --entrypoint traceroute datadotworld/dwcc:{version} {traceroute-options}

    1. Replace {version} with the appropriate version of the collector.

    2. Replace {traceroute-options} with the host and any necessary options to reach the host.

    For example, the command could look like:

    docker run -it --rm --entrypoint traceroute datadotworld/dwcc:2.232 data.world

  2. Review the output.

    1. Successful routing indicates that the container has internet connectivity to the destination host.

    2. Failures indicate the full network path to the destination host is not accessible for the container. A network administrator may need to investigate routing issues or unexpected network paths.

Using nslookup to Test DNS Resolution

Use the nslookup tool to verify that DNS resolution is functioning correctly and that the Docker container can resolve domain names to host IP addresses. Use this reference guide for common commands for the nslookup utility.

To use the nslookup tool to test DNS resolution:

  1. Run the following command:

    docker run -it --rm --entrypoint nslookup datadotworld/dwcc:{version} {nslookup-options}

    1. Replace {version} with the appropriate version of the Collector.

    2. Replace {nslookup-options} with the host domain name and any necessary options to reach the host.

    For example, the command could look like:

    docker run -it --rm --entrypoint nslookup datadotworld/dwcc:2.232 data.world

  2. Review the output.

    1. Successful resolution indicates that DNS is working.

    2. Failures to resolve indicate that DNS is not working or the domain name is within an internal firewall. A network administrator will need to check DNS configurations and network access restrictions.

Using curl to Test HTTP/HTTPS Requests and Connections

Use the curl tool to test connectivity to specific servers or API endpoints by sending HTTP/HTTPS requests. Use this reference guide for common commands for the curl utility.

To use curl to Test HTTP/HTTPS requests and connections:

  1. Run the following command:

    sudo docker run -it --rm --entrypoint curl datadotworld/dwcc:2.232 --request GET --url <https://api.data.world/v0/datasets/catalog-sources/ddw-microsoft-sql-server-catalog-sandbox> --header 'accept: application/json' --header 'authorization: Bearer {DW_API_AUTH_TOKEN}'

    1. Replace {version} with the appropriate version of the Collector.

    2. Replace {curl-options} with the necessary options to connect to the server or API endpoint.

    For example, the command could look like:

    docker run -it --rm --entrypoint curl datadotworld/dwcc:2.232 --request GET --url <https://api.data.world/v0/datasets/catalog-sources/ddw-microsoft-sql-server-catalog-sandbox> --header 'accept: application/json' --header 'authorization: Bearer {DW_API_AUTH_TOKEN}'

  2. If you are using JAR files, run:

    curl  <URL>

  3. Review the results for errors or connection issues. Results will vary depending on the specific Request executed and the Response received may need further investigation.

In this section: