Skip to main content

Adding files from a URL

Need to import data from another site into data.world for easy sharing, collaboration, and querying? As long as you have a direct URL to a supported file type and the necessary access permissions, you can import data effortlessly. This method is ideal for pulling data from web sources, data portals, cloud storage apps, GitHub, and API endpoints.

Note

If you do not own the data you are importing, review the Licensing and data page to ensure proper usage.

To add files from a URL:

  1. Open the dataset where you want to add the files. Click the Add data button. The Add data from anywhere window opens.

  2. In the Add data from anywhere window, click the Sync from URL option.

  3. In the Add file from URL window, provide the URL of the data source and any authentication required to access the data. If the URL does not require authentication to access it, enter the shareable link provided by the data source into the source URL field and click Continue. The authentication options are:

    1. Username and password (HTTP): Provide the username and password required to access the source file.

    2. OAuth: This is the modern and more secure authentication used by many sites. Click one of the options next to the field to set the OAuth connection with your data source. Once the connection is established, you can select it from the dropdown added under the Authentication field.

    3. Token: This authentication type is for sources that require token based authentication.

    4. Set the Headers and POST body to make API calls to sites that support REST API.

    5. Click Continue.

  4. On the next screen provide the File name. Choose carefully, as this name cannot be edited later. Changing the name would require deleting the connection and creating a new one with the new name. Click Done.

  5. If the application encounters an issue with the source URL, an error message will prompt you to verify the link and settings. To view details about the error, hover over the ? icon next to the message.

    360021960054-mceclip3.png

    Click Edit to go back to the file window to fix the errors.

    360021960114-mceclip4.png

    In some cases, an authentication failure may return a 404 Not Found error instead of 403 Forbidden, even if the URL is correct. This behavior is a security feature of the API.

  6. Once your files are added, configure them to update regularly using the automated sync settings.Configuring sync options for datasets

In this section: