Your first data source in eagle.io

So you have just got your trial account, you’ve logged in, and you're wondering, “What now?” This guide will step you through the connection of a live data source and auto-creation of charts. 

In this example, we will pull real-time data from a public domain United States Geological Survey (USGS) rain gauge in Charlotte, North Carolina. 

Live source of data: CRN-15 Raingage at Charlotte-Mecklenburg Govt Cntr (USGS).

Navigating eagle.io

Starting with your fresh instance of eagle.io, you will see on the far left-hand side the workspaces tree (Item 8). This is where all your nodes will be organized. Nodes include managed accounts (if used), workspaces, locations, dashboards, and data sources. There is a hierarchy involved, but it is intuitive. Accounts are found at the top and contain everything. 

A schematic showing key eagle.io elements is provided in Figure 1, and a legend in Table 1.  

Figure 1. Layout of eagle.io in Map view

Table 1: Legend

Create a Workspace

Outcome: Create a workplace (top-level node) to store your location, data source, and custom charts (think of this like a project folder in Windows Explorer). 

1. Rename “My Workspace” in the workspaces tree to “My First Datasource” by right-clicking.

Figure 2. Workspace creation

Create a Location 

Outcome: Accurately place a map marker at the monitoring station location.

2. In the workspaces tree, open the dropdown menu next to the "My First Datasource" workspace, hover over “Create”, and then select “Location.” 

3. Open the dropdown for location and select “Properties.”  

4. Under the “General” tab, enter the latitude 35.22138889 and longitude -80.8397222 and save.

Figure 3. Location creation

Create a data source 

Outcome: Connect a live source of data to eagle.io, which we can use to generate alarms, calculate new values, and develop custom charts.

5. Open the dropdown menu next to the location node where the data source is to be located. Navigate to Create > Source > Data. This will open the Data Source Wizard.

Figure 4. Data Source creation

Source Type

6. In the wizard that appears, select the “File” option and click next.

How will the live data from the Charlotte Weather Station be acquired by eagle.io? This will be configured using the “Transport Type.” You will see the following:

Figure 5. Source type selection in the Data Source Wizard

Set the Connection, Collection, and Time values in the Data Source Wizard, as shown in Figure 6. 

7. Select “Download via HTTP.”

8. USGS data is transmitted every hour, so set the collection interval to 1 hour and your “Initial Collection” to “Collect All Data” as you want to grab as much historical data as you can for the first download. 

9. “Timezone” is set to the timezone of the gauge: Eastern Time.

Figure 6. Configure source

Now you are set up to start ingesting data!

Data Parser

To configure the parser, eagle.io will ask for a sample file. In this case, the URL for the HTTP download is requested. The URL we will use can be copied below:

http://waterservices.usgs.gov/nwis/iv?format=rdb&sites=351320080502645&period=P1D&modifiedSince=PT30M

In the data parser, eagle.io will make some automatic suggestions, but the file still needs a few settings changed. 

Figure 7. First look at the Parser

 10. Switch the column delimiter to tab:

Figure 8. Parser after switching to tab delimited

11. The labels row is to be row 27. You can use the dropdown at the top, and note that row 27 is now highlighted in purple:

Figure 9. Parser after setting the labels row to 27

12. Use the dropdown to set the data type of the third column (labeled “datetime”) from "Number" to “Record Time”. This will inform eagle.io which column contains timestamps for the sample data. 

13. Click the settings cog icon of the "datetime" column to update the date format from YYYY-MM-DD HH:mm:ss to YYYY-MM-DD HH:mm (to match the format of the sample data):

Figure 10. Parser after setting the time format

14. Disable the “agency_cd”, “site_no.”, “tz_cd” and “90521_00045_cd” columns by clicking each column’s dropdown and selecting “Disabled.” These are metadata fields that don’t change over time:

Figure 11. Parser after disabling columns

15. After all the adjustments, click “Apply”, then “Next.”

More help configuring files can be found here: Data Source (file) — eagle.io documentation.

Timestamps configuration can be found here: Timestamps — eagle.io documentation.

Configure Parameters

16. Each enabled column in the sample data will now be assigned to a new parameter. You can change the new parameter names if the column names in the header row of the raw file are not suitable. For example, the column name of "90521_00045" is not very descriptive, so in the screenshot below we have instead named this parameter "Rainfall". Units can also be specified. Since we are importing our first dataset, each parameter should be set to “New” under “Parameter Assignment.” Now we can click “Finish,” and everything should start to run!

Figure 12. Configuring parameters

Under the “Events” tab at the top, you should see a list of events that just occurred creating the source (this may take 10-30 seconds).

Your instance of eagle.io now has data! 

Click on the “Charts” tab in the top right to view plots of each parameter you have ingested. As currently configured, eagle.io will poll the USGS URL every hour, updating the data and charts (whether logged in or not).

From here, you can create alarms, custom charts, dashboards, and more.