Difference between revisions of "DATA FLOWS"

From Anitra Support
Jump to: navigation, search
(Created page with "{{DISPLAYTITLE: IN/OUT DATA FLOWS}} = (IN) Inbound data flows= A possibility of plugging to external telemetry data sources and having them fed into Anitra : ''Note the devi...")
 
Line 1: Line 1:
{{DISPLAYTITLE: IN/OUT DATA FLOWS}}
+
{{DISPLAYTITLE: DATA MANAGEMENT}}
  
 
= (IN) Inbound data flows=  
 
= (IN) Inbound data flows=  
Line 63: Line 63:
 
= (OUT) Manual data EXPORT=
 
= (OUT) Manual data EXPORT=
 
* allows manually exporting data from Anitra to CSV, XLSX, KML, and GPX formats for offline use
 
* allows manually exporting data from Anitra to CSV, XLSX, KML, and GPX formats for offline use
* data can be filtered by a date range
 
* the column editor feature allows modifying output columns in terms of columns list, column order, and naming of exported columns (a user preference)
 
  
 +
* export feature can be launched in various contexts / places of the platform:
 +
# '''Customer''' level (data integration section of customer profile) - full data for all ANITRA devices owned by Customer are bulk exported in this case. <br>[[File:export_customer_profile.JPG|500px]] <br>''Note having collected lot of data, this type of export is prone to fail - for mass downloading all your data we strongly recomend using our API functionality instead!.''
 +
# '''Device '''level  - the "Export" button in Device profile or using context menu in Device grid.<br>[[File:export_device_grid.JPG|900px]]
 +
# '''Tracking ''' level  - the "Export" button in Animal/Tracking profile or using context menu in Tracking grid<br>[[File:export_tracking_profile.JPG|900px]]
  
* export feature can be launched in various context in multiple places of the platform:
+
* exported data can be filtered by a date range
** Customer profile (data integration section) - full data for all ANITRA devices owned by Customer are bulk exported in this case (available for CustomerOwner and CustomerAdmins).
+
* the column editor feature allows modifying output columns in terms of columns list, column order, and naming of exported columns (a user preference)<br>[[File:export_edit_columns.JPG|900px]]
Note having collected lot of data, this export is likely to fail - for mass downloading all your data we strongly reomandd using our API functionality instead!).
 
** Device profile (Export button):''' (available for all Customer/Device Owner/Admins)
 
** Tracking  profile (Export button):''' (available for all Customer/Device Owner/Admins)
 
** Context menu from Device/Trackings list:''' all data for a single Device or Trackings (available for all Customer/Device Owner/Admins)
 
  
* /* mass data takeout not available for people with shared access only */
+
* Note the data export functionality is typicaly only available for device admins users (Customer/Device Owner/Admins). Users with shared access ("view only") are not supposed to download and store the records outside of the system. In case of need a "share with data export" can be used.
* Ref 1) Mass export ANITRA data on "Customer profile"
 
::::(Both CSV and formatted Excel sheet available):
 
:[[File:Export001.JPG|900px]]
 
 
 
* Ref 2) Export data for one particular device <br>[[File:Export002.JPG|900px]]
 
 
 
* Export columns specification:
 
:[[File:Export004.JPG|900px]]
 
 
 
* CSV file sample preview:
 
:[[File:Export005.JPG|900px]]
 
 
 
* XLSX file sample preview:
 
:[[File:Export006.JPG|900px]]
 

Revision as of 21:24, 2 November 2021


(IN) Inbound data flows

A possibility of plugging to external telemetry data sources and having them fed into Anitra

Note the devices imported from the data source are imported to your profile and listed in the device list. A unified metadata structure can be then created on top of the imported data (Device - Tracking - Animal). Ultimately data from various sources can be browsed and visualized side by side in a unified metadata-rich structure

Connectors for the following data sources currently exist:

  • Movebank
  • Ecotone
  • Ornitela

Inbound connections are managed in the "Integrations In" section. The overview of existing connections allows adding, editing/removal and running/scheduling the connection
Inbound connection overview.jpg

Use the "+create new" button to launch the connector creation wizard and enter the information required for running the feed

Note each of the connectors works differently and hence also the information needed for running the data flow and scheduling the run slightly differs.

  • Ecotone
    • details needed: Ecotone panel name (=URL of CSV files storage), user name, and password
    • user can manage this connector fully i.e. create, edit, run (once or schedule regular run), and delete
      Ecotone2Anitra setup wizard.JPG
  • Ornitela
    • details needed: Ornitela user name and password
    • for Ornitela the data integration relies on Ornitela SFTP functionality.
    • i.e. in this case user creates, edits the data connector, and requests activation (or requests deactivation) of the data flow
    • by requesting activation user also requests to have the SFTP configuration in his Ornitela pannel configured by us.
      Ornitela2Anitra setup wizard.JPG
    • Note some related aspects using SFTP:
      • only new data are integrated once the data flow is activated.
      • for importing older data (records created before the feed activation) use import functionality (any duplicate records are merged or skipped so you can import the same period repeatedly).
      • only the devices marked for SFTP export in the Ornitela panel will be automatically updated into Anitra
      • make sure you also update the connector definition anytime you are changing your Ornitela user password
      • also bear in mind that modifying the list of devices marked for SFTP (unchecking and checking again) will result in the data flow interruption and thus to some possible gaps in the data.


  • Movebank
    • configuring this connector a Movebank data + Movebank metadata are connected
    • Movebank API is employed for accessing the records
    • in this case user is supposed to enter a movebank username and password in combinatio with Movebank Study ID and request the data flow activation
      Movebank2Anitra setup wizard.JPG

(IN) Manual data import

  • allows importing data files in the original structure and format into customer/organization profile (e.g. Ecotone CSV)
  • usable e.g. for uploading older data files not available for automatic data flow anymore
  • import dialog can be accessed in the left-side application menu using the "Import" icon
  • for importing data in the particular customer profile (not in personal profile) make sure you chose one of your organization accounts in the "Upload in profile" selection
    Manual customer import.JPG

(OUT) API data access

  • allows grabbing Anitra data from external tools such as from GIS systems API interface
  • service currently available on request only
  • curent version of API make following item available
    • Device List
    • Device Data
    • Tracking List
    • Tracking Data
  • for detailed documentation and examples of API implementation refer to Anitra Public API documentation

(OUT) Anitra to Movebank data feed

  • Anitra to Movevebank data feed can be set up in the "integration out" section of the customer profile using the "+create new" button
    Movebank customer profile integrationOUT.JPG
  • a Movebank data owner username needs to be specified.
  • the account provided will be used to host all the synchronized data in Movebank
    Movebank customer profile integrationOUT setup.JPG
  • you can switch ON/OFF particular tag data synchronization by the "Export to Movebank" checkbox in the device profile.
  • by default the synchronization is inactive for all the new devices
    Movebank check deviceprofile.JPG
  • you can review the status of the check and the last time of synchronization of each of your devices at the device list


Movebank device list columns.JPG

(OUT) Manual data EXPORT

  • allows manually exporting data from Anitra to CSV, XLSX, KML, and GPX formats for offline use
  • export feature can be launched in various contexts / places of the platform:
  1. Customer level (data integration section of customer profile) - full data for all ANITRA devices owned by Customer are bulk exported in this case.
    Export customer profile.JPG
    Note having collected lot of data, this type of export is prone to fail - for mass downloading all your data we strongly recomend using our API functionality instead!.
  2. Device level - the "Export" button in Device profile or using context menu in Device grid.
    Export device grid.JPG
  3. Tracking level - the "Export" button in Animal/Tracking profile or using context menu in Tracking grid
    Export tracking profile.JPG
  • exported data can be filtered by a date range
  • the column editor feature allows modifying output columns in terms of columns list, column order, and naming of exported columns (a user preference)
    Export edit columns.JPG
  • Note the data export functionality is typicaly only available for device admins users (Customer/Device Owner/Admins). Users with shared access ("view only") are not supposed to download and store the records outside of the system. In case of need a "share with data export" can be used.