Links
Comment on page

Configuration

This section explains each of the properties in the configuration blade.

General

Property
Description
Collection
This associates the Agent to a specific Collection (default to that of the current Data Stream).
Polling Interval (seconds)
A timespan to specify how often should the Agent check for new data/changes or update its cache.
Refer to Collections and Stream Hosts to understand more about Collections.

Website Details

Property
Description
Url
The URL of the Web Site to scrape.
License Key
The IronWebScraper license key.

Selectors

Property
Description
Container Selector
The CSS Selector that identifies the first element node to match in the HTML document, which locates the container.
Item Selector
The CSS Selector to filter the node retrieved from the Container Selector to locate the elements.
Elements Grid (Name, Selector, Type)
Define the elements to scrape and how they will appear in the output payload.
  • The attribute name should combine words or use a _, rather than a space.
  • The selector of the HTML element to retrieve within the Item Selector node.
  • The attribute data types used for the output payload are String, Long, Double, Boolean, DateTime, or Integer.
For more details regarding CSS Selector, click here.

Endpoints

Name
Description
Output
An event per poll will be made available on this endpoint, with attributes as defined in the Elements Grid.
Error
Errors encountered during the execution of this Agent will be published on this endpoint.