DocsData Gateways configurations
A Data Gateways configuration is a collection of outputs that define where data from session logs, published dashboards, and/or custom objects is sent. For each output, data can be sent to a file, HTTP endpoint, Splunk instance, or a database.
Add an output to a Data Gateways configuration
A Data Gateways configuration can consist of any number of individual outputs. The data from each of the outputs is added to a single configuration file.
- Click the System tab and select Data Gateways > Configuration.
-
Click Add new gateway output. The Data gateway output wizard displays.
- Enter a unique name for the output.
- Select an output type, File, HTTP endpoint, Splunk, or Database and complete the fields for that output type.
- Click Next.
- Select the data to send to the configuration file for the specified output type:
- Session logs – Data Gateways will process session log data for the specified output type.
- Published dashboards – Data Gateways will process data from the configured published dashboards for the specified output type.
- Work queue analysis snapshot data – Data Gateways will process the work queue analysis data for the specified output type.
- Custom object data – Data Gateways will process data from any Blue Prism action configured to use the Data Gateways internal business object.
-
Click Next. A preview of the output data displays.
The data in the produced output can be edited directly by clicking Edit advanced output. For more information about advanced configurations, see Advanced outputs.
-
Click Finish to save the output to the configuration. A Data gateways message displays.
- Click OK to close the message. For more information on restarting the data gateway process see Start and stop the Data Gateways engine.
Output types
File
Export the selected data to the specified file. Although creating outputs in the Data Gateways Configuration wizard only supports sending data to .txt files, advanced outputs and custom configurations can be edited to support other file formats, such as .csv.
For details about how to achieve this, see Configuration file structure.
Dynamic file paths can be used to include the creation date in filenames using variables for year, month, day, and time.
For example: C:\Session Logs\session_logs_%{+YYYY-MM-dd}.txt
This will create a new text file each day in the specified location, that includes the current date in the filename: session_logs_2019-04-30.txt
The variables can also be used in file paths to create date-based folder names. For example:
C:\Session Logs\%{+YYYY}\%{+MM}\%{+dd}\session_logs_%{+YYYY-MM-dd-hh}.txt
This will create folders based on the current date and append the filename. For example:
C:\Session Logs\2019\04\30\session_logs_2019-04-30-11.txt
The file/folder timestamp generated when using dynamic file paths is in UTC format.
Dynamic file paths can be used in configurations created using the data gateways wizard and advanced configuration.
HTTP endpoint
Route data through the specified HTTP endpoint using the selected method. Select a Data Gateways credential if authentication is required to access to the endpoint.
Splunk
Send the selected data to the specified Splunk URL using the associated API token.
To successfully configure a Splunk output, the HTTP event collector must be enabled in your Splunk configuration. For further details, see https://docs.splunk.com/Documentation/Splunk/7.3.1/Data/UsetheHTTPEventCollector.
Database
Send the selected data to the specified SQL server database table. Select a Data Gateways credential if authentication is required to access to the database.
To use a port, other than the default port, the server name can be appended with an alternative port number, separated by a colon. For example, bp-server-1:8001.
Advanced outputs
When creating or editing an output, users with the Data Gateways – Advanced Configuration permission can edit the code in the output directly to create an advanced configuration.
From the output configuration preview screen of the output wizard, click Edit advanced output to open the advanced editor.
The output can be edited by entering data or pasting it from the clipboard. Files up to 10k in size can also be imported into an output, replacing all current data. When the required changes have been made, click Save advanced output.
Once an output has been saved as an advanced output, even if no changes have been made, it can no longer be configured in the Data Gateway output wizard – only in the advanced editor.
Manage a Data Gateways configuration
The outputs for a Data Gateways configuration are listed in the Data Gateways > Configuration screen. Click Manage to display the options to copy, edit, or delete one or more configurations.
Every time a Data Gateways configuration is changed, whether that is new, updated, or deleted output, the Data Gateways engine must be restarted. For more information, see Start and stop the Data Gateways engine.
Copy an output
Click Copy for the output you want to duplicate. Depending on the output type, the Data Gateways wizard or advanced editor will open enabling the duplicated output to be updated as required. When the output has been saved, restart the Data Gateways process.
Delete an output
Click Delete for the required output to remove it from the configuration and restart the Data Gateways process.
Edit an output
Click Edit for the required output. Depending on the output the output wizard or advanced editor displays allowing the output to be updated. When the output has been saved, restart the Data Gateways process.
Delete selected output
Click the Select all check box in the Output name column to select all of the outputs for group deletion. Select/deselect the check box for a specific output, as required. Click Delete selected to delete all selected output(s).
Custom configurations
Users with the Data Gateways – Advanced Configuration permission can create a custom configuration by directly editing the underlying code. This allows users to create a configuration that has functionality beyond what is available through the wizard such as the ability to only send specific event fields to an output or customizing the format of the data that is sent to an output.
For further information about the composition of configuration files, see Configuration file structure.
Create a custom configuration
-
From the Data Gateways – Advanced Configuration click View configuration
The configuration code displays.
- Click Edit configuration to make the configuration code editable.
-
When the required updates have been made, click Save and confirm.
The Data Gateways engine must be restarted for the changes to take effect. For more information, see Start and stop the Data Gateways engine.
The custom configuration overwrites the outputs in the existing one. This is reflected in the Data Gateways screen, where the advanced configuration code replaces the list of outputs.
Click Delete in a custom configuration at any time to revert the configuration to its original composition and display the original outputs.
