Decipher IDP 2.3: January 2024

Users can now configure an autosave document threshold, which automatically saves the currently loaded batch after the configured number of documents have been reviewed. This feature is disabled by default, and if enabled, can be configured to save after a minimum of 100 documents, and a maximum of every document. Learn more...

Batch files can now be downloaded from the Batches page in the Admin Panel, enabling users to view the input files and troubleshoot exceptioned batches. Learn more...

Users can now add a new page to a document during class verification. Only PNG files are accepted, and can either be dragged and dropped into the browser, or selected from a folder. Learn more...

A new Decipher VBO is available as part of this release, extending the capabilities available in Blue Prism processes. Learn more...

The new VBO does not include the obfuscateFileName functionality included in the existing VBO. As such, it is the user's responsibility to ensure file names do not disclose information which could be considered sensitive.

Decipher IDP can now be integrated with Microsoft Entra ID, previously known as Azure Active Directory, enabling users to log into Decipher IDP using their Mircosoft Entra ID credentials. Learn more...

A new accuracy report, Performance, has been added which can be used to determine the accuracy returned by Decipher IDP for particular batch and document types. The report provides a granular breakdown of how many fields were captured correctly, required manual intervention, or captured incorrectly. Learn more...

Exceptions are now marked at the document level, rather than the batch level. This means that if a batch contains an invalid document, for example, an unsupported PDF, only that document is flagged as an exception and the rest of the batch can be processed. Previously, the whole batch was marked as an exception.

The Decipher Server Plugin is not required for this version of Decipher IDP. As a result, the Decipher Server Configurator, which was used to modify the RabbitMQ credentials of the Server Plugin, is no longer required.

Support for capturing multi-line fields has been improved, where previously multi-line data could only be correctly recognized if it was in a fixed position or if validation logic was in place. Now, additional elements, such as header values and nearby content, are used when attempting to identify multi-line content.

Multiple enhancements to how training data is managed and maintained in the system have been implemented. The changes include:

• Training data is now selectively updated in the database to improve performance in systems with large training data sets. Previously, the entire training data index was rebuilt when identifying and updating one training data set.

• The process of merging training data, where conflicting training data from multiple users is processed and combined into the existing data set, has been optimized. Previously, the merge operation caused bottlenecks in the training data process for systems with large training data sets and multiple users processing documents.

• Decipher IDP now automatically deletes old training data items, removing the need for administrators to manually maintain systems with large training data sets. Users can define the maximum number of training items retained in the system, and how long redundant items are kept before they are automatically deleted. This is controlled by two new parameters, MaxTrainingDataItemCount and MaxTrainingDataItemInactivityDays, which are defined in the Decipher Server configuration file. Learn more...

• Training data can now be locked at the document form definition (DFD) level, enabling users to prevent additional training data accumulating after suitable training is in place. Training data can be locked for individual fields, or for the entire DFD. Learn more...

• Training data can now be separated by DFD, making it easier to manage large training data sets. This functionality can be enabled by selecting the Segregate training option on the Document form definitions page. Learn more...

Decipher IDP now supports the following Asian languages for OCR extraction: 

• Chinese – Simplified

• Chinese – Traditional

• Japanese

• Korean

• Vietnamese

To support these new languages, the following new locales have also been added to handle the processing of dates, numbers, and currency:

• China

• Hong Kong

• Japan

• Korea

• Singapore

• Taiwan

• Vietnam

The new languages and locales can be configured when setting up batch types in the Admin Panel.

The logic of the miscellaneous parameters TopStop and BottomStop has been enhanced, with updates including improvements to handling longer phrases, and how the parameters work in conjunction with the Updated Table Detection (UTD) mode.

The Decipher Web Client no longer utilizes RabbitMQ for internal communication between components within the client, and has been replaced with TCP communication. The components within the client, called SDK Wrappers, are always contained within the boundaries of the same machine, and as such, do not require RabbitMQ for communication.

When importing batch types, if a batch type with the same ID already exists in the system, users now have the option to overwrite the existing file. This replaces the batch type in all instances where it is used, such as in document types and DFDs. A message presents to users during the import process, where they can view any existing objects, and overwrite if required.

A new AuditLogEvent table has been added to the DecipherServerDBReporting database, which stores logs for all system events and includes details such as EventAction, EventTarget, UpdateType, and PerformedByUserID. The logs can only be viewed in the database by administrators.

In addition, new configuration file parameters, AuditReportingLevel and RemoveAuditLogsOlderThanDays, have been added to enable users to define the log level, and how long logs are retained. The parameters are configured in the Decipher Server configuration file.

Learn more...

To access this feature, the Decipher IDP system must have a separate reporting database, initially configured by selecting the Enable reporting database checkbox when installing the Decipher Server.

Enhancements have been made to the StrictPosition miscellaneous parameter to improve capture rates in instances where the document layout has changed, or if the region has expanded to accommodate longer text strings.

In addition, the following miscellaneous parameters have been added to extend the functionality of the existing StrictPosition feature:

• StrictPositionAnchorText – Used to specify the position of a region based on text with a fixed distance from the target region.

• StrictPositionBottomBoundary – Used to specify the position of a region based on text below the region.

• StrictPositionRightBoundary – Used to specify the position of a region based on text to the right of the region.

• StrictPositionTopBoundary – Used to specify the position of a region based on text above the region.

• StrictPositionLeftBoundary – Used to specify the position of a region based on text to the left of the region.

Learn more...

Region detection has been updated with the following features to improve the system's ability to detect and outline document text regions:

• Regions are now separated based on graphical lines that intersect them.

• Region segmentation for images with low contrast and JPEG compression has been improved.

• The detection of barcode and checkmark fields has been enhanced.

• The handling of overlapping regions has been improved.

RegEx expressions for non-standard date formats can now be programmatically added into Decipher IDP, enabling developers to extend the supported data formats in the system.

Two new miscellaneous parameters have been added, DateFormat and DateLocale, which can be used to specify non-standard date formats for specific document types. This improves the capture of date values, ensuring they are correctly validated and interpreted. Learn more...

Users can now define keywords during data verification, enabling them to dynamically update the DFD during the verification process. Once added, a toaster message displays with an Edit keywords option, allowing the user to view and edit all keywords defined for the field. Learn more...

New user management logic has been introduced to add more granular user permissions, and to closer align with roles in Blue Prism Enterprise. The changes include:

• Permissions are now predominately set at the user group level, and users inherit the permissions assigned to their user group. As such, the following changes have been made when adding users and user groups:

  • The Manage roles field has been removed when adding and editing users and user groups.

  • The Manage permissions field no longer displays by default when adding and editing users. This is determined by a new Show permissions settings option on the Users page.

• Added new permissions:

  • ManageUsers

  • ManageDocumentConfiguration

  • ManageBatches

  • DownloadBatchInputFiles

  • ManageSystemConfiguration

  • ManageSystemConfigurationWorkflow

  • ManageSystemConfigurationRetention

  • ManageSystemConfigurationTimeout

  • ManageSystemConfigurationSso

  • ManageClassificationModels

  • ManageCaptureModels

  • ManageTrainingData

  • ManageActiveClients

• Added new pre-defined user groups:

  • Decipher IDP Specialist

  • General Users

  • Input Users

  • Ocr Users

  • Process Controller

  • Release Manager

  • Reports Users

  • Runtime Resource

  • Support Analyst/Verifier

• With the introduction of more granular permissions, the Admin permission is no longer required to access the Admin panel. Users can access the Admin panel and view tabs they have been assigned permissions for, without being granted access to all Admin functions.

• A new menu has been added to the Users page, containing the options: 

  • Show service users – Determines if service users, such as the Automated Client users, display in the Users list.

  • Show permissions settings – Determines if the Manage permissions field displays when adding users, enabling permissions to be defined at the user level.

Learn more...

To make it easier for users to configure formulas on a DFD, the Formula editor on the DFD configuration page has been updated with the following improvements:

• Bracket pairs are highlighted.

• Automatic indentation while typing.

• Automatic formula formatting, such as adding new lines and spaces between operators.

• When an element is selected, other instances of the element are highlighted.

• A formula menu enables users to select a function to insert at the current position. Available functions are grouped by functionality, for example, Text, RegEx, and Dates.

• A tooltip displays on hover to show the function, format, and any supported parameters.

• Comments and syntax is correctly highlighted and colored.

In addition, developers are now able to create .NET formula plugins to address specific formula requirements in their systems.

To improve the user experience, various improvements have been made to the DFD and data verification interfaces. In addition, the table re-sizing logic has been improved to ensure adjusted table column widths are retained.

Added the ability to filter training data when exporting or deleting training data sets. Training data can be filtered by:

• Batch type

• Document type

• Created before

• Created after

• Updated before

• Updated after

• Created by user

• Unique field value

Learn more...

Users can now capture data from table rows that span across page breaks. To add the section of the row that is on the next page, users can hold the Shift key while clicking the text region. The region is then marked with a dotted line to indicate the data continues on another page.

The Updated Table Detection (UTD) mode has been updated to improve handling of complex tables.

New single sign-on (SSO) options have been added to the Configuration tab of the Admin panel, making it easier for users to set up SSO without the need to manually edit configuration files. Learn more...

During data verification, tables in the Form View now support multi-line table cells.

Training data for a specific document or page can now be deleted during data verification. Two new options, Delete training for this page and Delete training for this document, are available when right-clicking a page thumbnail and selecting Training data. Learn more...

When viewing the Batches tab in the Admin Panel, batches that are marked with an exception now contain a list of the invalid files in the Description column. Learn more...

A new Reprocess batch option has been added during Data Verification, enabling users to send the current batch back for additional processing. Learn more...

A new ReadOnly flag has been added to the DFD configuration options. When this flag is enabled, the field cannot be selected during data verification. Learn more...

Decipher IDP now supports single log out. Users are now automatically logged out of Decipher IDP when they log out of their identity provider.

Page auto-orientation has been improved by using additional data from the OCR engine to determine page orientation. Previously, this mechanism relied heavily on character-level confidence information returned by the OCR engine, which was subject to inconsistencies.

Specific versions can now be configured for the following additional DFD settings:

• Keywords

• Data type

• RegEx

• Formulas

• Table detection mode

This extends the previous functionality which was limited to field values only. This provides more flexibility for systems processing documents from multiple vendors, and removes the need to configure vendor specific DFDs.

The Classification client can now mark training batches with an exception, and return an exception reason to the user. Previously, the client failed if processing a training batch with an exception.

An issue has been corrected during data verification where checkboxes did not display next to table items which were assigned the Checkable flag.

An issue has been fixed where Decipher IDP crashed with an exception when configuring a new DFD if a space character was entered when defining a specific version. Now, an error message displays, prompting users to enter a value.

An issue has been fixed where, if a DFD field label was triple-clicked to highlight all values in a field, the highlighted text could not be over-typed. Now, a user can triple-click a field and enter a new value as required.

An issue has been fixed where the Auto-calculate flag did not work as expected for fields which also had the Multiline flag assigned.

An issue has been corrected where, when installing the Decipher Server, the Back button on the Destination Folder screen incorrectly returned the user to the Welcome screen. Now, the End-User License Agreement screen displays as expected.

Users are now prompted to re-enter their single sign-on credentials when logging into Decipher IDP if they have just logged out. Previously, credentials were cached by the browser and users did not need to be re-authenticated.

Users are no longer logged out and redirected to the Decipher IDP login page after enabling the single sign-on in the Admin panel.

An issue has been corrected during data verification where checkboxes did not remain checked if a batch was returned and re-loaded.

When adding a new row to a table during data verification, any enabled checkboxes now remain checked if a new row is added, or an existing row is removed.