Data requirements and mapping

Data structure

While uploading data to Process Intelligece, it is important for a file to have a specific structure. The data file should consist of rows, each of them representing a record that something happened to a specific object at a particular time. The first line defines the fields names. Other rows contain records in the order related to the fields names. Such information can be extracted from IT systems, databases, or logs.

The file must have three mandatory columns, related to Timeline ID, Timestamp, and Event name and can include any number of optional columns. All columns can have arbitrary names, as no naming rules are imposed.

Data structure sample

TimelineID	Timestamp	Event name	Employee	Location	Comment
N008	1/16/2017 7:20:15	Ticket Registered	John Smith	Charlotte	Insurance claims
N123	3/10/2017 16:54:10	Ticket Closed	Anna Brown	Boston	Refund

Mandatory and optional fields

These columns can be named arbitrarily in the uploaded file.

• Timeline ID – A column for some identifier of an object you want to track over time. This could be an Order ID, Claim ID, Patient Encounter Number, Support Ticket Number, and so on.
• Event Name – A column describing what happened to the object at that time – Order Submitted, Patient Departed, Adjuster Assigned, Ticket Escalated etc.

• Timestamp – A column for the timestamp showing when something happened in the life of the object. This column generally contains a date and time (see format description below). If a date with no time is provided, midnight (12:00 AM, 00:00:00) will be used.

  Make sure you save timestamps in the file in one of the following formats:

  • 1/6/2017 7:20:15
  • 1/6/2017 7:20:15 AM
  • 2017-01-06 7:20:15
  • 2017-01-06T7:20:15Z

Other columns are optional and become attributes for timelines. For example, you can map a column with the office location to see what processes occur in this place mostly.

File format

The data should be placed into a CSV file which typically stores tabular data in plain text. In this file, no two columns should have the same name. The file should be Locale English (United States) and US ASCII or UTF-8 encoded.

File sample CSV

TimelineID;Timestamp;Event name;Employee;Location

A;1/16/2017 7:20:15;Student Applied;John;Boston

A;03.10.2017 16:54;Student Accepted;Mary;Boston

A;04.11.2017 15:04;Bill Generated;Ann;Charlotte

B;02.01.2017 9:15;Student Applied;John;Boston

B;03.02.2017 16:20;Student Accepted;Mary;Boston

If the values in any field include commas, the format of the file may break. In this case, use double quotes to specify a new string. For example, Microsoft Excel does it automatically, however, some tools like MS SQL Export Wizard require manual settings.

Mapping data

Once you have uploaded your file, you need to map the data. This is the process of associating data received from IT systems with attributes that display in Process Intelligence. For example, map a mandatory column Timestamp so that the application pulls timestamps for the processes from this column. Or map a column with employees' names using a New attribute label to make a breakdown by dimensions and see who is in charge of specific processes.

Process Intelligence imports only mapped table columns as event attributes. Fields without mapping labels are not uploaded to the project.

Map fields

At the Map columns step, you create a mapping for the uploaded data. Drag and drop labels to the respective columns. You should map at least three mandatory columns: Timeline ID, Timestamp, and Event name. For example, drag and drop a Timeline ID label to mark the column that contains such data.

While uploading a file to the non-empty project, the application may show previously created mapping if it matches the uploaded data. You can reassign mapping if you want to change it.

After you map the required fields, click Label all as attributes to automatically create attributes from all other columns.

Mandatory fields

The application uses information from the uploaded file to create timelines for further analysis. You should map the following columns with relative labels so that the program extracts timelines from the uploaded data:

• Timeline ID – Map this label to the column that contains the IDs of the monitored objects. An object may correspond to the ID of an order, claim, patient form, support ticket, and so on. The ID will link all the events associated with the given object.

• Timestamp – Drag and drop this label to the column containing the time of the events occurring throughout the lifetime of the object.

  Make sure that the data file uses the correct date and time format. See Mandatory and optional fields.

• Event name – Map this label to the column containing the events associated with the object at any given time. Examples of such events are Order received, Customer called, Survey commissioned, and so on.

Optional fields

In addition to the mandatory columns, the file can contain any number of additional fields to be used as dimensional attributes. You can filter by these fields, group, and break down by them, or use them as additional information when analyzing the processes.

Auxiliary columns

While mapping, the application has two pre-configured labels that can be mapped optionally:

• Event category – This is an auxiliary field that must be used if your data file contains events of many different types. It is recommended to use the Event category field for files containing events of more than 150 different types. The Event category field contains a group of all related events of the same type as a separate subset.

• Event number – This is an auxiliary field that you may want to use if your data file contains events with identical data and/or time stamps. To order such events, you can map them to a field that specifies their order. This order will be used if multiple timeline events have identical data and/or time stamps. The data in this column will be numbered 1, 2, 3, and so on, depending on the numbering system used.

Other optional columns

Besides the auxiliary columns, you can map columns to any other fields which will be used as attributes. You will then be able to use these fields as filters, group or ungroup elements by these fields, or use them as a source of additional information for process analysis.

Drag and drop a New attribute label to any of the columns. The label will be named as the column in the original file. Click the pencil icon to rename the attribute. It will be uploaded to the project for all events that have this attribute. After setting up an optional attribute and uploading data, you can, for example, filter timelines using attribute values and configure analysis modules based on them, such as interval measurements.

Process view page

After the data upload, the application generates timelines using the processed data and opens the Process view page. By default, the application displays the Primary path view. This graph displays the most common flow of events in timelines. You can switch to Milestone view to discover the current timeline set. The default board shows the overview statistics for the selected view module, such as count of timelines, time range, and so on. For more information, see Process view.