Setting Up Plutora Extract, Transform and Load (ETL)

Extract, Transform and Load (ETL) is an automated process which:

  1. Extracts data from a customer’s application or database.
  2. Transforms the customer’s data into a format suitable for Plutora.
  3. Loads the transformed data into Plutora.

Plutora’s ETL process is flexible and powerful. It can handle thousands of transactions an hour, and it can be customized to add, edit or delete any record on Plutora, including releases, environments or deployment plans.

Plutora’s Report Center has ETL reports that provide metrics, such as the number of jobs, transform processing duration, error counts and so on.

Customers should contact Plutora Support at Plutora.com to set up ETL processing.

 

Mapping

During the ETL set up process with a Plutora representative, customers discuss:

  • What data file type they will be sending, as the file type affects the way the file processing.
    Data files types can be XML, CSV, XLS or TXT. 
  • How they would like each column of their data mapped to Plutora’s fields.
    For example, a customer may want one of their data columns to be mapped to Release Type in Plutora.

Mapping can be as simple, such as mapping a single data column to a single field in Plutora. Or it can be more complicated. Such as:

  • Concatenating customer data two rows and adding them to a single field in Plutora.
  • Making a value equal 4 if the previous values are 2 and 1.

Once the mapping is worked out, a developer programs the logic into Plutora. This takes between a day and a week, depending on the complexity of the mapping logic.

Unique Identifier

One column of the customer’s data file must contain a unique identifier. For example, if the customer wanted to use ETL to add Releases, one column must represent and map to Release ID.

If the unique identifier is already present in Plutora, the record will be updated. If the unique identifier is not present, Plutora will create a new record.

Mandatory Fields

Some of Plutora’s fields are mandatory, for instance, Release Name. This still applies even if the data is received by ETL. During the mapping process, the customer must supply all the mandatory fields by mapping them to their own data.

XML

Customer data files are processed and transformed by Plutora into an XML file that can be human-read and is suitable to be loaded into Plutora’s database.

Here is an example of an XML file, showing how the Plutora fields (<Map>) are mapped to the customer fields (<CustomField>).

<Source>
<Map ReleaseID=”0Name=”1Impacts=”2ReleaseSummary=”3ScheduledStartDateTime=”4ScheduledStartDate=”5RiskLevel=”6Location=”7ReleaseManager=”8Stage=”9ReleaseType=”10PortfolioAssociation=”11ReleasePriority=”12ChangeTiming=”13ChangeSummary=”14ChangeManager=”15SubmitDate=”16ScheduledEndDateTime=”17RootCause=”18“/>
<CustomFields>
<CustomField Index=”{0}CustomFieldName=”ID“/>
<CustomField Index=”{4}CustomFieldName=”Scheduled Start Date and Time“/>
<CustomField Index=”{17}CustomFieldName=”Scheduled End Date and Time“/>
<CustomField Index=”{13}CustomFieldName=”Change Timing“/>
<CustomField Index=”{14}CustomFieldName=”Change Summary“/>
<CustomField Index=”{18}CustomFieldName=”Root Cause“/>
<CustomField Index=”{8}CustomFieldName=”Release Manager“/>
</CustomFields>
<Change SendLogEmail=”” DateCultureFormat=”” ImplementationDate=”{17}DefaultParentReleaseID=”On HoldUserManager=”support@plutora.comDisplayColor=”#004B46ParentReleaseType=”MajorPlutoraReleaseType=”IntegratedReleaseProjectType=”IsProjectDefaultReleaseStatusType=”DraftReleaseType=”Major – Release“/>
</Source>

 

Plutora supplies each ETL customer with their own SFTP drop zone folder, username and password details, so they can securely upload data files via SFTP. (For a definition of SFTP, see the Glossary.)

Once set up, customers can manually log into their drop zone folder and upload data files using any SFTP application, or they can create an automated system that can upload data files to their drop zone folder as they are created.

 

 

Once set up, the ETL process has three steps:

  1. The customer extracts data from their application or database, usually automatically and on a regular schedule (daily or every few minutes).
  2. The customer sends the data file by SFTP to the drop zone folder at Plutora.

 

  1. The monitor software on Plutora’s SFTP server checks the customer’s drop zone folder for a new data file once a minute.
  2. Once detected, the data file is transformed, using the customer rules programmed into Plutora, into an XML file suitable for Plutora to load.
  3. Plutora logs the transform process.

 

  1. The transformed data is loaded into Plutora.
  2. Plutora emails the log file to the customer.

Example ETL log file

 

Back to the top arrow

Was this article helpful?

6 found this helpful.