Enterworks - ETT 117 - How to Launch Scheduled Imports from a Custom Widget in the New UI using a Micro-Service

The new UI provides the ability to launch an import by uploading a file for a repository and current view or import template.   When submitting Excel import files, there is a limit to the size the Excel file can be.  Larger files can cause the UI to hang as it tries to load the entire file into memory to present the user with the column mappings and the first 100 rows.  Also, the template selected for import may be altered (temporarily) by the contents of the file, which may result in unintended consequences, such as loading additional data that may be present in the file but not mapped in the template.  Finally, launching an import in the UI does not provide the opportunity for the submitted file to be pre-processed nor to launch any dependent operations after the file is loaded.

Imports can also be processed using the Scheduled Import functionality.  For launching a Scheduled Import on-demand, the import file must be placed in either a directory on the EnterWorks server or in a drop location in an AWS S3 Bucket, Azure Blob, or FTP Server and then the Import Now flag on the desired Scheduled Import must be set to Yes.  This limits the import to a single user at a time and does not allow the job to be associated to the submitting user for e-mail notifications and security purposes.  It also requires granting edit access to the Scheduled Import repository, which may not be desirable for security reasons - especially if external groups such as Suppliers or Vendors need to submit files for importing.

The above limitations can be addressed by constructing a custom widget an Micro-service that allows the user to select a Scheduled Import from a list and data file for upload.  The import widget would likely list a subset of imports defined in the Scheduled Imports repository.  Different Groups can have different sets of available imports listed.  More than one widget can be configured in the same dashboard to segregate imports by repository or other criteria.  Once an import and file are selected, the file is uploaded the server and the designated Scheduled Import is launched for that file.  The Scheduled Import job runs on the user's behalf so notifications would be sent to that user via e-mail and updates to the target repository/repositories would be limited to the users assigned security.  This approach has the following benefits:

• The import will only use what is defined in the associated Scheduled Import for consistent results
• Large Excel files can be uploaded through the UI (100K+ rows)
• The submitted file can be pre-processed before actually being imported into EnterWorks
• The submitting user will be identified as performing the import and receive any e-mail notifications (if configured)
• The import can be configured to run the actual import as the submitting user, subjecting the import to the user's security restrictions
• The Scheduled Import can invoke dependent operations after processing the import

This session shows how to implement and configure a widget and microservice to upload files from a widget and launch a selected Scheduled Import.  Sample files for the Import widget and Miro-service are provided and covered in detail.

The example micro-service used for this session has been modified significantly as it is used for other related Tech Talks.  Please download and deploy the set of files bundled with ETT 125 – How to Create a Simplified REST API Layer Using a Micro-Service to utilize the functionality covered in this Tech Talk.

 Prerequisites - ETT 108, ETT 020