Writing Integrations for Pervasive Integration Agent

Overview

The audience for this document are developers and engineers who will be writing custom integrations that run from Pervasive Integration Agent. This document also provides advanced topics and usage scenarios.

Contents

Below are the contents:

Advantages of Developing for Integration Agent

The advantages of developing for Integration Agent are:

  • Using the DataCloud deployment model is the easiest way towards zero touch sales and delivery.
  • Integrations can be accessed from both the DataCloud and on-premise resources.
  • Push updates to integration packages for new features or maintenance releases.
  • Centralized management and support of customers and their log files via our publicly available DataCloud Management Console web interface.
  • Scheduling and on-demand setups are easy for end-users to deploy and to customize.

Back to top

Differences Between v9 and v10 Integration Agent

The v9 Integration Agent can only run v9 integrations with the v9 engine via the command line. The V9 Integration Agent requires a v9 license.

The v10 Integration Agent can run both v9 and v10 integrations with the v10 engine via the command line. The V10 Integration Agent requires a V10 license.

There are four versions of the v9 installer.  The v9 installers exist for 32- and 64-bit Windows and come with or without V9 Data Integrator installation bundles. The v10 Integration Agent only supports the 64-bit platform. Currently, the v10 Data Integrator is not bundled with the v10 Agent installer.

Version Platform Installer Bundles Data Integrator
9 Windows 32-bit no
9 Windows 32-bit yes (w Full DI Installer)
9 Windows 64-bit no
9 Windows 64-bit yes (w Engine-Only installer)
10 Windows 64-bit no

Back to top

Running v9 jobs in v10 Integration Agent

It is possible to run v9 jobs with the v10 Integration Agent if the license is compatible with v10.  Combining a v10 license with a v9 job requires assistance from the DataCloud team and potentially IT, especially if the integration will be sold through partner portal.

Back to top

JVM Requirements

If you're installing a 32-bit Agent you have to use a 32-bit JVM and 64-bit Agent requires the 64- bit JVM.

See the Pervasive Integration Agent Installation Guide for the latest environment variable configuration.

Back to top

Using the On Demand runnow Parameter

In order to build an on demand job, a parameter has to exist in the DataCloud for the selected provisioning called "runnow".  This parameter controls if the job is allowed to run.  This parameter triggers a block to an Agent run attempt if the runnow parameter is not set to 1. If the value of the parameter is anything other than 1, then Integration Agent will not run the integration on a schedule.  The runnow parameter also triggers a default run schedule for every 10 seconds.

When Integration Agent completes an integration run of an on-demand integration, it will reset the runnow parameter to 0 in the DataCloud so as not to run it again on the next iteration.

Due to the way parameter changes are propagated to Integration Agent.  A user should enable automatic installs before each run to catch when a parameter change has occurred. This is also suggested in the the Pervasive Integration Agent User's Guide.  

 Note: this setting is not the default setting enabled when Agent is installed.

 Back to top

Understanding Offline Integrations

Integration Agent can be used off-line for the the centralized management of on-premise to on-premise integrations.  Although, Integration Agent is typically used for on-premise to DataCloud service integrations.

Back to top

Writing Integrations for Common DataCloud Use Cases

A common use case is to access local data, send the data to the DataCloud, and process the information on the DataCloud hardware. This use case tries to take advantage of the cost savings and scalability benefits of processing data in the DataCloud.

However, Integration Agent does not have a built-in mechanism to accomplish these processes. Integration Agent does not wait on data, will not help data get to DataCloud, and will not notify a cloud process to work on that data once the data is on the DataCloud.

There are Pervasive processes available accomplish this use case, however you can not accomplish this use case solely through Integration Agent.

Back to top

Viewing Updates

Updates are visible with when following conditions occur:

  • The refresh button has been clicked.
  • A periodic update cycle looks for updates.  This will only happen if the Integration Agent is setup to periodically check for updates in the Agent settings.
  • A run has occurred and Integration Agent is configured to install updates automatically before each run from the Integration Agent Settings window.

Updates will only be counted as updates if:

  • The set of component names in a manifest changes.
  • The version number in a manifest file changes (does not have to get bigger, just has to be different).
  • The arguments in a component in a manifest changes.
  • The main DJAR changes.
  • license file's name changes.
  • The location of file moves from provisioning to product or vice versa.
  • The entry point changes (v10 only).

Back to top

Downloading External Files with the Manifest

Additional files can be downloaded along with integration installations. If a file ends in .exe, then Integration Agent will try to execute the file during the installation. Users will be prompted to allow any .exe to run. 

Note that automatic updates cannot trigger .exe calls. If an integration incorporates an .exe file, the installation of the integration will fail.

Back to top

Using Manifest Files

Manifest files can alter how an integration works inside of Integration Agent.  Here are some important points about a manifest file:

  • A manifest file must be named manifest.xml.
  • It is not necessary for Integration Agent to be able to download and run an integration. 
  • The manifest can be used to instruct the Agent to download external files.
  • The manifest can instruct Integration Agent to install .exe's when the integrations is installed and notify an integration when an update is available.
  • The manifest can alter what the <Configuration> button does for any integration. 
  • A manifest can exist at both the product and provisioning level. The manifest only refers to files in whatever level the manifest file is on.  See product and provisioning levels.
  • The packagelocation file is case sensitive. integrationSpec is not the same as integrationspec. Any spelling error may produce an error stating Unable to find the correct files on S3.

Example

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<manifest>
<product>
  <name>Agent Test Product</name>
  <description>Agent Test Product</description>
  <sku>8859e394-d190-41b4-924b-e8d70dcafeb0</sku>
<components>
<component>
  <name>cosmosini</name>
  <version>1.0</version>
  <packagelocation>integrationSpec/MsgBox.exe</packagelocation>
  <arguments>Hello v10 Round Robin!!</arguments>
<changelogs>
  <log>Version 1.0</log>
  </changelogs>
  </component>
<component>
  <name>p_Siebel_Invoker_Demo</name>
  <version>1.0</version>
  <packagelocation>integrationSpec/p_Siebel_Invoker_Demo.1.0.djar</packagelocation>
  <arguments/>
<changelogs>
  <log>Version 1.0</log>
  </changelogs>
  </component>
  </components>
  </product>
  </manifest>

The Integration Agent only checks the following in manifest file:

  • The xml is properly formed.
  • The package location (this is the reference as to where to download the file from).
  • The version tag to identify if the version has changed.
  • The arguments tag:
    • for a .exe file are the arguments to pass to the .exe.
    • if the file is a main djar the arguments can alter what happens when the configuration button is clicked on that integration. 

Back to top

Running Custom Config URL and .exe files

The following actions will occur when the <Configuration> button is clicked for an integration when the integration's manifest file contains a component for the main DJAR and that component has an arguments tag:

  • If the arguments value starts with "http", then arguments value will be interpreted as a url and the configure button for that integration will direct the user to the URL plus the provisioning id.

    Example:
     If the arguments tag equals http://someurl/agenthelp? for the provisioning 10000, then the configuration button will direct the user to http://someurl/agenthelp?10000.
     
  • If the arguments tag does not start with "http", then the parameter will be interpreted as a command to run. The path to the integrationSpec folder will be prepended and the provisioning ID will be appended as an argument.

    Example: The <Configuration> button for provisioning 10000 containing a manifest tag's value of "x.exe" will run the following command: "C:\Users\[username]\AppData\Roaming\Pervasive\DataSolutions\Deployables\10000\integrationSpec\x.exe" 10000

Back to top

Locating File Directory Paths

Files are stored in a specific users' directory path. Storing the files in a specific user's directory enables multiple users on the same server to use Integration Agent simultaneously with different DataCloud credentials, integrations installed in different states and different schedules without any conflicts.

On newer versions of Windows, the directory path is at C:\Users\[username]\AppData\Roaming\Pervasive\DataSolutions\Deployables\.

On older versions of Windows, the directory path is c:\documents and settings\[username]\ApplicationData\Pervasive\DataSolutions\Deployables\.

Under the path, all integration files will be downloaded into folders named by the provision ID.  Under the provision ID, a license folder should house the license file, and most other files should be stored in the integrationSpec folder as that is the default in the DataCloud when you upload files.

Back to top

Using DJWORKSPACE

If the DJWORKSPACE parameter is not set in the product or in the provisioning, then the path will automatically be set by Integration Agent to the integrationSpec folder. The folder location is helpful for referencing temporary files or files that have been installed locally from the DataCloud such as configurations or lookups.

Back to top

Installing License Updates

All license updates are installed manually or through one of the automatic options in the Integration Agent Settings window. The license files have to be updated in the DataCloud first and the updates should be done through automated DataCloud maintenance.  A license error may occur if the user has not enabled automatic updates or if manual updates are not done on a regular basis.

Back to top

Using a Default Schedule

The default schedule parameter "interval" has a special meaning to Integration Agent.  The parameter defines an integration's default schedule in minutes unless the integration is on-demand as signaled by the "runnow" parameter.  See Using the On Demand runnow Parameter.

 Back to top

Product and Provisioning Levels

The provisioning files and provisioning parameters that make up an integration will follow the same rules on Integration Agent that are followed when run from the DataCloud. Provisioning files and provisioning parameters will override the files and the parameters in a product with the same name.

The only exception is manifest.xml files that can be at the product and provisioning level without conflicting as they each describe their own level. The product manifest xml file describes the product level and does provisioning manifest xml  file.

Back to top

Using Best Practices

If you need multiple provisionings for each product, try to keep the provisionings at the product level so there will not be a variance from provision to provision. An example is DJARs. If the integration on Integration Agent is intended only for one provisioning, then keep everything at the provisioning level for ease of use when manipulating the integration contents.

Back to top

Using Multiple DJARs

To use multiple DJARs, mark the required DJARs as additional components in a manifest file.  The DJAR marked in the DataCloud UI as the "current process file" will be the DJAR that is executed in the engine. The main DJAR can then call other DJARs using an .exe step and the DJWORKSPACE macro.

Back to top

Locating Log files

Integration Logs

Integration log files are uploaded to the DataCloud at the end of a successful integration run and then the log file is removed from the local integrationSpec folder.  All logs should go to Project.log in the integrationSpec folder with the other integration artifacts. If Integration Agent cannot upload to the DataCloud, then the log file will remain in the local integrationSpec folder.

Note: The primary cause of logfiles not being copied to the DataCloud are fatal errors in executing the integration where the log could not be created. The most common cause is invalid license errors.

Agent Logs

The agent log file is located at [Agent installation path]/IntegrationAgent/logs/agent.log. The main log is rolling 2 MB file and backup logs will be named agentBack#.log. A maximum of three logs (6 MB) will be created at any point in time.

The agent.log file captures the following information:

  • API calls to cloud.
  • Start execution times.
  • End exe times.
  • Errors or exceptions.

 

Back to top

 

Running Integration Agent as a Service in Windows 7

To automatically start Integration Agent to run as a service in Windows 7, complete the following steps:

  1. Launch Control Panel and open Administrative Tools.
  2. Select Task Scheduler.
  3. In the Task Scheduler dialog box, select Action -> Create Task.
  4. Name your task. In the example below will call the task "autoRunAgent".

    Create Task Dialog Box
     
  5. Select how you would like the task to run under Security Options. In this example. We will use Run whether a user is logged in or not. This will start Integration Agent as soon as the system is turned on, regardless of a user logging into Windows.
  6. Click on the Actions tab and select New.... Leave the action as Start a program. For this example we show the 32-bit v9 agent, so we are pointing to "Program/script" to "C:\Program Files (x86)\Pervasive\Cosmos9\IntegrationAgent\runAgent.bat". This path will change depending on your version of the Integration Agent, however you should always want to point to runAgent.bat.

    New action dialog box
     
  7. Browse all tabs and determine the settings depending on how you would like your Integration Agent to run.
  8. Click <OK> and enter your admin credentials to install the service.
  9. Monitor and update your task from the Task Scheduler. In the left pane select Task Scheduler Library and you should see "autoRunAgent" to the right. From here you can run, edit, or remove the task.

    Task Scheduler Dialog Box
     
  10. You will now see a corresponding "java.exe*32" under your processes in Windows Task Manager. Keep in mind that stopping the service will not stop this instance of Agent, and the only way you can stop Agent from running is by stopping the javaw.exe process.

    javaw.exe file

Notes:

  • You can not see the User Interface for the “task” agent. So you are not able to change any settings or schedules when Agent is running without stopping the “service” Agent process, making changes in another Agent instance, closing the other Agent instance,  and then triggering the “task” Agent to start again.
  • By default, another copy of Integration Agent will start when you log into Windows. Running two instances with the same user may cause issues. To prevent parallel runs, you will also want to stop the Agent from starting when Windows starts.
  • The “task” Agent can still install updates automatically if you configure the Agent settings to perform this action before running the Agent as a task. You will have a difficult time monitoring because you will have to keep checking the management console.

Back to top