Intelligent Equipment Accelerator Getting Started in the Cloud

Pre-Requisites

The Intelligent Equipment Accelerator can be run as either an on-premise or hybrid cloud architecture. This wiki is written to guide the user in setting up the Accelerator using the hybrid cloud architecture.

In the hybrid cloud architecture, you will require access to certain TIBCO Cloud services as well as an active AWS account and services. This is what you will be provisioning:

TIBCO Cloud Requirements

For TIBCO Cloud, you will require an account with access to the following enabled subscriptions:

• TIBCO Cloud Data Streams (with either User or Administrator privileges)
• TIBCO Cloud Integration
• TIBCO Cloud Messaging
• TIBCO Cloud Spotfire®

At the moment only installations in the TIBCO Cloud EU or US regions are supported. All subscriptions must be enabled in the same region.

TIBCO Cloud Data Streams Notes

1. You must have either User or Administrator privileges for your Data Streams subscription in order to be able to create streams. If you do not have these privileges then the automated provisioning will fail.
2. Most Data Streams subscriptions have a limit to the number of simultaneous running streams. For example, 5 streams is a common configuration. if you have running streams in your subscription, ensure you have at least 1 available stream to start for the Accelerator. If there is not 1 available stream then the automated provisioning will fail.

AWS Requirements

For AWS you will require an account that has access to the following services:

• AWS CloudFormation
• AWS Elastic Kubernetes Service

AWS Account Setup

Access Key

In order to provision Kubernetes you will need to have an AWS Access Key Id and Secret Access Key token. To generate these, go into your AWS account. If you have a federated user account, do not assume your account administrator role. Next, select the IAM service, and then select the Users option in the left hand menu. This should display the currently logged in user.

Select the currently logged in user from the list by clicking the link. Then select the Security credentials tab, and then the Create access key button. This will generate a new Access key ID and Secret access key. Make a note of these as we will need them later. These keys are private to your account so they should be treated the same as any username/password combination and not shared. 

Assume Role

If you need to assume a role to perform tasks in AWS you will need an AWS Role ARN. To obtain this, switch to the role you need to assume using the AWS console. Then go to the IAM service and select the Roles option in the left hand menu. This will display a list of roles. Select the role you need to assume. This will display a Summary page and the Role ARN will be visible here. Make a note of this as it will be needed later.

Next, select the Trust relationships tab and then click Edit trust relationship. This will display a Policy Document in JSON format. You will need to configure the ARN for your user account in the AWS block that has the Action sts:AssumeRole. See the screenshot below for an example.

Click Update Trust Policy to save the changes.

Download Accelerator Distribution

The Accelerator is distributed as two packages:

• Distribution -- distribution for building the application
• Source -- to view and make changes to the source

Download the distribution package to your local machine and extract the ZIP into a temporary directory.

Once extracted you will have a directory called IOT containing the files manifest.json and iot-webapp.zip. 

Deploy Accelerator Webapp

To deploy the Accelerator webapp you will need to login to your TIBCO Cloud Integration account. Click here for EU region and here for US region. You may be prompted to login using your TIBCO Cloud credentials.

Click the Create/Import button, and then choose a Node.js application under the Develop section.

Click the Next button. Drag and drop both the manifest.json and iot-webapp.zip into the displayed dialog, then click Import App. The Creating a new application.. message will display for some time while the app is created.

Once the app is successfully created, the following page will be displayed.

To start the app, hover over the 0 next to Instances and click the up arrow to increment the value to 1. Then click the Scale button. The status will go to Scaling and it will take some time for the app to start.

If all goes well the status will eventually go to Running. At this point the application is ready to be accessed. If the app doesn't start, review the logs for error messages. If this still doesn't help then please open a Support request.

Login to Accelerator

To get the URL to connect to the web UI, click the Copy URL button in the middle of the application page to copy the value to the clipboard. 

Paste this into a new browser tab or window and append /app/iot/index.html at the end of it. For example:

• eu-west-1.integration.cloud.tibcoapps.com/abc123/app/iot/index.html

It is recommended to bookmark this page.

The browser will display the Login page for the Accelerator.

The credentials to login is an OAuth token generated from your TIBCO Cloud account. Click here for EU region and here for US region to display the tokens page. Click the Generate Token button and given the token a name. Select a validity period. Beware that once the token expires you will need to generate another one in order to access the Accelerator. You must select the following domains:

• Connected Intelligence Cloud
• Spotfire®
• Data Streams
• Integration
• Messaging

Click the Generate button to generate the token. Note the confirmation dialog and then click the Copy to clipboard button to copy the token value.

Back on the Accelerator Login page paste the token value in the OAuth Token box and select the correct Region where your subscriptions are provisioned, then click Login.

If the credentials are valid then the Welcome page will be displayed. Both the OAuth token and Region will be stored in your browser local storage so will be saved for the next login. If you are using an incognito window or clear the browser cache the values will be lost, so it is worthwhile making a note of your OAuth token for future use.

Generate Messaging Key

TIBCO Cloud Messaging comes preconfigured with a default role, so this step is optional. If you would like to make a specific role for the Accelerator click here for EU region and here for US region. Click the Create Role button. Enter a value for **Role Name** and keep the remaining options as defaults, then click the Create button. You do not need to note the role details as you will choose this later using a drop down.

Configure Accelerator

The Welcome Page will be displayed on first load of the Accelerator and will walk you through the process of finalizing the provisioning of components.

General Configuration

The first page will present some application options, adjust these as required.

• Hide Welcome page -- tick this box to hide the Welcome page on next login
• Restrict UI -- select the desired option for restricting access to the user interface. Note that your group will be different than the group Accelerators show in the screenshot.

Click the Save button to save any changes, and Skip to move to the next page without saving.

TIBCO Cloud Messaging Configuration

The next page will present some TIBCO Cloud Messaging options. The OAuth Token will be preconfigured based on the one used to login. Select a TIBCO Cloud Messaging Channel from the drop down. This channel will be used for all Accelerator component communications. All your configured channels will be displayed, choose the desired option, if in doubt, select default. Click the Save button.

Accelerator Component Provisioning

The next page presents component provisioning options. The TIBCO Cloud Messaging Channel will be displayed, as well as the OAuth Token used during login. You have the option of which components to provision. Select both TIBCO Streaming on AWS Elastic Kubernetes Service and and TIBCO Cloud Data Streams. Click the Next button.

On this page enter the AWS Region where your Kubernetes cluster will be deployed, as well as AWS Access Key Id and AWS Secret Access Key for the user account. If this account requires assuming a role in order to execute the commands, enter the AWS Role ARN. Click the Next button.

You will be prompted whether or not you already have a Kubernetes cluster. For this guide we will select the I need a cluster option. This will present a script which must be executed using AWS CloudShell. Copy the contents of the box into the clipboard.

Open a CloudShell session from your AWS account. If you are using an assumed role, you should do this as that role. This can be accessed from the icon in the upper right corner of the console.

Once the session is ready and the command prompt is shown, paste the script from the Welcome page into the console. This will execute several commands:

1. Setup environment variables
2. Install the eksctl utility
3. Print the cluster name to the console
4. Execute the eksctl command to create a cluster with 1 node

The Kubernetes cluster name will be logged to the console. It will look something like iot-accelerator-abc123. Enter this into the Kubernetes Cluster Name field on the Welcome page, but DO NOT click Next just yet.

This script will take quite some time to complete, up to 20 minutes. AWS CloudShell sometimes has a bad habit of timing out your session, so the eksctl command is being run using nohup, and the output is displayed using a tail command. If your CloudShell session does get disconnected, simply reconnect and then enter this command to continue observing the output: 

• tail -f eksctl.out 

If all goes well, eventually the console will display a line similar to this with your cluster name:

• 2021-04-20 16:12:32 [?]  EKS cluster "iot-accelerator-abc123" in "eu-west-1" region is ready

At this point you can go back to the Welcome page and click the Next button.

Now you will be prompted to create a Kubernetes service account and presented with another script. This must also be executed using AWS CloudShell. You should reuse the existing CloudShell session you used for the previous step if it is still active, otherwise you can start a new session. If you reuse the previous CloudShell it might still be tailing the previous command. Just use CTRL-C to break out to the command line.

Copy the contents of the box into the clipboard. Paste the script from the Welcome page into the console. This will execute several commands:

1. Setup environment variables
2. Install the kubectl utility
3. Execute the kubectl command to create a service account
4. Execute the kubectl command to apply roles to the service account
5. Execute the kubectl command to create a cluster role binding
6. Execute the kubectl command to retrieve the secret name and token
7. Print the Kubernetes token to the console

This script will complete relatively quickly. The Kubernetes authentication token will be displayed at the end and this will be a long string of characters. Copy and paste this into the Kubernetes Authentication Token field on the Welcome page. 

At this point everything has been configured and you can click the Provision button. It will take some time for the provision process to complete.

Once the provisioning has completed you can verify TIBCO Cloud Data Streams by clicking here for EU region and here for US region. You should have 3 streams called IOT_Alerts, IOT_Devices, and IOT_Entities. It will take several minutes for these to start so they may be in Building and Starting status for some time.

To verify TIBCO Streaming on AWS Elastic Kubernetes Service has started correctly you can open an AWS CloudShell session and enter the following command to follow the pod logs.

• kubectl logs -f iot-app-cloud-eventmanager-0

It will take several minutes for Spotfire® Streaming engine to start. It will be ready when you see something similar to this in the log:

• 2021-04-20 16:37:13.022000+0000 [262:Thread- ThreadPool - 1] INFO com.streambase.sb.sbd.net.StreamBaseHTTPServer: sbd at iot-app-cloud-eventmanager-0:10001; pid=262; version=10.6.1_0a0fdf7f8f3d4f25851d53e0e55c97ce2ece3d22; Listening

When all components have started correctly, back in the Welcome page you can click the Next button to continue.

Spotfire® Configuration

The next page will display Spotfire® configuration options. Unfortunately it is not possible to automatically provision a Spotfire® DXP, so this will have to be done manually. The OAuth Token will be preconfigured based on the one used to login. Click the Download DXP button to dowload a file called esp.dxp to your local machine that has been pre-configured to connect to the data stream previously provisioned.

Navigate to your TIBCO Cloud Spotfire® subscription by clicking here for EU region and here for US region. Click the Browse local file... option and navigate to the esp.dxp that you downloaded. Once loaded, the DXP should connect to your running streams showing an analytics dashboard. Everything will be empty for now though.

On the right hand side of the menu bar, where it says Viewing, change this to Editing. Then select the File > Save As > Library Item... menu option. Choose a location to save the analysis and then click the Save button.

Once it's saved you will need to extract the path to the analysis. The easiest way to do this is just extract it from the path in your browser. It will look something like this:

• https://eu.spotfire-next.cloud.tibco.com/spotfire/wp/analysis?file=/Teams/XYZ4499/IOT/esp&waid=abc123&wavid=0

The part we are interested in here is the file query parameter. The value is between the "=" and the following "&". In the example above it will be:

• /Teams/XYZ4499/IOT/esp

Back in the Accelerator Welcome page you can paste this in the Analysis Path box. The Web Player Server will be pre-populated based on the region that you logged in as, but please verify that it is correct by checking the URL of the Spotfire® page where you uploaded the DXP to ensure they match. Click the Save button once completed.

The Welcome page settings are now completed so you can click Close to go to the main dashboard.

Access Main Dashboard

The first time the Main Dashboard is accessed the UI will attempt to load the configured Spotfire® analysis. Note that third-party cookies must be enabled, so if you are using Incognito mode then this will need to be enabled by clicking the icon on the address bar, then Site Not Working and then Allow Cookies. You may need to manually refresh the page.

If you have not logged in to a TIBCO Cloud service in the current browser session, then dashboard will display a message prompting you to login to Spotfire®. Click the Log in button and complete the login process.

If you have previously logged in to a TIBCO Cloud service in the current browser session then the login step should be skipped and the dashboard will be displayed. It will be empty for now.

Run a Simulation

Select the Simulator menu option. It may take a minute or two for the simulation status to become available to the webapp. Once the status is displayed, choose a value for Test Case from the list of options. The testcase name and a brief description are displayed. Then click the Start button.

Once the simulation is running, you can adjust the Time Compression to increase the message rate if you wish. You can also pause the simulation by clicking the Pause button and Resume button to resume. To stop the simulation click the Terminate button.

Otherwise, select the Home menu option to display the dashboard view. A stream of quote requests will appear and key metrics will be displayed. Be sure to stop the simulation when you are finished.

Adjust Settings

Select the Settings menu item to adjust any of the configurations defined during the Welcome page process. Note that components should not be re-provisioned without manually deleting any existing events applications or data streams. If data streams are re-provisioned then the Spotfire® DXP will require updating to point at the new streams.

Shutting Down

There is a cost associated with running Kubernetes, Data Streams, and Integration services. So it is recommended to stop these services manually when not in use.

1. Open TIBCO Cloud Data Streams by clicking here for EU region and here for US region then click the Stop button for each of IOT_Alerts, IOT_Devices, and IOT_Entities. This will take a few minutes to stop.
2. Open TIBCO Cloud Integration by clicking here for EU region and here for US region then then clicking the Stop button for the iot-webapp application
3. To remove the Kubernetes cluster, go to the AWS Cloud Formation service. Make sure your region matches the location where the cluster was created, which is Ireland by default. View the stacks using the menu option in the upper left. There will be 2 stacks created for the Kubernetes cluster that you created earlier. Select the nodegroup stack by clicking the radio button, and then click the Delete button. Then click Delete Stack. It will take some time for this stack to be deleted. You will have to click the refresh button to see it has been removed. Once completed, select the other stack, and click the Delete button for that one. This will completely remove your Kubernetes cluster from your AWS environment.