Amazon IoT core is a MQTT server based architecture that you can utilize with your IoT projects.

The implementation requires the use of SSL/TLS that requires client side certificates to identify the connected device, or in Amazon terms a "Thing".

This article shows how you can connect Spotfire Streaming with AWS IoT Core.

AWS Configuration

This is not a tutorial on setting up AWS IoT core, all this will show is the bits you need to do to gain connectivity.

Step 1 - Define a Thing

If you are setting up AWS IoT for the first time the getting started wizard will guide you to a step called "Configuring a device". If you have already set up IoT core then select "Onboard" from the left hand menu and then click on Get Started under configuring a device.

Read the options and click on "Get started"

The getting started wizard will generate a testing kit for an OS and software stack of your choice. You are welcome to run the test, but all we need is the certificates that are part of the package. Make you choice and click "Next"

You can now download the zip file that contains the test harness and the certificates for the "thing". Place these files in a new directory and we will be using them in the next step.

At this stage AWS has created your thing and a IAM policy for that thing. This policy will need to be updated as it has restricted access permissions by default. You can view these by clicking on "Preview policy"

You can test the configuration outside Spotfire Streaming by following the instructions on the last wizard page.

Step 2 - Create Keystores

Intro and preparation

This article is not going to teach you about keystores, please read up on keystores to understand what they are.

You will need to have two tools installed on your machine. Keytool which is part of the Java JDK and OpenSSL. You can check that you have these installed by entering the following two commands in a terminal window.

PS C:\Users\ahampshi> openssl version
OpenSSL 1.1.0h  27 Mar 2018
PS C:\Users\ahampshi> keytool

Key and Certificate Management Tool

Commands:

 -certreq            Generates a certificate request
 -changealias        Changes an entry's alias
 -delete             Deletes an entry
 -exportcert         Exports certificate
 -genkeypair         Generates a key pair
 -genseckey          Generates a secret key
 -gencert            Generates certificate from a certificate request
 -importcert         Imports a certificate or a certificate chain
 -importpass         Imports a password
 -importkeystore     Imports one or all entries from another keystore
 -keypasswd          Changes the key password of an entry
 -list               Lists entries in a keystore
 -printcert          Prints the content of a certificate
 -printcertreq       Prints the content of a certificate request
 -printcrl           Prints the content of a CRL file
 -storepasswd        Changes the store password of a keystore

Use "keytool -command_name -help" for usage of command_name

If they don't work you will need to install current versions of both tools. Also check that the tools are on the current users PATH.

Creating the keystores

We need two keystores ... The first is for the TrustStore.... Basically this is a keystore that contains the remote server's certificate file, it's used to verify the TLS connection is to the target server. The second is the "thing" keystore that contains the thing's key pair.

1 - Obtain AWS IoT server root certificate

To do this we need to get the server's certificate. It is possible to pull the key direct from the server but the simplest way it to download it from the AWS documentation page here: https://docs.aws.amazon.com/iot/latest/developerguide/managing-device-ce... . In this example i have downloaded the file root ca 1. This is a text file that contains a text string that will look like this:

Store this in the same directory as the keys downloaded earlier and call it AmazonRootCA1.crt

You should now have a directory that contains the two keys, the user certificate and the root certificate.

You can test the files you have by:

openssl s_client -connect yourservername.iot.yourregion.amazonaws.com:8883 -CAfile .\AmazonRootCA1.crt -cert .\StreamingThing.cert.pem -key .\StreamingThing.private.key

From the output you should see that openssl can connect to the remote server.

2 - Create Server KeyStore

Spotfire Streambase requires a keystore that has a password. And it expects the keys/certs to also be password protected too. The keys that were downloaded are not password protected, so we have to go through a minor hoop to create the user keystore.

First password protect the key.

 openssl rsa -aes256 -in StreamingThing.private.key -out StreamingThing.private.key.encrypted.pem

Create a P12 version of the certificate/key pair:

 openssl pkcs12 -export -in StreamingThing.cert.pem -inkey StreamingThing.private.key.encrypted.pem -certfile StreamingThing.cert.pem -out tempkeystore.p12

Import into a JCEKS keystore

 keytool -importkeystore -destkeystore StreamingThing.jceks -deststoretype JCEKS -destalias "aws iot cert"  -srckeystore tempkeystore.p12 -srcstoretype PKCS12 -srcalias 1

The resultant file will be used as the sslKeystore parameter in the adapter-configurations.xml.

3 - Create Truststore From root CA certificate

The truststore creation is simpler, all that is required it to load it to a keystore

Import to jceks format keystore:

 keytool -importcert -alias "amazon root ca 1" -file AmazonRootCA1.crt -keystore Truststore.jceks -storetype JCEKS

The resultant file will be used as the sslKeystore parameter in the adapter-configurations.xml.

Step 3 - AWS Policy Configuration

AWS uses both the keys and it's own policy infrastructure to secure the usage of AWS IoT. The "getting Started" wizard has created a default policy that will work with the supplied SDK that was downloaded but you will need to make changes to connect from Spotfire Streaming.

From the AWS IoT console select Secure -> Policies from the left hand menu and then select the created policy... in our cases StreamingThing-Policy

The default policy looks like this:

{
 "Version": "2012-10-17",
 "Statement": [
   {
     "Effect": "Allow",
     "Action": [
       "iot:Publish",
       "iot:Receive"
     ],
     "Resource": [
       "arn:aws:iot:eu-west-1:544643958209:topic/sdk/test/java",
       "arn:aws:iot:eu-west-1:544643958209:topic/sdk/test/Python",
       "arn:aws:iot:eu-west-1:544643958209:topic/topic_1",
       "arn:aws:iot:eu-west-1:544643958209:topic/topic_2"
     ]
   },
   {
     "Effect": "Allow",
     "Action": [
       "iot:Subscribe"
     ],
     "Resource": [
       "arn:aws:iot:eu-west-1:544643958209:topicfilter/sdk/test/java",
       "arn:aws:iot:eu-west-1:544643958209:topicfilter/sdk/test/Python",
       "arn:aws:iot:eu-west-1:544643958209:topicfilter/topic_1",
       "arn:aws:iot:eu-west-1:544643958209:topicfilter/topic_2"
     ]
   },
   {
     "Effect": "Allow",
     "Action": [
       "iot:Connect"
     ],
     "Resource": [
       "arn:aws:iot:eu-west-1:544643958209:client/sdk-java",
       "arn:aws:iot:eu-west-1:544643958209:client/basicPubSub",
       "arn:aws:iot:eu-west-1:544643958209:client/sdk-nodejs-*"
     ]
   }
 ]
}

In the policy there are three resource groups and you will need to amend all three. If we assume that our Spotfire Streaming application is going to read from one topic and write to another, these will be /IoT/widgetfeed and /IoT/serverout.

The feed app will be connecting using a unique client ID of Widget-123456 where 123456 is a random number. The server app will be using a Spotfire Streaming generated Client ID.

As a result our policy will look something like this. Please note that a production application may want to be more specific about its access requirements.

{
 "Version": "2012-10-17",
 "Statement": [
   {
     "Effect": "Allow",
     "Action": [
       "iot:Publish",
       "iot:Receive"
     ],
     "Resource": [
       "arn:aws:iot:eu-west-1:544643958209:topic/IoT/widgetfeed",
       "arn:aws:iot:eu-west-1:544643958209:topic/IoT/serverout"
    ]
   },
   {
     "Effect": "Allow",
     "Action": [
       "iot:Subscribe"
     ],
     "Resource": [
       "arn:aws:iot:eu-west-1:544643958209:topicfilter/IoT/widgetfeed",
       "arn:aws:iot:eu-west-1:544643958209:topicfilter/IoT/serverout"      ]
   },
   {
     "Effect": "Allow",
     "Action": [
       "iot:Connect"
     ],
     "Resource": [
       "arn:aws:iot:eu-west-1:544643958209:client/Widget-*",
       "arn:aws:iot:eu-west-1:544643958209:client/*"
     ]
   }
 ]
}

Step 4 - Setup the Connection in Spotfire Streaming

Edit your adapter-configurations.xml file in the Eclipse editor.

Find or add the MQTT configuration

<adapter-configurations>
       <adapter-configuration name="mqttclientconnections">
           <section name="mqttclientconnection">
                <setting name="id" val="AWSIoTBrokerTLS"/>

You will need to set the following:

URI

 <setting name="URIs" val="ssl://yourservername.iot.yourregion.amazonaws.com:8883" />

The URI value can be found in the AWS IoT console under Settings - Custom endpoint:

Client ID

This needs to match with the pattern or fixed value on the Policy iot:Connect definition:

<!-- The client identifier to use when creating a connection to the broker.  If this value is blank an identifier will be generated.
  <setting name="ClientId" val="Widget-345689" />
-->
  <setting name="ClientId" val="" />

Both examples would work with the sample definitions above.

SSL Protocol

TLS is the minimum supported by AWS IoT

<!-- One of: SSL, SSLv3, TLS, TLSv1, SSL_TLS -->
<setting name="sslProtocol" val="TLS" />

Keystore details

<!-- Context can be left blank unless you have specific requirements -->
<setting name="sslContextProvider" val="" />

<!-- The name of the file that contains the KeyStore object that you want the KeyManager to use. -->

<!-- Sample linux or docker file location -->
<setting name="sslKeyStore" val="/usr/share/tls/StreamingThing.jceks" />

<!-- Sample Windows file location -->
<setting name="sslKeyStore" val="C:/path/to/your/project/src/main/resources/StreamingThing.jceks" />

<!-- The password for the KeyStore object that you want the KeyManager to use -->
<setting name="sslKeyStorePassword" val="mypassword" />

<!-- for example "PKCS12", "JKS", or "JCEKS" -->
<setting name="sslKeyStoreType" val="JCEKS" />

Truststore details

<!-- Context can be left blank unless you have specific requirements -->
<setting name="sslKeyStoreProvider" val="" />

<!-- The name of the file that contains the KeyStore object that you want the KeyManager to use.  -->

<!-- Sample linux or docker file location -->
<setting name="sslTrustStore" val="/usr/share/tls/Truststore.jceks" />

<!-- Sample Windows file location -->
<setting name="sslTrustStore" val="C:/path/to/your/project/src/main/resources/Truststore.jceks" />

<!-- The password for the KeyStore object that you want the KeyManager to use -->
<setting name="sslTrustStorePassword" val="mypassword" />

<!-- for example "PKCS12", "JKS", or "JCEKS" -->
<setting name="sslTrustStoreType" val="JCEKS" />

Step 5 - Test it !

At this point you have everything in place to make a connection and publish/subscribe to messages from Amazon IoT.

Sign In

Working with Amazon IoT from Spotfire Streaming