Jump to content
  • Accessing OSDU R1 environment using AWS APIs in Spotfire


    Access OSDU Data with Spotfire

    Overview:

    TIBCO is a proud member of Open Group's Open Subsurface Data Universe? ForumThe OSDU is developing a standard data platform for the oil and gas industry, which will reduce silos and put data at the center of the subsurface community. As part of Release 1, OSDU has released a reference implementation with a well-defined set of APIs providing a standard way of accessing the data in the OSDU Data Platform and making sure that all applications can use a single set of APIs to access all data sources in the OSDU platform. TIBCO Spotfire simplifies access to these datasets and APIs by providing a ready-to-use dashboard app for viewing and analyzing OSDU data.

    Note: This Spotfire application was designed to access R1 OSDU data using the AWS APIs

     

     

     

    Prerequisites:

    • Acquire AWS OSDU credentials - Please contact srihap@amazon.com  from AWS OSDU team.  
    • Spotfire 10.7 or later comes by default installed with the Native Python data function. (This is the preferred version. Spotfire 10.8 was used to create the current example.)
    • Spotfire 7.13 through Spotfire 10.6 client and server. Spotfire Server should have the Community Python data function extension installed to enable the execution of Python code.
    • If you don't have Spotfire you can download a trial version here.
    • Working knowledge of Python data functions in Spotfire dashboards. Please refer to the above links and also please take a look at this link for data functions in general.
    • Please install these required python packages - hashlib, binascii, requests, json , hmac, pandas, lasio, and boto3. For installing packages, please refer to 
      .

    Running the Spotfire Analysis:

    Please download the dashboard attached to this blog (scroll down on this page to the "attachments" section) and open it in the Spotfire application. When prompted to Trust the scripts in the dashboards, navigate to ?Menu->File->Manage trust and scripts? and click ?Trust All?.  All the data functions used to create this example can be found under ?Menu->Data->Data function properties?. Every data function here is either linked to an action button or a hyperlink in the dashboard. 

    Now, please follow the step by step approach to see how we can consume the Authentication, Search and Delivery APIs in the data functions to download the data into Spotfire:

    • Enter your user credentials in the below fields in the dashboard and click the ?Authentication? button. As long as the Authentication Status is OK you are good to go. In this step, we utilize the Authentication API in AWS Authentication data function to get an ID token and Access token for accessing protected resources in AWS using the entered credentials. These credentials are stored in the Spotfire Document Properties and used accordingly in other data functions.

        1.png.8f686ab84dab9000ee478bb35692447d.png

    • Zoom in and Zoom out in the map chart to set your boundary extents. All the wells in this boundary extent will be retrieved in the next step         
    2.png.d98b3052413512f7983e5e3de0c3b2b4.png
    • After setting the boundary, please click on the hyperlink in ?Click here to get well list? to get all the well names that reside in the marked boundary.  In this step we leverage the Search API in Get_Well_From_Extent data function to perform this operation

        3.thumb.png.ba2fd19b8d8afb1f4dfe0c8f4ba280c8.png

    • Next, we select one or more wells in the Table1 that we populated in the last step and click the hyperlink in ?Click here to get Wellbore SRN and review Table2? to retrieve the WellBore SRNs for the selected wells. Here, we use the Search API in the Get_WellBore_SRN data function to retrieve the WellBore SRNs. 4.thumb.png.ae64bacba31622f6cd31ed3fd4e1c3e9.png
    • Once we have the Wellbore SRNs, we can now retrieve the Work Product Components such as well logs, WellBoreMarker and WellBoreTrajectory data. In order to achieve this, select a SRN from Table2 and click on one of the Work Product Component buttons at a time to get the respective data. In this step, we use the Search and Delivery APIs in the Get_WellLog_LAS, Get_Marker_Data and Get_Trajectory_Data data functions to retrieve the data. 
    5.thumb.png.d54a1da5a5fef213406aec90271c9fc6.png
    • You will also notice that each visualization gets updated with the latest data when a data function is successfully executed.

         6.thumb.png.b03ab64933220a9e0061134903a271f3.png

     

     

     

    OSDU R1 includes Work Product Components such as well logs, WellBoreMarker and WellBoreTrajectory data. Once the data is loaded in Spotfire, anyone can explore data to bring insights.

    Whether creating simple dashboard metrics, predictive applications, or dynamic real-time analytics applications, Spotfire® delivers numerous capabilities at-scale, including visual analytics, data wrangling, predictive analytics, location analytics, and streaming analytics.

    Next Steps:

     

    Related Assets:

     

    Acknowledgments & References:

    Special thanks to the TIBCO Data Science team who are working on these analyses using Spotfire (Visual Analytics; Python): Michael O'ConnellVinoth Manamala, Andrew Berridge, Catalina Herrera.

                7.png.fbe9f45610e53532ad04098168febb0d.pngMichael O'Connell, Ph.D., is the chief analytics officer at TIBCO, where he helps clients with analytics software applications that drive business value. He has written a bunch of scientific papers and software packages on statistical methods. He also likes listening to electronic music; watching basketball, football and cricket; going to art galleries and walking around neighborhoods.
    8.jpg.bb832f268ad1ddbe9864b625f92d16dd.jpgWith a passion for data and analytics, Catalina Herrera is a Data Scientists at TIBCO and has spent her entire career helping industry push beyond digitization to business transformation. She?s held both educational and technical positions, worked with state-of-the-art technology solutions across multiple industry verticals, and served as a data scientist and a SaaS technology evangelist. Today she works with Fortune 100 companies and global technology leaders on digital transformation initiatives.
    9.thumb.jpg.03cecc1fc19b7cb8d31e6ba09c931f7c.jpgVinoth Manamala is a Senior Data Scientist working in the Data Science Team at TIBCO, based in Toronto. He is passionate about solving complex industry problems using Big Data and machine learning. He brings his experience to his current role where he works with organizations to use their data assets to their fullest potential in support of their business operations. He is an avid sportsperson; plays cricket and badminton, and also loves to travel for nature and food.
    10.png.9f9b6adc00d95e984d690bfc9863a75b.png

    Andrew Berridge is a Senior Staff Data Scientist at TIBCO. He specializes in extending, customizing and developing Spotfire and driving its future in all industries. He is the author of TIBCO Spotfire, A Comprehensive Primer 2nd Edition (Packt). He loves restoring classic cars and playing in orchestras in his spare time.

     

     

    After you download the application, remember to "trust" the Scripts to avoid the "Unstrusted Data Function" message. If you are not sure how, take a look at this 

    .

    osdu_r1_aws.dxp


    User Feedback

    Recommended Comments

    There are no comments to display.


×
×
  • Create New...