This article provides the steps with Spotfire to access to these datasets and APIs by providing a ready-to-use dashboard app for viewing and analyzing OSDU data.

Overview:

Spotfire is really proud to support and enable its customers who are part of Open Group's Open Subsurface Data Universe Forum. The OSDU Forum is developing a data platform for the oil and gas industry, which will reduce silos and put data at the center of the subsurface community. As part of Release 1 through Release 3, the OSDU Forum released reference implementations with well-defined set of APIs providing a standard way of accessing the data in the OSDU Data Platform and making sure that all applications can use a single set of APIs to access all data sources in the OSDU Data Platform. Spotfire simplifies access to these datasets and APIs by providing a ready-to-use dashboard app for viewing and analyzing OSDU data.

Note: This Spotfire application was designed to access OSDU R3 data

Prerequisites:

Acquire OSDU credentials and instance details - Please contact the representative from your chosen OSDU cloud service provider
Download the OSDU Custom Connector for Spotfire and follow the step-by-step user guide in the release to set it up
Download the Map Chart Observer for Spotfire and follow the step-by-step user guide in the release to set it up
Spotfire 10.7 or later comes by default installed with the Native Python data function. (This is the preferred version. Spotfire 11.2 was used to create the current example.)
Spotfire 7.13 through Spotfire 10.6 client and server. Spotfire Server should have the Community Python data function extension installed to enable the execution of Python code.
If you don't have Spotfire you can download a trial version here.
Working knowledge of Python data functions in Spotfire dashboards. Please refer to the above links and also please take a look at this link for data functions in general.
Please install these required Python packages - pandas, numpy, requests, json, lasio. For installing packages, please refer to the below video

Running the Spotfire Analysis:

Please download the dashboard attached to this blog (scroll down on this page to the "attachments" section) and open it in the Spotfire application. When prompted to Trust the scripts in the dashboards, navigate to 'Menu->File->Manage trust and scripts' and click 'Trust All'. All the data functions used to create this example can be found under 'Menu->Data->Data function properties'. Every data function here is either linked to an action button or a hyperlink in the dashboard.

Now, please follow the step-by-step approach to see how we can consume the Authentication, Search, and Dataset APIs in the data functions to download the data into Spotfire:

We utilize the Authentication API built into the OSDU Custom Connector to retrieve an ID token and Access token for accessing protected resources in the following Cloud Service Providers: Google GCP, Microsoft Azure, and Amazon AWS. We plan to incorporate IBM Cloud. These credentials are stored in a Spotfire Table and used accordingly in other data functions. Upon opening the dashboard, a login window for the connector will display. The first time you open the analysis, you'll need to configure the connector according to the documentation supplied with the connector.

The Access Token expires every 30 minutes. At the top of the dashboard, click the 'Refresh Tokens' button in order to renew authentication credentials accordingly.
The 'Reset Page' button will empty all visuals and data tables; use it to restart the walkthrough from step 1 (there are 5 sequential steps numbered accordingly to run through this dashboard)

Zoom-in and Zoom-out in the top-left map chart to set your boundary extents. All the wells in these boundary extents will be retrieved in the next step.
After setting the boundaries, please click on the link in 'Click here to find all Wells in the map boundaries' to get all the well names and visualize these wells that reside in the selected extents. In this step we leverage a custom extension that finds the map extent coordinates, and we then use the Search API in the Get_Well_From_Extent data function to find wells within the boundaries.
Next, we mark one or more wells on the populated map to retrieve the selected wells' wellbore IDs. The selected wells and the newly retrieved wellbore information update the corresponding map, data table, and KPIs. Here, we use the Search API in the Get_Wellbore_From_Well data function.
Once we have the wellbore IDs, we can now retrieve their work product components such as well logs, wellbore markers, and wellbore trajectory data. First, please click on the link in 'Click here to filter for Specific Wells then get their Trajectory, Marker, and Well Log files' (you may also filter for certain wells from the 'Selected Wells and their Wellbores' table). This retrieves the dataset information for each wellbore. Each wellbore can have datasets for its trajectory, marker, and well log components. The 'Wellbores and their Datasets' table and corresponding unique count KPIs will update. We use the Search API and synchronously run the Get_Trajectory_Files, Get_Marker_Files, and Get_LAS_Files data functions.
Mark one of the rows in the table 'Wellbores and their Datasets'. By selecting a wellbore and its dataset files, we retrieve the data for each of the work product components available for that wellbore. We use the Dataset API and synchronously run the Get_Trajectory_Data, Get_Marker_Data, and Get_Welllogs_LAS data functions.

The visualizations at the bottom of the dashboard get updated with the latest work product component data from that single wellbore we just selected. Note that not all work product components contain data, so some might not display anything accordingly. The Well Log Mod is customizable for any work product component data.

The volume dashboard shows the data lineage of master data and work product components (i.e. where the data is from, when it was uploaded, which users uploaded the data, etc). Blue represents the master data; orange represents the work product component data. The digital format table shows file types and file sizes of work product component data.

We can drill into the visuals and data tables using the filters on the right panels. Here, we filtered for data in Norway.

Some master data and work product component data are related. For example, Wellbores (master) are associated with Marker and Trajectory data (work product components). Clicking on either the master data or work product component data table counts will filter for these relationships. Here, we click on master-data-Wellbore and the visuals only show master-data-Wellbore, work-product-component-WellboreMarkerSet, and work-product-component-WellboreTrajectory data.

Additional Notes:

Please watch the video above for a sample run through the dashboard. We believe this provides a good example that showcases a small subset of wells where the corresponding 'Wellbores and their Datasets' table is fully populated. Most of these wellbores have populated scatterplot visuals for these datasets (i.e. populated data across the work product components, which can be checked by marking 'Wellbores and their Datasets' table).

In Step 3 ('Lasso select groups of wells on the map'), we limit the number of returned wells in the Get_WellBore_From_Well data function to 1000. Starting from Step 3, we intend to study a subset of wells. Selecting a large number of wells in this step can slow subsequent operations.

OSDU R3 includes Work Product Components such as Well Logs, WellBoreMarker, and WellBoreTrajectory data. Once the data is loaded in Spotfire, anyone can explore data to bring insights.

Whether creating simple dashboard metrics, predictive applications, or dynamic real-time analytics applications, Spotfire® delivers numerous capabilities at-scale, including visual analytics, data wrangling, predictive analytics, location analytics, and streaming analytics.

Next Steps:

Download Spotfire Free Trial if you don't have Spotfire installed
Keep an eye on our OSDU community Site for the OSDU Spotfire App updates and next releases
This is work ongoing, please send your feedback or suggestions to Andrew.Berridge@cloud.com if related to OSDU data and visualizations
If you have a Spotfire question please add it here

Related Assets:

Acknowledgments & References:

Special thanks to the Spotfire Data Science team who are working on these analyses using Spotfire (Visual Analytics; Python): Michael O'Connell, Vinoth Manamala, Andrew Berridge, Sweta Kotha, Adam Faskowitz.

	Michael O'Connell, Ph.D., is the chief analytics officer at Spotfire, where he helps clients with analytics software applications that drive business value. He has written a bunch of scientific papers and software packages on statistical methods. He also likes listening to electronic music; watching basketball, football and cricket; going to art galleries and walking around neighborhoods.
	Vinoth Manamala is a Senior Data Scientist working in the Data Science Team at Spotfire, based in Toronto. He is passionate about solving complex industry problems using Big Data and machine learning. He brings his experience to his current role where he works with organizations to use their data assets to their fullest potential in support of their business operations. He is an avid sportsperson; plays cricket and badminton, and also loves to travel for nature and food.
	Andrew Berridge is a Senior Staff Data Scientist at Spotfire. He specializes in extending, customizing and developing Spotfire and driving its future in all industries. He is the author of TIBCO Spotfire, A Comprehensive Primer 2nd Edition (Packt). He loves restoring classic cars and playing in orchestras in his spare time.
	*Sweta Kotha* is a Data Scientist at Spotfire and a recent graduate from Carnegie Mellon University. Her experience spans data science, natural language processing, and biostatistics. She likes trying out new technologies and methods to address analytics challenges and is interested in effectively communicating with data. She enjoys reading, running, and traveling.
	*Adam Faskowitz* is a Senior Data Scientist at Spotfire and recent graduate from UC Berkeley. His interests include data visualization, machine learning, and communication within data science. His passion for data science is driven by his curiosity for trying to understand complex problems and goal of creating meaningful solutions. In his free time, Adam enjoys watching and playing sports, going to art museums, and relaxing at the beach.

Reminder: After you download the application, remember to "trust" the Scripts to avoid the "Untrusted Data Function" message. If you are not sure how to take a look at this

osdu_r3m5_1.0.dxp

Sign In

Accessing the OSDU Environment in Spotfire