SAP Datasphere – review of selected updates in Q1 2024.

In the following article, I outline some of the SAP Datasphere updates that have been rolled out in the first quarter of 2024. The updates are designed to make the product easier for users to use.

Possibility to set the download frequency

Datasphere offers several ways to update the data in the system. Data can be retrieved, among other things, using delta and by scheduling the frequency of new data retrieval. When downloading data using delta, a special indicator is used (e.g. Timestamp showing the date and time for a particular record), based on which it is checked whether the records are already in the warehouse and only those that are not in the database are downloaded.

By planning the frequency of retrieval of data from the source, we do not have the possibility to check during retrieval whether the data already exist in the warehouse. With this approach, all data issued by the source is downloaded.

In both cases, the frequency of downloading new data can be set. Until recently, it was mostly possible to refresh the data most often every hour, which was a limitation. In Q1 2024, an update was introduced where the frequency can be set in minutes. As a result, we can refresh the data more frequently and thus benefit from newer data during analyses.

Data transformation with SQL

Several data processing scenarios are available in SAP Datasphere, ELT and ETL. In ELT (Extract, load, transform), we first load data from the source system into the warehouse, then process it. In this approach, we can, for example, create views based on data that already exists in the warehouse and then make them available for analysis. In the ETL (Extract, transform, load) approach, data from the source system, is first processed and then uploaded to the warehouse. The ETL approach is used in Transformation Flow, where we process the data using an SQL editor and save it to a local table. In Transformation Flow, only a graphical editor was available, which had limitations compared to an editor where you can write a SQL script. This quarter, a SQL editor has been added to Transformation Flow, so we can create scripts that give us more options than the graphical editor.

Improved organisation of facilities in spaces

Spaces in SAP Datasphere are places where data is processed and held. Spaces can be divided thematically, e.g. one space holds HR data and another holds sales data. Spaces can hold different objects, e.g. remote tables, which is the source of the data, transformation flows, where we create transformations or models, from which we create reports. By keeping a lot of different objects in one place, there is disorder, which makes it harder to find what we need. Folders allow you to organise objects in space so that you can move around more efficiently.

Monitoring changes in remote tables

Remote tables are a kind of connection to a data source located on another system. They do not hold the data within them. When we want to check the data, SAP Datasphere retrieves it directly from the source. Remote tables can be used as sources for other objects in the warehouse, and any change to them can have a negative impact on the objects, e.g. Data Flow, using such a table, can break down. In the last quarter, a mechanism was introduced to monitor changes to the sources on which remote tables are based. With this mechanism, SAP Datasphere compares the remote table with its source. Remote tables that are inconsistent with their source receive a status indicating an error. This happens when, for example, a column has been deleted in the source table. In this case, it must also be removed from the remote table. Remote tables that are consistent with their source receive a status indicating changes to be implemented.

Summary

The above article describes a selection of updates that have been made to SAP Datasphere in the fourth first of 2024.