Skip to content

Using the synchronous RDDMS Client

The ETP protocol is designed to be used for long running connections using websockets. However, in many cases it is not practical nor necessary to keep a connection lying around. Instead we might do a limited set of tasks, e.g., downloading data, and then we no longer need the connection. To ease these shorter workflows, we have set up a synchronous RDDMS client — called RDDMSClientSync — that lets the user call the main functionality from RDDMSClient without having to manually set up and close the connection. Furthermore, this also avoids the need for using async and await.

Info

Even though RDDMSClientSync is made synchronous, the methods wrap the concurrent RDDMSClient. As such the "heavy lifting" of the methods are still executed concurrently.

In this tutorial we will repeat what is done in the previous tutorial "Using the RDDMS Client" to make it easier to compare the two clients.

See "Accessing the ETP server" for instructions on how to use the local open-etp-server or get an access token to a published instance. We will in this tutorial also work towards the local server. Connection towards the server is handled in the background by the RDDMSClientSync, and there is no need to use a context manager or similar as for the concurrent RDDMSClient.

Token expiration

The RDDMSClientSync stores the access parameters towards the ETP server in the class, and only use them when one of its methods is called to establish a connection. If the client is used in a long running program, e.g., in a notebook where it is set up once and then called much later, the token might expire and the client will then be unable to connect to the server. In this case you should fetch a new token, and then re-create the client using the non-expired token.

Uploading a regular surface to the ETP server using the synchronous client

This tutorial is more or less a copy of the tutorial "Uploading a regular surface to the ETP server". We will therefore make it more brief and avoid any terminal output.

The regular surface is set up in exactly the same way as in the tutorial or the concurrent client: 

import numpy as np

from rddms_io import RDDMSClientSync
import resqml_objects.v201 as ro


z = np.random.random((101, 103))
origin = np.array([10.0, 11.0])
spacing = np.array([1.0, 0.9])
u1 = np.array([np.sqrt(3.0) / 2.0, 0.5])
u2 = np.array([-0.5, np.sqrt(3.0) / 2.0])

originator = "<name/username/email>"
epc = ro.obj_EpcExternalPartReference(
    citation=ro.Citation(title="Demo epc", originator=originator)
)
crs = ro.obj_LocalDepth3dCrs(
    citation=ro.Citation(
        title="Demo crs",
        originator=originator,
    ),
    vertical_crs=ro.VerticalCrsEpsgCode(epsg_code=6230),
    projected_crs=ro.ProjectedCrsEpsgCode(epsg_code=23031),
)
gri = ro.obj_Grid2dRepresentation.from_regular_surface(
    citation=ro.Citation(
        title="Demo grid",
        originator=originator,
    ),
    crs=crs,
    epc_external_part_reference=epc,
    shape=z.shape,
    origin=origin,
    spacing=spacing,
    unit_vec_1=u1,
    unit_vec_2=u2,
)
The main difference is that instead of importing rddms_connect from rddms_io we instead import RDDMSClientSync from rddms_io directly.

Notably, we also do not import asyncio. As we do not need use async and await for the RDDMSClientSync we therefore avoid wrapping the script in an asynchronous function.

Setting up the client

The connection parameters are given by: 

uri = "ws://localhost:9100"
data_partition_id = None
access_token = None
dataspace_path = "rddms_io/sync-demo"
And we can then create an instance of the client via: 
rddms_client = RDDMSClientSync(
    uri=uri, data_partition_id=data_partition_id, authorization=access_token
)
The parameters are identical to the ones passed to rddms_connect, but no connection is established yet.

Connecting and creating a dataspace

We can now call RDDMSClientSync.create_dataspace to set up a new dataspace. The parameters are identical to the ones in the concurrent counterpart, RDDMSClient.create_dataspace. 

rddms_client.create_dataspace(
    dataspace_path,
    legal_tags=["legal-tag-1", "legal-tag-2"],
    other_relevant_data_countries=["country-code"],
    owners=["owners"],
    viewers=["viewers-1", "viewers-2"],
    ignore_if_exists=True,
)

Inner workings of RDDMSClientSync

When a method is called on RDDMSClientSync will set up a context manager via rddms_connect to establish a connection, then call the equally named method on RDDMSClient, and finally close the connection when leaving the context manager. The pattern goes like this: 

class RDDMSClientSync:
    ...
    def <some-method>(self, <parameters>):
        async def <some-method>():
            async with rddms_connect(
                self.<connection-parameters>
            ) as rddms_client:
                return await rddms_client.<some-method>(<parameters>)
        return asyncio.run(<some-method>())

Uploading the surface

We upload the surface using RDDMSClientSync.upload_model similarly to how it is done in the previous tutorial. 

rddms_client.upload_model(
    dataspace_path,
    ml_objects=[epc, crs, gri],
    data_arrays={
        gri.grid2d_patch.geometry.points.zvalues.values.path_in_hdf_file: z,
    },
)

Searching on the ETP server

We can search using RDDMSClientSync in the same way we did the concurrent client. 

dataspaces = rddms_client.list_dataspaces()

gri_resources = rddms_client.list_objects_under_dataspace(
    dataspace_path, data_object_types=[ro.obj_Grid2dRepresentation]
)

gri_lo = rddms_client.list_linked_objects(start_uri=gri_resources[0].uri)
See the previous tutorial for example output and a wider discussion on how these three different searching methods work.

Downloading the surface

We download the surface using RDDMSClientSync.download_models. 

ret_models = rddms_client.download_models(
    ml_uris=[gri_lo.start_uri],
    download_arrays=True,
    download_linked_objects=True,
)
As we only asked for a single uri (the gri_lo.start_uri) in the RDDMSClientSync.download_models-call, we get a list containing a single RDDMSModel in return. The obj_Grid2dRepresentation-object is then found via: 
ret_model = ret_models[0]
ret_gri = ret_model.obj
Since we used the flags download_arrays=True and download_linked_objects=True, the fields RDDMSModel.arrays and RDDMSModel.linked_models (respectively) will also be populated (if there are any linked objects and arrays). Our uploaded obj_Grid2dRepresentation only links to a obj_LocalDepth3dCrs-object (the obj_EpcExternalPartReference-objects are excluded from being added to the RDDMSModel.linked_models), and to get it we run: 
ret_crs = ret_model.linked_models[0].obj
The array is found using the path_in_hdf_file from the grid-object: 
ret_z = ret_model.arrays[
    ret_gri.grid2d_patch.geometry.points.zvalues.values.path_in_hdf_file
]

Delete objects and dataspaces

Here as in the previous tutorial we end by deleting all the objects and then delete the dataspace. 

all_resources = rddms_client.list_objects_under_dataspace(dataspace_path)
rddms_client.delete_model(ml_uris=[a.uri for a in all_resources])
rddms_client.delete_dataspace(dataspace_path)

Full script

Finally, we list the full script used throughout this tutorial. Here as well there are a few extra assert-statements that were not included in the examples above, but are kept to ensure that the tutorial example is kept up to date. 

import numpy as np

from rddms_io import RDDMSClientSync
import resqml_objects.v201 as ro


z = np.random.random((101, 103))
origin = np.array([10.0, 11.0])
spacing = np.array([1.0, 0.9])
u1 = np.array([np.sqrt(3.0) / 2.0, 0.5])
u2 = np.array([-0.5, np.sqrt(3.0) / 2.0])

originator = "<name/username/email>"
epc = ro.obj_EpcExternalPartReference(
    citation=ro.Citation(title="Demo epc", originator=originator)
)
crs = ro.obj_LocalDepth3dCrs(
    citation=ro.Citation(
        title="Demo crs",
        originator=originator,
    ),
    vertical_crs=ro.VerticalCrsEpsgCode(epsg_code=6230),
    projected_crs=ro.ProjectedCrsEpsgCode(epsg_code=23031),
)
gri = ro.obj_Grid2dRepresentation.from_regular_surface(
    citation=ro.Citation(
        title="Demo grid",
        originator=originator,
    ),
    crs=crs,
    epc_external_part_reference=epc,
    shape=z.shape,
    origin=origin,
    spacing=spacing,
    unit_vec_1=u1,
    unit_vec_2=u2,
)

uri = "ws://localhost:9100"
data_partition_id = None
access_token = None
dataspace_path = "rddms_io/sync-demo"

rddms_client = RDDMSClientSync(
    uri=uri, data_partition_id=data_partition_id, authorization=access_token
)

rddms_client.create_dataspace(
    dataspace_path,
    legal_tags=["legal-tag-1", "legal-tag-2"],
    other_relevant_data_countries=["country-code"],
    owners=["owners"],
    viewers=["viewers-1", "viewers-2"],
    ignore_if_exists=True,
)

rddms_client.upload_model(
    dataspace_path,
    ml_objects=[epc, crs, gri],
    data_arrays={
        gri.grid2d_patch.geometry.points.zvalues.values.path_in_hdf_file: z,
    },
)

dataspaces = rddms_client.list_dataspaces()

gri_resources = rddms_client.list_objects_under_dataspace(
    dataspace_path, data_object_types=[ro.obj_Grid2dRepresentation]
)

gri_lo = rddms_client.list_linked_objects(start_uri=gri_resources[0].uri)

ret_models = rddms_client.download_models(
    ml_uris=[gri_lo.start_uri],
    download_arrays=True,
    download_linked_objects=True,
)

assert len(ret_models) == 1

ret_model = ret_models[0]
ret_gri = ret_model.obj

assert len(ret_model.linked_models) == 1

ret_crs = ret_model.linked_models[0].obj
ret_z = ret_model.arrays[
    ret_gri.grid2d_patch.geometry.points.zvalues.values.path_in_hdf_file
]

assert ret_gri == gri
assert ret_crs == crs
np.testing.assert_equal(z, ret_z)

all_resources = rddms_client.list_objects_under_dataspace(dataspace_path)
rddms_client.delete_model(ml_uris=[a.uri for a in all_resources])
rddms_client.delete_dataspace(dataspace_path)