Persistable references (OSDU)¶
geodetic_engine.persistablereference reads and writes OSDU
persistableReference payloads. These are JSON envelopes around ESRI WKT
that OSDU records use to say which CRS, transformation or unit their numbers
are in.
Use it when your data comes from OSDU, or anywhere else that states CRSs and transformations as ESRI WKT, and you need to transform with exactly what the payload states.
What is different about it: a payload is read as a definition, not as a code to look up. The WKT parameters are what gets built. The authority code next to them is kept as provenance only. A payload whose code is wrong, or names a register this machine has never heard of, still means exactly what its WKT says.
The payloads used on this page¶
All are verbatim from an OSDU reference-data catalogue. Expand the cell to copy them.
The six kinds of payload¶
The type member says what a payload states. It is read as a
Kind:
|
Kind |
Parsed as |
Build with |
|---|---|---|---|
|
Late-bound CRS: a CRS on its own |
|
|
|
Early-bound CRS: a CRS bound to a hub by one transformation |
|
|
|
Single transformation |
|
|
|
Concatenated transformation |
|
|
|
Unit by scale and offset |
|
|
|
Unit by the ABCD formula \(y = (A + Bx)/(C + Dx)\) |
same |
Parsing¶
parse_persistable_reference()
reads a payload and returns the matching reference type:
from geodetic_engine.persistablereference import parse_persistable_reference
for payload in (ED50_UTM32N_VIA_1612, WGS84, ED50_TO_WGS84_1612, CHOS_MALAL_TO_WGS84, FOOT):
ref = parse_persistable_reference(payload)
print(f"{type(ref).__name__:19} {ref.kind.value:4} {str(ref.authority_code):16} {ref.name}")
CrsReference EBC OSDU:23032023 ED50 * EPSG-Nor N62 2001 / UTM zone 32N [23032,1612]
CrsReference LBC EPSG:4326 GCS_WGS_1984
OperationReference ST EPSG:1612 ED_1950_To_WGS_1984_23
OperationReference CT EPSG:8517 Chos Malal 1914 to WGS 84 (1)
UnitReference USO None foot
Every reference keeps the original payload (raw), its name, version (the
producer’s ver, such as PE_10_9_1) and authority_code, an
AuthorityCode or None.
Encoded and embedded payloads¶
Payloads are often URL-encoded, and URL encoding is decoded automatically:
import json
from urllib.parse import quote
parse_persistable_reference(quote(WGS84)).name
'GCS_WGS_1984'
A payload is also often stored as a string inside another JSON document, such
as an OSDU record’s persistableReference field. Take the string out of the
document first. The package does not search inside other documents, and a
payload that is itself JSON-encoded a second time is refused as malformed:
record = json.loads(json.dumps({"kind": "osdu:wks:...", "persistableReference": WGS84}))
parse_persistable_reference(record["persistableReference"]).name
'GCS_WGS_1984'
Recognising a payload cheaply¶
looks_like_reference() decides
from the shape of a string whether it is a payload, without parsing it. It is
cheap enough to run on every CRS argument, which is how
transform() accepts payloads and codes in the
same argument:
from geodetic_engine.persistablereference import looks_like_reference
for value in (WGS84, quote(WGS84), "EPSG:4326", 'GEOGCS["WGS 84",...]'):
print(looks_like_reference(value), "|", value[:40])
True | {"authCode":{"auth":"EPSG","code":"4326"
True | %7B%22authCode%22%3A%7B%22auth%22%3A%22E
False | EPSG:4326
False | GEOGCS["WGS 84",...]
CRS references¶
An early-bound payload builds a pyproj.crs.BoundCRS that carries the
transformation:
ref = parse_persistable_reference(ED50_UTM32N_VIA_1612)
print("bound:", ref.is_bound)
print("late-bound part:", ref.late_bound.name, ref.late_bound.authority_code)
print("bound operation:", ref.operation.name, ref.operation.authority_code)
crs = ref.to_crs()
print(type(crs).__name__, "|", crs.source_crs.name, "->", crs.target_crs.name, "via", crs.coordinate_operation.name)
bound: True
late-bound part: ED_1950_UTM_Zone_32N EPSG:23032
bound operation: ED_1950_To_WGS_1984_23 EPSG:1612
BoundCRS | ED50 / UTM zone 32N -> WGS 84 via ED_1950_To_WGS_1984_23
The package’s own CRS type can be built from a payload directly:
from geodetic_engine.geodesy import CoordinateReferenceSystem
crs = CoordinateReferenceSystem.from_persistable_reference(ED50_UTM32N_VIA_1612)
crs.name, crs.value_axis_abbreviations, crs.axis_units
('ED50 / UTM zone 32N', ('E', 'N'), ('metre', 'metre'))
Transforming with payloads¶
Payloads can go wherever a CRS is accepted. Here both ends are payloads, so nothing is looked up by code. The bound source CRS names its own datum shift, so no operation is passed. Input is ED50 / UTM 32N (E, N in metres). Output is WGS 84 / UTM 32N:
from geodetic_engine.geodesy import transform
result = transform(ED50_UTM32N_VIA_1612, WGS84_UTM32N, (500000.0, 6650000.0))
print(result.coordinates)
print(result.operation.route, "|", result.operation.name)
((499920.4690604057, 6649793.771683395),)
bound | ED_1950_To_WGS_1984_23
The same CRS named by its plain EPSG code has no binding, so the datum change is ambiguous and refused:
transform("EPSG:23032", WGS84_UTM32N, (500000.0, 6650000.0))
AmbiguousOperationError: EPSG:23032 to EPSG:32632 involves a datum change; every datum operation must be named explicitly. allow_any_operation no longer permits automatic selection or ballpark results
A stated operation¶
An ST or CT payload, a bare ESRI GEOGTRAN string, or a parsed
OperationReference can be the
operation=. It is applied exactly as stated: its parameters are used,
not the parameters PROJ’s database has under the same code.
result = transform("EPSG:4230", "EPSG:4326", (2.5, 63.5), operation=ED50_TO_WGS84_1612)
print(result.coordinates)
print(result.operation.route, "|", result.operation.name)
# The bare GEOGTRAN inside the payload works the same way.
geogtran_wkt = json.loads(ED50_TO_WGS84_1612)["wkt"]
print(transform("EPSG:4230", "EPSG:4326", (2.5, 63.5), operation=geogtran_wkt).coordinates)
((2.4981894757094416, 63.49961374934551),)
chained | ED_1950_To_WGS_1984_23
((2.4981894757094416, 63.49961374934551),)
A stated operation is applied alone and cannot be mixed with other named operations in a list.
Operation references¶
chain = parse_persistable_reference(CHOS_MALAL_TO_WGS84)
print("concatenated:", chain.is_concatenated)
print("methods :", chain.method_names)
operation = chain.to_operation() # a pyproj CoordinateOperation
print(operation.name, "|", operation.type_name)
for step in operation.operations:
print(" ", step.name, step.method_name)
concatenated: True
methods : ('Geocentric_Translation', 'Geocentric_Translation')
Chos Malal 1914 to WGS 84 (1) | Concatenated Operation
Chos_Malal_1914_To_Campo_Inchauspe Geocentric translations (geog2D domain)
Campo_Inchauspe_To_WGS_1984_2 Geocentric translations (geog2D domain)
operation_from_geogtran() builds a
pyproj.crs.CoordinateOperation from a bare GEOGTRAN string with no
envelope. PROJ cannot read GEOGTRAN, so this package parses it itself.
Units¶
A unit payload converts values to and from SI, or directly to another unit of the same quantity:
foot = parse_persistable_reference(FOOT)
metre = parse_persistable_reference(METRE)
print(foot.symbol, foot.measurement, "scale:", foot.scale, "offset:", foot.offset)
print("1000 ft in m:", foot.to_si(1000.0))
print("100 m in ft :", foot.from_si(100.0))
print("1000 ft -> m:", foot.convert_to(metre, 1000.0))
ft length scale: 0.3048 offset: 0.0
1000 ft in m: 304.8
100 m in ft : 328.0839895013123
1000 ft -> m: 304.8
Converting between units that measure different things, such as length and
temperature, raises
UnsupportedReferenceError.
Writing payloads¶
to_persistable_reference() writes a
CRS (bound or not) or a transformation as a payload. PROJ writes ESRI WKT for
CRSs only. For a bound CRS it silently drops the transformation. So the
GEOGTRAN is assembled by this package from PROJ’s definition, with ESRI’s
method names and each parameter in ESRI’s units:
from pyproj import CRS
from pyproj.crs import CoordinateOperation
from geodetic_engine.persistablereference import AuthorityCode, to_persistable_reference
payload = to_persistable_reference(CoordinateOperation.from_epsg(1612))
print(payload[:260], "...")
{"type":"ST","name":"ED50 to WGS 84 (23)","wkt":"GEOGTRAN[\"ED50 to WGS 84 (23)\",GEOGCS[\"GCS_European_1950\",DATUM[\"D_European_1950\",SPHEROID[\"International_1924\",6378388.0,297.0]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433]],GEOGCS[\"G ...
No authority code or version is written unless you supply one. A code is a claim about a register, and the package will not make it for you:
payload = to_persistable_reference(
CRS.from_epsg(23032), authority=AuthorityCode("EPSG", "23032")
)
print(json.loads(payload)["authCode"], json.loads(payload)["type"])
{'auth': 'EPSG', 'code': '23032'} LBC
Anything written can be read back to the same definition:
round_trip = parse_persistable_reference(to_persistable_reference(CRS.from_epsg(23032)))
round_trip.to_crs().equals(CRS.from_epsg(23032), ignore_axis_order=True)
True
geogtran() builds just the
GEOGTRAN element as a tree, and
write() serialises it:
from geodetic_engine.persistablereference import geogtran
from geodetic_engine.persistablereference.esriwkt import write
print(write(geogtran(CoordinateOperation.from_epsg(1612)))[:200], "...")
GEOGTRAN["ED50 to WGS 84 (23)",GEOGCS["GCS_European_1950",DATUM["D_European_1950",SPHEROID["International_1924",6378388.0,297.0]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],GEOGCS["GCS ...
Supported and refused methods¶
A payload is translated exactly, or refused. Several ESRI methods differ from a supported one only by a convention, or by a parameter ESRI does not state. Translating such a near miss would give coordinates that look right and are metres out. The tables below are generated from the package’s own method tables at build time.
| ESRI method | EPSG code | EPSG method | parameters |
|---|---|---|---|
| Geocentric_Translation | 9603 | Geocentric translations (geog2D domain) | X_Axis_Translation, Y_Axis_Translation, Z_Axis_Translation |
| Position_Vector | 9606 | Position Vector transformation (geog2D domain) | X_Axis_Translation, Y_Axis_Translation, Z_Axis_Translation, X_Axis_Rotation, Y_Axis_Rotation, Z_Axis_Rotation, Scale_Difference |
| Bursa_Wolf | 9607 | Coordinate Frame rotation (geog2D domain) | X_Axis_Translation, Y_Axis_Translation, Z_Axis_Translation, X_Axis_Rotation, Y_Axis_Rotation, Z_Axis_Rotation, Scale_Difference |
| Coordinate_Frame | 9607 | Coordinate Frame rotation (geog2D domain) | X_Axis_Translation, Y_Axis_Translation, Z_Axis_Translation, X_Axis_Rotation, Y_Axis_Rotation, Z_Axis_Rotation, Scale_Difference |
| Molodensky_Badekas | 9636 | Molodensky-Badekas (CF geog2D domain) | X_Axis_Translation, Y_Axis_Translation, Z_Axis_Translation, X_Axis_Rotation, Y_Axis_Rotation, Z_Axis_Rotation, Scale_Difference, X_Coordinate_of_Rotation_Origin, Y_Coordinate_of_Rotation_Origin, Z_Coordinate_of_Rotation_Origin |
| Molodensky_Badekas_Position_Vector | 1063 | Molodensky-Badekas (PV geog2D domain) | X_Axis_Translation, Y_Axis_Translation, Z_Axis_Translation, X_Axis_Rotation, Y_Axis_Rotation, Z_Axis_Rotation, Scale_Difference, X_Coordinate_of_Rotation_Origin, Y_Coordinate_of_Rotation_Origin, Z_Coordinate_of_Rotation_Origin |
| Longitude_Rotation | 9601 | Longitude rotation | Longitude_Offset |
| Geographic_2D_Offset | 9619 | Geographic2D offsets | Latitude_Offset, Longitude_Offset |
Grid methods (NADCON, NTv2, HARN) are also supported. The grid file’s
method and real filename are taken from PROJ’s grid_alternatives table. The
reversible polynomial of degree 4 (EPSG method 9651) is also supported: PROJ
has no implementation, so this package evaluates it with PROJ’s horner
operation.
| refused ESRI method | why |
|---|---|
| Molodensky | ESRI states no semi-major axis or flattening difference for it, so the EPSG form cannot be stated without inferring both from the two ellipsoids |
| Molodensky_Abridged | ESRI states no semi-major axis or flattening difference for it, so the EPSG form cannot be stated without inferring both from the two ellipsoids |
| Time_Based_Helmert_Position_Vector | a 14-parameter Helmert reads a coordinate epoch, which a persistableReference does not carry |
| Time_Based_Helmert_Coordinate_Frame | a 14-parameter Helmert reads a coordinate epoch, which a persistableReference does not carry |
| Time-specific_Position_Vector_transform_geocen | it is valid only for coordinates at its transformation reference epoch, and moving coordinates there needs velocities, which a persistableReference does not carry |
| GEOCON | it reads a grid format PROJ does not ship a reader for |
| NADCON5 | it reads a set of grids ESRI names as one dataset, which does not resolve to the single EPSG grid reference PROJ expects |
| NTv2_Velocity | it reads a velocity grid that needs a coordinate epoch, which a persistableReference does not carry |
| Null | ESRI states no parameters for it, so there is nothing to translate |
| Unit_Change | it states a change of unit rather than of datum, which belongs to the CRS definitions either side rather than to a transformation |
A refused method raises
UnsupportedMethodError with the
reason:
parse_persistable_reference(WGS84_TO_ETRS89_TIME_SPECIFIC).to_operation()
UnsupportedMethodError: Time-specific_Position_Vector_transform_geocen is not supported because it is valid only for coordinates at its transformation reference epoch, and moving coordinates there needs velocities, which a persistableReference does not carry
Errors¶
Exception |
Raised when |
|---|---|
The payload is not valid JSON, states no type and nothing that implies one, or has WKT PROJ will not read |
|
A well-formed payload states something this package cannot represent, such as a chain whose steps do not join |
|
The method or a parameter has no exact equivalent (table above) |
|
A grid-based transformation names a grid PROJ’s database has no mapping for |
All derive from
PersistableReferenceError.
When a payload is passed to transform() as a
CRS, these are reported as
UnresolvableCRSError.