@stefcud/geopicker

GeoPicker

Geospatial dataset picker via fast Api Rest interface written in NodeJs for GDAL bindings and Fastify

• Online Demo Map

• Swagger API

• Article about this project

Scope

It is basically an advanced elevation service and Geopicker has been specially designed to offer the widest range of formats and methods of data requests that is possible, to adapt to any context of use by the client. Each endpoint and the parameters it accepts have been designed on the basis of the functioning of already existing services, gathering a complete and coherent collection of APIs. At present the index.html page contains a large implementation of browser side requests using LeafletJs as basemap and jQuery.

Features

• Large Rest API: ergonomic endpoints suitable for any type of use case
• Validation: full validation of endpoint and parameters via JSON-Schema which allows output optimization
• Configuration: friendly configs and to help devs in many deployment contexts
• Formats: support for different geospatial input and output formats
• Compression: configurable output compression if client accept encoding: deflate,gzip

and includes some other additional functions:

• Densify: add more interpolated points in input coordinates, this improves the display on an elevation graph, adding intermediate positions at a minimum fixed distance.
• Simplify: unlike densify it removes points that are too close together from coordinates.
• Height: add the vertical distance from the ground, if input has elevation add a fourth coordinate with this value.
• Metadata: get additional informations for a certain geometry, for example: length, direction, bbox, centroid, middlepoint

API Rest endpoints

The API is work in progress. This basic structure can be extended starting from the environment variable PREFIX which by default /

(✔️ Work ❌ TODO 🚧 Work in Progress)

Status	Method	Path	Return	Description
✔️	GET	/	html	default demo map page if enabled by env var DEMO_PAGE=true
✔️	GET	/status	object	service status, versions, datasets
✔️	GET	/datasets	array	list available datasets and their attributes
✔️	GET	/datasets/:datasetId	object	search dataset by id
✔️	GET	/datasets/:lon/:lat	array	search dataset contains lon,lat
✔️	GET	/:datasetId	object	show attributes of a certain dataset by id
✔️	GET	/:datasetId/:lon/:lat	array	get single location value of dataset, densify not supported
✔️	GET	/:datasetId/:locations	array	locations is a string (format: `lon,lat
✔️	POST	/:datasetId/lonlat	arrays	accept array or object in body
✔️	POST	/:datasetId/locations	arrays	accept array or object of locations in body (format is [[lon,lat],[lon,lat],[lon,lat]])
✔️	POST	/:datasetId/geometry	object	geojson Point or LineString in body (support feature/geometry/f.collection)
✔️	GET	/metadata/:locations	object	return info about direction, length, centroid, middlepoint of locations
✔️	POST	/metadata/geometry	object	return info about direction, length, centroid, middlepoint of geometry

Global Parameters

Status	Parameter	Default	Description
✔️	precision	input	rounded to digits decimal precision
✔️	format	input	output format conversion
✔️	densify	input	enable densification of points in the result
✔️	simplify	input	enable simplication geometry of the result
❌	height	false	add vertical distance from the ground(only input has elevation)

Some behaviors to know about parameters are that:

• precision and densify parameters is only supported by endpoints and formats that return coordinates
• datasetId can have the value default to referring the main dataset defined in config
• from version v1.6.1 /<datasetId>/... is the same of /datasets/<datasetId>/... /datasets/ is implicit.

Formats

If the format parameter is not specified the default behavior is to output the same format as the input

• input format can be specified by Content-type: header in request
• output format can be specified by format parameter

the support for various input and output formats is summarized in the table

Value	In	Out	Description
input	✔️	✔️	means the same format as the input data
array	✔️	🚧	each location is Array and a Z dimension as value [lon,lat,val]
json	✔️	🚧	each location is Object having lon,lat and val attributes
geojson	✔️	🚧	standard GeoJSON objects Feature, Geometry with a Z dimension in coordinates as value
polyline	🚧	✔️	Encoded Polyline Algorithm
gpx	🚧	✔️	GPS eXchange Format is an XML textual format
csv	❌	❌	Comma-separated values is an textual format
kml	❌	❌	Keyhole Markup Language is an XML format for Google Earth

each endpoint has its own default format, for example endpoint /dataset/lon/lat return a simple array of one value.

Usage

Running by official Docker image:

docker run -v "/$(pwd)/tests/data:/data" -e DEMO_PAGE=true -p 9090:9090 stefcud/geopicker

Running from source code in development mode, requirements: nodejs 16.x > and glibc 2.28 (Ubuntu 20.x > ):

npm install
cd server && npm install && cd -
npm run dev

Browse the demo page: http://localhost:9090/

Configuration

Full configuration options can be found in docs config

Development

some useful tools for contributors npm run <scriptname>

• start run in production mode
• dev run in development mode
• docker-up run in local docker-compose container
• bench run benchmarks
• npm publish . build and publish new docker image

Requests Example

Get single location exchanging a few bytes:

 $ curl "http://localhost:9090/default/11.123/46.123"

[195]

$ curl "http://localhost:9090/default/11.123/46.123?format=gpx"

output is a waypoint in GPX format:

<gpx version="1.1" creator="Geopicker">
<metadata/>
<wpt lat="46.123" lon="11.123">
<name/>
<desc/>
<ele>400</ele>
</wpt>
</gpx>

Post a json object and receive the same decorated with the result(still works with longitude,latitude):

$ curl -X POST -d '{"lon": 11.123, "lat": 46.123"}' \
  -H 'Content-Type: application/json' \
  "http://localhost:9090/elevation/lonlat"

{"lon": 11.123,"lat": 46.123,"val":195}

Get many stringified locations in one time(designed for not too long LineString):

curl "http://localhost:9090/elevation/11.1,46.1|11.2,46.2|11.3,46.3"

[195,1149,1051]

Post a very long LineString saving bytes:

$ curl -X POST -d '[[10.9998,46.0064],[10.9998,46.0065],[10.9999,46.0066],[11.0000,46.0067]]' \
  -H 'Content-Type: application/json' \
  "http://localhost:9090/elevation/locations"

[[10.9998,46.0064,900],[10.9998,46.0065,898],[10.9999,46.0066,898],[11.0000,46.0067,900]]

Post anyone GeoJSON geometry, the same input geometry is always returned which has a third dimension:

$ curl -X POST -d '{"type":"LineString","coordinates":[[11.1,46.1],[11.2,46.2],[11.3,46.3]]}' \
  -H 'Content-Type: application/json' \
  "http://localhost:9090/elevation/geometry"

{"type":"LineString","coordinates":[[11.1,46.1,195],[11.2,46.2,1149],[11.3,46.3,1051]]}

Post a GeoJSON geometry of 2 points and Densify the linestring adding point each 400 meters, this will help you build a less angular elevation graph:

$ curl -X POST -d '{"type":"LineString","coordinates":[[11,46],[11.01,46.01]]}' \
  -H 'Content-Type: application/json' \
  "http://localhost:9090/elevation/geometry?densify=400&precision=5"

output contains 3 additional interpolated locations with reduced precision digits:

{
    "type": "Feature",
    "geometry": {
        "type": "LineString",
        "coordinates": [
          [11, 46, 897],
          [11.003, 46.003, 968],
          [11.006, 46.006, 1029],
          [11.009, 46.009, 1122],
          [11.01, 46.01, 1187]
        ]
    }
}

Get the elevation value between two locations every 100 meters

curl "http://localhost:9090/elevation/11,46|11.01,46.01?densify=100"

[925,858,909,963,968,1001,1018,997,1025,1062,1064,1102,1115,1163,1187]

Benchmarks

benchmarks scripts: tests/benchmarks.js using AutoCannon

cd tests && npm install && cd -
npm run bench

The results testing a dataset of 2x2km geotiff

┌─────────┬──────┬──────┬───────┬──────┬─────────┬─────────┬──────┐
│ Stat    │ 2.5% │ 50%  │ 97.5% │ 99%  │ Avg     │ Stdev   │ Max  │
├─────────┼──────┼──────┼───────┼──────┼─────────┼─────────┼──────┤
│ Latency │ 0 ms │ 0 ms │ 0 ms  │ 1 ms │ 0.02 ms │ 0.16 ms │ 6 ms │
└─────────┴──────┴──────┴───────┴──────┴─────────┴─────────┴──────┘
┌───────────┬─────────┬─────────┬─────────┬─────────┬──────────┬─────────┬─────────┐
│ Stat      │ 1%      │ 2.5%    │ 50%     │ 97.5%   │ Avg      │ Stdev   │ Min     │
├───────────┼─────────┼─────────┼─────────┼─────────┼──────────┼─────────┼─────────┤
│ Req/Sec   │ 18111   │ 18111   │ 22783   │ 23471   │ 22175.28 │ 1473.21 │ 18099   │
├───────────┼─────────┼─────────┼─────────┼─────────┼──────────┼─────────┼─────────┤
│ Bytes/Sec │ 4.02 MB │ 4.02 MB │ 5.05 MB │ 5.21 MB │ 4.92 MB  │ 327 kB  │ 4.01 MB │
└───────────┴─────────┴─────────┴─────────┴─────────┴──────────┴─────────┴─────────┘

Req/Bytes counts sampled once per second.
# of samples: 11

244k requests in 11.01s, 54.1 MB read

Roadmap

for details see the descriptions in the Roadmap issues

Status	Goal
✔️	Swagger Documentation Interface
🚧	manage multiple datasets
🚧	ES6 modules
🚧	extend benchmarks for any endpoints
✔️	enable densify function
🚧	enable simply function
❌	unit testing
❌	support vector format in datasets, such as shapefile
❌	limit access by api key
❌	caching responses
❌	interfaces: websocket, jsonrpc
❌	command line interface

Copyright

Created by Stefano Cudini @zakis Distributed under the BSD 2-Clause license.