Tools and utilities for working with Uber Movement data
🚀 Getting Started
Using this toolkit requires Node.js to be installed on your machine, see Downloading and installing Node.js and npm. We recommend using nvm as described in that document, or brew install node for homebrew users. Using the official GUI installer from nodejs.org may lead to permissions issues when installing new commands.
Once node is installed, you can install and run the toolkit as shown in the sections below.
🛠 Usage
$ npm install -g movement-data-toolkit$ mdt COMMANDrunning command...$ mdt (-v|--version|version)movement-data-toolkit/1.3.0 darwin-x64 node-v8.16.2$ mdt --help [COMMAND]USAGE $ mdt COMMAND...🕹 Commands
mdt clear-cachemdt create-geometry-file [CITY] [YEAR]mdt help [COMMAND]mdt speeds-to-geojson [CITY] [STARTDATE] [ENDDATE]mdt speeds-transform [TYPE] [CITY] [STARTDATE] [ENDDATE]
mdt clear-cache
Some commands in this toolkit download files to a temporary directory. These files can be rather large, and it can be helpful to clear this data to reclaim disk space.
USAGE
$ mdt clear-cache
EXAMPLE
$ mdt clear-cache
mdt create-geometry-file [CITY] [YEAR]
Creates a geojson file that contains the geometries for each road segment in the specified city, as well as both the corresponding OSM identifiers for each road segment.
USAGE
$ mdt create-geometry-file [CITY] [YEAR]
ARGUMENTS
CITY Slug of the city to download data for. You can find this value in the URL when accessing data on the Movement
website.
YEAR Movement publishes mapping files and road geometries yearly, specify the year you would like to use here
OPTIONS
-o, --output=output File to write output to (writes to stdout by default)
--bbox=bbox Features outside of this bounding box will be removed from output. Should be of the form
minX,minY,maxX,maxY like "-10.0,-10.0,10.0,10.0" (see
https://tools.ietf.org/html/rfc7946#section-5)
--driving-side=LEFT|RIGHT [default: RIGHT] Road geometries are "twinned" in the output file to visualize speeds in
both directions, to do this accurately you should use --driving-side="LEFT" for places like
the UK or Australia.
--format=GEOJSON|CSV [default: GEOJSON] Format to use for the output file
--geom-format=GEOJSON|WKT [default: GEOJSON] Changes format of geometries in output file (only supported with
--format=CSV)
--log-level=INFO|DEBUG [default: INFO] What types of logs should be emitted?
--[no-]twin Whether or not to "twin" road geometries in the output file – this duplicates and offset
two-way roads so you can visualize data in both directions. Defaults to true.
DESCRIPTION
The produced file can help avoid the need to download and manipulate OSM geometries from the web yourself.
EXAMPLE
$ mdt create-geometry-file cincinnati 2018 > my-output-file.geojson
mdt help [COMMAND]
display help for mdt
USAGE
$ mdt help [COMMAND]
ARGUMENTS
COMMAND command to show help for
OPTIONS
--all see all commands in CLI
See code: @oclif/plugin-help
mdt speeds-to-geojson [CITY] [STARTDATE] [ENDDATE]
Downloads and processes Movement speeds data into a geojson file that can be visualized with Kepler.gl or other geospatial visualization tools.
USAGE
$ mdt speeds-to-geojson [CITY] [STARTDATE] [ENDDATE]
ARGUMENTS
CITY Slug of the city to download data for. You can find this value in the URL when accessing data on the
Movement website.
STARTDATE Fetch data starting from this date
ENDDATE Fetch data up to and including this date
OPTIONS
-o, --output=output File to write output to (writes to stdout by default)
--bbox=bbox Features outside of this bounding box will be removed from output. Should be of the form
minX,minY,maxX,maxY like "-10.0,-10.0,10.0,10.0" (see
https://tools.ietf.org/html/rfc7946#section-5)
--driving-side=LEFT|RIGHT [default: RIGHT] Road geometries are "twinned" in the output file to visualize speeds in
both directions, to do this accurately you should use --driving-side="LEFT" for places like
the UK or Australia.
--log-level=INFO|DEBUG [default: INFO] What types of logs should be emitted?
--[no-]twin Whether or not to "twin" road geometries in the output file – this duplicates and offset
two-way roads so you can visualize data in both directions. Defaults to true.
DESCRIPTION
Note: Becuase our speeds data CSVs have an hourly granularity, the results produced by this command are technically an
average of averages and may differ slightly from results shown in the app at movement.uber.com.
EXAMPLE
$ mdt speeds-to-geojson cincinnati 2018-01-01 2018-01-31 > my-output-file.geojson
mdt speeds-transform [TYPE] [CITY] [STARTDATE] [ENDDATE]
Downloads and transforms Movement Speeds data, filtering by date range and optional bounding box and producing a CSV with speed values mapped to OSM identifiers.
USAGE
$ mdt speeds-transform [TYPE] [CITY] [STARTDATE] [ENDDATE]
ARGUMENTS
TYPE Either "historical" for the monthly historical time series or "stats" for the quarterly statistics file.
CITY Slug of the city to download data for. You can find this value in the URL when accessing data on the
Movement website.
STARTDATE Fetch data starting from this date
ENDDATE Fetch data up to and including this date
OPTIONS
-o, --output=output File to write output to (writes to stdout by default)
--bbox=bbox Features outside of this bounding box will be removed from output. Should be of the form
minX,minY,maxX,maxY like "-10.0,-10.0,10.0,10.0" (see
https://tools.ietf.org/html/rfc7946#section-5)
--log-level=INFO|DEBUG [default: INFO] What types of logs should be emitted?
EXAMPLE
$ mdt speeds-transform historical cincinnati 2018-01-01 2018-01-31 > filtered.csv
📝 License
The code in this package is licensed under the MIT License, see LICENSE file at the root of this project.
Data specific to Uber Movement – including speeds or travel times – is licensed under the Creative Commons, Attribution Non Commercial License
Data from OpenStreetMap – including road geometries or other OSM metadata and identifiers – is licensed under the Open Data Commons Open Database License (ODbL) by the OpenStreetMap Foundation (OSMF).
⚠️ Troubleshooting
Out of memory Errors (OOM)
Occasionally you may run into an error that looks something like this:
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
Some of the scripts in this toolkit are processing a lot of data. By default Node.js uses 512mb, but we can increase that limit with another command line flag:
export NODE_OPTIONS="--max_old_space_size=8192" && mdt [command]
The above command would allow the node process to use up to 8gb of memory while it's running.
EACCES permissions errors when installing
The default Node installer from nodejs.org installs Node in a way that can cause permissions issues when you run npm packages globally.
You can either re-install node using a node version manager or change npms global package directory to one you have access to.
👋 Issues & Feedback
Questions? Issues? Feedback? Let us know on Movement's Contact Form.
