Maps Wiki

Documenting the lessons learned in trying to make a topographic world map using ​Maperitive and other tools. Most of this wiki will describe the process used to create a world topographic map (ie: shading based on mountains and other terrain). Adding urban data such as streets, place names, and borders was attempted but not finished.

All of the files referenced here have been downloaded to or backed up on a WD Elements 6TB USB drive currently plugged into the Michigan computer at the SDI office. README files on this drive will describe the organization of all the raw and processed data there.

Other pages in this section:

• PythonScripts describes (and includes the sources) of python scripts used to generate the world map so far.
• ZoomLevels discusses a better way to have generated the map that wasn't quite what was used, and is an overview of the math behind Web Tiles.

Maperitive

Maperitive is a free application for generating maps from ​Open Street Map data. It is also able to generate terrain maps using a set of NASA-created global height data called SRTM. The end goal of the operation is to output the map as ​Web Tiles, a set of static images at varying "zoom levels" which can be tiled together to make a seamless map.

The downside to this is that the entire earth is very big and requires a lot of RAM to hold all the data when the map is zoomed in far enough. The solution to this was to use scripts to break the planet up into smaller pieces which can be individually processed.

Scripting

All commands in Maperitive that can be performed by clicking in the GUI can also be performed by writing and running a script. Most of the commands can be found on the ​Maperitive docs website, or by running Maperitive and clicking Help/Documentation, then the Commands Tab. You can run a script with the run-script command.

Maperitive scripts can be run from the command line without opening a GUI, which is helpful for memory-draining operations that would be made significantly slower by having to draw them on the screen. You can do this by entering the script name as a parameter to Maperitive.Console.exe.

Maperitive also has it's own Python library, "Maperipy", which exposes a lot of useful, more complicated functionality. Documentation can be found by running Maperitive and clicking Help/Documentation, then the Maperipy Tab. You can run python scripts using the Console version or with the run-python command. Maperitive provides many examples of Python scripts in the Maperitive\Python folder.

SRTM Data

Maperitive uses NASA SRTM data to generate shaded terrain maps. In the past, Maperitive was able to connect to NASA's servers and download the data as needed. Since then, NASA has locked down the servers behind a free login. A newer version of Maperitive exists which might have a workaround for this issue, but this documentation presumes Maperitive can't download the data. Instead of Maperitive downloading the data, it can be downloaded manually and then just placed in a location that Maperitive expects SRTM data to be; then it will not attempt to download it again.

The SRTM data comes in two resolutions:

• ​SRTMGL1, 1-arcsecond data, divides the earth into .hgt data files covering 1x1 degree lat/lon. Each file is 3601x3601 16-bit pixels (~26MB).
• ​SRTMGL3, 3-arcsecond data. Each file is 1201x1201 pixels (~2.8MB).

If downloading more than one file at a time, SRTM data is most easily obtained from NASA's servers using the ​Daac2Disk utility. This example command will download the entire SRTMGL1 dataset to the SRTMGL1 folder: Daac2Disk_win.exe --shortname SRTMGL1 --versionid 003 --nometadata --outputdir SRTMGL1. This utility can also be used to download data within a set of lat/lon bounds using the --bbox parameter.

To download the data from the web or using Daac2Disk, you will need NASA Earthdata account: ​https://urs.earthdata.nasa.gov/home. Once you have created your account, log in and from "My Applications", add the LPDAAC Data Pool.

Using SRTM in Maperitive

Once the entire world data is downloaded, a few things need to be set to use it properly in Maperitive. Copies of the STRMGL1 and GL3 datasets are on the USB drive in the folders STRMGL1 and SRTMGL3

Filling in the gaps: Areas of the world which are entirely ocean have no corresponding SRTM file, but Maperitive will try to download the file anyway. The fix for this was to generate all-zero height files for any lat/long missing from the downloaded data set. The Python script fill_terrain_tiles.py automates this process. The filled-in datasets are on the USB drive in the folders SRTMGL1_full and SRTMGL3_full

Linking the SRTM to Maperitive: In order for Maperitive to find the SRTM data, it must be placed in the correct folder: X:\Maperitive\Cache\Rasters\SRTM1 or X:\Maperitive\Cache\Rasters\SRTM3. On the USB drive, this is performed by creating a softlink to the SRTMGLx_full folder, rather than making another copy of the dataset. You can use the windows utility ​mklink to do this; it can be run from the command prompt.

To use the data in Maperitive, you must enter the command set-dem-source name=SRTM1 or set-dem-source name=SRTM3.

Generating the Shading: The nicest shading method Maperitive provides is called "Igor Shading", named after the creator of Maperitive. This will generate a single-color terrain shading of the world similar to the image below. Another method of shading called "hypsometric" provides a multi-colored shading where different altitudes have different colors.

Igor shading can be generated with the ​generate-relief-igor command. Explicit bounds can be specified, or the command will use the currently set geo bounds in Maperitive (​set-geo-bounds). An important parameter of the shading is sample-rate. This reduces the size of the image data by subsampling. A sampling rate of 1 will load the entire height image into RAM; a sampling rate of 2 divides this by 4, 3 by 8, and so on. However this also reduces the quality of the image accordingly, so higher sampling rates should only be used for lower zoom levels.

Black is the default color for shading but it can be set to other colors.

Other Utilities

GDAL

​GDAL, the Geospatial Data Abstraction Library, is a set of executables and python scripts which can be used to manipulate geo-referenced data in a lot of different ways. This includes the raw SRTM data as well as images (such as terrain shaded png files).

When Maperitive saves an image of the map as a PNG (not a web tile), it also generates a .georef text file which contains the coordinate bounds and size of the image. Unfortunately, GDAL uses a similar file in a different coordinate format, called an ESRI world file. Image.png should have a cooresponding worldfile Image.pngw. The python script convert_georef.py can be used to read in a folder of Maperitive .georef files and output ESRI worldfiles. Then GDAL tools can be used to manipulate the images.

osmfilter and osmconvert

​Osmfilter can be used to filter specific features out of an existing map. For example, you can use this command to create a map that contains only borders, major roads, major waterways, railways, airports, and place names of larger cities and towns. Osmfilter works fastest (but not fast) with .o5m inputs, which you can create with Osmconvert.

osmfilter d:\north-america.o5m --keep="highway=motorway =trunk =primary =secondary =tertiary =motorway_link =trunk_link =primary_link railway=rail =station boundary= natural= waterway=river =canal aeroway=taxiway =runway =aerodrome place=locality =village =hamlet liesure=park" > d:\na-filter.osm

​Osmconvert converts between common map file formats (ex: osm, o5m, pbf). It can also be used to trim maps down using a lat/lon bounding box or a specific shape in the map. For example, you can use Osmconvert to create a map of the state of Virginia from a map of North America.

Osmium

​https://osmcode.org/osmium-tool/ is a linux-based tool which provides many of the same features as Osmfilter and Osmconvert.

Osmosis

​https://wiki.openstreetmap.org/wiki/Osmosis can be used to trim maps down by lat/lon bounds, but can also be used to load data into and otherwise manipulate map databases.