Rendering TopoJSON with Javascript code

Despite the wealth of libraries and APIs available to assist us in writing solutions for otherwise complex tasks, there are times when I enjoy ignoring such conveniences and take a shot at learning how to write my own solutions for such problems. That and sometimes you cannot rely on having access or availability to external resources. As luck would have it, both of these conditions had presented themselves when I was recently tasked to create a map-based tool for a work project.

When wrestling with the logistics of rendering maps, optimal solutions may involve leveraging Google Maps API to greatly simplify your life. If you require greater control over your map styling and behavior, the D3.js data visualization library has hooks that allow for easy rendering of geological data, as well as a variety of supported projection options (with full animation capabilities to boot). Additionally, one could simply add an SVG map file into your markup for hassle-free rendering.

Sometimes we just want to roll up our sleeves and try writing solutions ourselves - you know, for educational purposes. Before diving into the task, it would be of great benefit to first understand the JSON representation formats of our map data. Let’s review.

GeoJSON

GeoJSON appears a lot like it sounds - a representation of geographic coordinates defining a collection of polygons in JSON format. The required member type describes the GeoJSON object and its expected structure. For mapping land masses, this object is typically of type Feature or FeatureCollection. An optional member properties of either Feature or FeatureCollection object type describes a key-value dictionary of unique object properties. A FeatureCollection will contain array member of features. Feature objects contain a required member geometry, which houses members type and coordinates. A sample structure might look something like the following:

sampleStructure.geo.json
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
{
"type": "FeatureCollection",
"properties": {
"name": "Collection of Things"
},
"features": [{
"type": "Feature",
"properties": {
"name": "Feature1"
},
"geometry": {
"type": "Polygon",
"coordinates": [ /* coordinate pairs housed here */ ]
}
}]
}

If one of GeoJSON’s strengths is its readability, one of its greatest flaws is its file size. Each GeoJSON coordinate pair is typically represented with floating-point values. High-fidelity maps can easily bloat into megabytes worth of data. Each polygon must contain a clock-wise LinearRing of coordinates to define the shape in its entirety. As a result, features within a collection with neighboring borders share potentially redundant information. The more neighboring features of higher fidelity your data represents, the greater the burden all that redundant data becomes!

TopoJSON

Enter TopoJSON, a specification co-authored by Mike Bostock and Calvin Metcalf intended to simultaneously enhance, codify and compress the GeoJSON format. Rather than storing each feature’s full geometry, geographic borders are codified and amassed into a required array member arcs. Each feature’s child array member arcs references the master arcs array by index, defining its shape. A sample structure of a collection of geometry may appear as follows:

sampleStructure.topo.json
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
{
"type": "Topology",
"transform": {
"scale": [0.004523089582143, 0.00042398310994],
"translate": [-55, 210]
},
"objects": {
"objectName": {
"type": "GeometryCollection",
"geometries": [
{
"type": "Polygon",
"id": 1234,
"properties": {"name": "PolyName"},
"arcs": [ [0, 1, 2] ]
}
]
}
},
"arcs": [
[ [55, 56], [56, 58], [58, 60] ],
[ [58, 60], [41, 72] ],
[ [41, 72], [49, 67] ]
]
}

In this scheme, shared border coordinates are stored only once. Additionally, all floating-point data is quantized, allowing integer representation within each coordinate pair. The end result is a greatly compressed scheme relative to its original GeoJSON source.

Mike Bostock has a remarkably in-depth blog post covering the details of converting shapefile data into TopoJSON format. Additionally, you can easily convert existing GeoJSON data to TopoJSON data using the distillery web-tool.

Converting TopoJSON to GeoJSON

Comparing the two data structures, it may become evident that rendering map data from the GeoJSON format would be more straight-forward. For all of its compression advantages over GeoJSON, I found rendering maps with TopoJSON-encoded data to be extremely challenging. When rendering a collection of features which share adjacent arcs, many features do not hold arc definitions that a neighboring feature already contains. The end result is excellent for rendering meshes, but leaves LineString polygon definitions incomplete, making polygon fills impossible. This short-fall can be solved by one of three methods:

  • Create an alternate object definition within the member objects which define arc patterns that completely encapsulate each feature’s perimeter geometry

  • attempt to sort through the raw arc definitions via script and stitch together shapes from mutually shared arcs

  • download (or link to) Mike’s TopoJSON API that makes conversion of TopoJSON to GeoJSON format a cinch

I know the whole goal of this exercise was to write as much coolness ourselves as possible, but I strongly encourage the use of the TopoJSON API option. It’s light-weight and already solves some of these difficult problems for us.

1
<script src="http://d3js.org/topojson.v1.min.js"></script>

Setting Up

Before we build a map we need to source the data. For this exercise, I figure it would be fun to map the congressional districts of the world’s beloved New York City. There are a number of GeoJSON data plots for various districts of interest within the city available on Github. I’ve saved a local copy of this raw data to upload to distillery and ultimately export the compressed TopoJSON data back to my local disk. Distillery allows you to choose from a short list of projection types for your convenience. I’ve chosen the equirectangular option for the time being as I’d like to apply our own projection algorithm at some point in the future.

I now have a 12kb TopoJSON file representing New York City’s congressional districts saved as a local resource.

If you are hoping to pull in this resource asynchronously while running on your local filesystem, I would encourage you to also run a virtual hosting service as not to trigger most browser’s cross-origin blocking routines. Asynchronous calls accessing the filesystem is typically prohibited practice. If you are looking for quick, no-cost solutions to get around this, both MAMP for OSX and WAMP for Windows are relatively easy to install and configure.

The Javascript

Lets first take some time to define some rendering options for ourselves within an object literal. These options will define some basic properties regarding how our data will be rendered, including fill and stroke colors, line widths, any padding we want to calculate into our display container.

1
2
3
4
5
6
var renderOptions = {
padding: 50,
fillColor: "#d5e6e4",
strokeColor: "#88a9bc",
strokeWidth: 1
};

We will ultimately need to reference our TopoJSON objects member by property name. This value will unique to your JSON data, so let’s store that value as a variable. My TopoJSON propery name happens to be “nyc_congress_districts”. You may edit this value in your TopoJSON file to configure the naming convention to your liking.

1
var topologyKey = "nyc_congress_districts";

Next, we need to fetch some map data. You can store it as a string within your document, or you can more elegantly pull that resource in asynchronously on-demand. Using an XMLHttpRequest, we can do this rather succinctly (and in this abbreviated example, naïvely).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
// Note: running this script locally will fail for most browsers because of cross-origin policies
(function fetchTopologyData(sourceURL, callback) {
// create our xhr instance
var xhr = new XMLHttpRequest();
// open the connection with async flag set to 'true'
xhr.open("GET", sourceURL, true);
// add listener to readystatechange event
xhr.addEventListener("readystatechange", handler);
// send the request
xhr.send();
// handler function definition
function handler(e) {
if (xhr.readyState === 4) {
callback(xhr.responseText);
}
}
}(dataSourceURL, handleResponse));

Our callback function will be looking for a stringified version of our JSON data. Using the browser’s native JSON object’s method stringify(), we can quickly convert the text response to a javascript object literal. Using the TopoJSON API’s feature() method we can convert a high-level TopoJSON object back into a GeoJSON structure. Attributes are preserved and the delta-encoded arc data are calculated back to absolute latitude/longitude values. We are then free to more easily render polygonal shapes in either SVG or canvas path form.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
function handleResponse(responseText) {
try {
// parse our stringified data
var topology = JSON.parse(responseText || {}),
// convert topology to geojson format using topojson API
geojson = topojson.feature(
topology,
topology.objects[topologyKey]);
// TODO: map geojson coordinate data to a stage for rendering
// and write a routine to render geojson features onto stage
} catch (e) {
console.log("JSON parse failed", e);
}
}

Logging variable geojson to the console should give you a break-down of its structure. Inspecting the coordinates member of any given feature’s geometry will present an array of geographical latitude/longitude pairs. We aren’t ready to render the data yet, however. The raw coordinate data has no relation to our rendering stage in its current form. Our goal from here is to project the raw coordinate data relative to the stage dimensions.

To achieve this, we first need to determine the extreme coordinate values of our FeatureCollection. If our JSON data does not contain a bbox member defining the FeatureCollection’s bounds, we must manually cycle through each features’ coordinates array and compare them to a reference of extreme latitude and longitude values. We want to evaluate both max and min values of latitude and longitude individually, not as a coordinate pair.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
// returns the extreme latitude & longitude values of the entire feature collection as an object
function findGeographicBounds(featureCollection) {
var bounds = {};
// loop through each feature's polygons
featureCollection.features.forEach(
function(feature) {
iterateFeaturePolygons(
feature,
compareExtremes);
});
return bounds;
// algorithm to determine bounds of all geometry positions in the featureCollection
function compareExtremes(polygon) {
polygon.map(
function(position) {
// compare subsequent positions and set min/max values conditionally
if (bounds.x && bounds.y) {
bounds.x.max = position[0] > bounds.x.max ? position[0] : bounds.x.max;
bounds.x.min = position[0] < bounds.x.min ? position[0] : bounds.x.min;
bounds.y.max = position[1] > bounds.y.max ? position[1] : bounds.y.max;
bounds.y.min = position[1] < bounds.y.min ? position[1] : bounds.y.min;
// first poly's first position becomes our reference for comparison
} else {
bounds.x = {};
bounds.y = {};
bounds.x.max = bounds.x.min = position[0];
bounds.y.max = bounds.y.min = position[1];
}
});
}
}

At this point, it becomes useful to write a function that will map a callback function to all polygons within a feature’s geometry set. Because the depth of the geometry’s coordinates array varies dependent on the geometry type, a MultiPolygon requires another iterative step than a Polygon. Our findGeographicBounds() function invokes iterateFeaturePolygons() to quickly iterate through all polygons of a feature regardless of its geometry type. The callback then iterates through each coordinate of the polygon and maps the comparison algorithm compareExtremes() to determine our absolute latitudinal/longitudinal range.

1
2
3
4
5
6
7
8
9
10
11
12
function iterateFeaturePolygons(feature, callback) {
switch (feature.geometry.type) {
case "MultiPolygon":
feature.geometry.coordinates.forEach(
function(polygons) {
polygons.map(callback);
});
break;
case "Polygon":
feature.geometry.coordinates.map(callback);
}
}

A console log of the resulting range for our New York City congressional district map looks something like this:

1
2
console.log( findGeographicBounds( GeoJSON ) );
// {x: {min: -74.25553579936584, max: -73.70000906387122}, y: {min: 40.49645096263256, max: 40.915532777004685}}

As we can see, in its current context, the entire map would not consume a single pixel. Now that we have our geographic bounds calculated with known maximum and minimum values, we can map each latitude/longitude pair as a relative value to our stage’s width and height.

We first find a range for both latitude and longitude values by subtracting the minimum from the maximum. With that calculated, we can then compute respective range to canvas dimension ratios for each axis. We will ultimately apply the greater of the two ratio values to our stage projection calculations. This leaves one axis in need of centering, for which a centering offset can easily be derived from the range, canvas dimension and greater ratio values. Once our variables are calculated, we then iterate through each feature in the collection to map the transformation algorithm to its polygons.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
function mapCoordinatesToStage(stage, featureCollection) {
// grab reference to stage dimensions
var canvasBounds = stage.getBoundingClientRect(),
// collect extreme bounding points in the featureCollection
geoBounds = findGeographicBounds(featureCollection),
// calculate ranges from max - min
rangeX = geoBounds.x.max - geoBounds.x.min,
rangeY = geoBounds.y.max - geoBounds.y.min,
// calculate ratio of ranges to bounding dimensions
ratioX = rangeX / (canvasBounds.width - renderOptions.padding),
ratioY = rangeY / (canvasBounds.height - renderOptions.padding),
// determine greater ratio to scale map coordinates to
greaterRatio = ratioX > ratioY ? ratioX : ratioY,
// determine axial centering offsets
centerX = (stage.width - rangeX / greaterRatio) / 2,
centerY = (stage.height - rangeY / greaterRatio) / 2;
// iterate through all features
featureCollection.features.forEach(
function(feature) {
// itrate through each polygon in the feature's geometry
iterateFeaturePolygons(
feature,
transformPositionsIn);
});
// coordinate transformation algorithm
function transformPositionsIn(polygon) {
polygon.map(
function(position) {
position[0] = (position[0] - geoBounds.x.min) / rangeX * (rangeX / greaterRatio) + centerX;
position[1] = (1 - (position[1] - geoBounds.y.min) / rangeY) * (rangeY / greaterRatio) + centerY;
});
}
}

Rendering The Data

Now that our FeatureCollection coordinate data can be mapped to our stage, we can can write a routine to render the geometry. The process to do this for both SVG and canvas stage’s is rather straight forward. Looping through each feature in the collection, we define a path by iterating through all mapped coordinate data within each feature polygon.

Syntactical requirements for building paths are very different between canvas and SVG. They both, however, will follow a similar pattern of iterating each coordinate and plotting the corresponding data. Both can be accomplished directly via javascript. Getting familiarized with building custom SVG structures can take some time, but the end process is also rather straight-forward.

Real-world applications of your map script will determine the best approach to render your data through either canvas or SVG format. This example will be rendering to an HTML canvas:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
// a method to draw a feature to an HTML canvas
function drawFeatureToCanvas(canvas, feature) {
// grab context reference and set context styling
var context = canvas.getContext("2d");
context.fillStyle = renderOptions.fillColor;
context.strokeStyle = renderOptions.strokeColor;
context.lineWidth = renderOptions.strokeWidth * canvas.ratio;
// grab all feature polygons and render them individually
iterateFeaturePolygons(
feature,
renderPoly);
// builds 2d path for canvas
function renderPoly(polygon) {
// start new path
context.beginPath();
polygon.forEach(
function plotPosition(position, index) {
// continue drawing path
if (index) {
context.lineTo(
position[0],
position[1]);
// move to first position in path
} else {
context.moveTo(
position[0],
position[1]);
}
});
// draw operations
context.stroke();
context.fill();
context.closePath();
}
}

A context needs to be created from the canvas element, and basic styling declarations need to be made. We will be using our rendering option data to provide a style guide for color theming and line-width. During each renderPoly() iteration, a new path is initialized. The context is moved to the first coordinate in the polygon, and creates a connecting path with successive coordinates in the array. The first and last coordinate in the polygon is automatically closed with a draw operation. Finally the path is closed and the loop repeats until there are no polygons left to render.

With our projection and rendering pieces in place, we can complete our json import handler to include those new function invocations:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
function handleResponse(responseText) {
try {
// parse our stringified data
var topology = JSON.parse(responseText || {}),
// convert topology to geojson format using topojson API
geojson = topojson.feature(
topology,
topology.objects[topologyKey]),
// reference to the canvas stage
canvas = document.getElementById("myCanvasID");
// project geojson coordinate data to the stage
mapCoordinatesToCanvas(
canvas,
geojson);
// iterate through each feature and render it to stage
geojson.features.forEach(
function(feature) {
drawFeatureToCanvas(
canvas,
feature);
});
} catch (e) {
console.log("JSON parse failed", e);
}
}

When executed, our rendering output results the following New York City congressional district map. All polygons are drawn directly to the canvas with our pre-defined rendering options applied. I’ve embedded a codepen to help visualize its components.

See the Pen topojson map rendering demo by steve s (@st0ven) on CodePen.

In future posts I hope to explore additional topics such as map projections, rendering labels and points of interest, more in-depth rendering options between HTML canvas and SVG formats, and data visualization. Each topic is worthy of its own write-up and would directly extend from the script covered in this post.