This week's lesson focuses on tiled web maps. Tiles are relatively small square-shaped "chunks" of data (either rasterized map images or raw vector coordinates) that have been pregenerated by the server and stored in a directory called a cache. When web users navigate the map, the server can just hand out the tiles rather than generating the map on the fly.

In this lesson, you'll learn the pros and cons of tiled maps, as well as strategies for building and maintaining a tile cache. You will learn about traditional rasterized tiled image formats, as well as a newer generation of tiles that store vector coordinates. Because rasterized image tiles have been around much longer and have mature support from tile generating engines, tile servers, and clients, most of the lesson content focuses on these types of tiles. The newer vector tile format is discussed near the end of the lesson content.

There are two walkthroughs associated with this lesson, both of which involve rasterized image tiles because these currently have the most mature tools for end-to-end FOSS workflows. The first walkthrough shows how to create a simple cache of your Philadelphia neighborhoods map using the GeoWebCache software that is integrated into GeoServer. In the second walkthrough, you'll use QGIS to create a Philadelphia basemap cache with some nicer cartography than you would get using GeoServer.

Objectives

• Describe the advantages of tiled web maps and identify when it is appropriate to use them.
• Recognize strategies and techniques for creating and updating large tiled web maps.
• Describe the differences between rasterized image tiles and vector tiles, and the reasons that each might be used
• Create tiles for a WMS using GeoWebCache.
• Use best practices in multiscale map design to create a tiled basemap using QGISl.
• Discuss hosting options for tiled maps. Unpack and upload your tiled map to your own webspace (on AWS).

Checklist

• Read all of the Lesson 5 materials.
• Complete the two walkthroughs.
• Complete the Lesson 5 assignment.

As mentioned in previous lessons, the earliest web maps were typically drawn on the fly by the server, no matter how many layers were available or requested. These are the types of maps you just created using GeoServer and WMS. As you may have noticed, the symbol sets and labeling choices for this type of map are relatively limited and complex to work with. In fact, for many years, web cartographers had to build a map with a minimal layer set and simple symbols to avoid hampering performance. In many cases, a cartographer was not even involved; instead, the web map was made by a server administrator tweaking XML files that defined the layer order, symbol sizes, and so forth. This was the case with both open specification web services (like WMS) and proprietary web services (like Esri ArcIMS).

Part of this approach stemmed from early efforts to make web GIS applications look exactly like their desktop counterparts. Sometimes these are referred to as “Swiss Army Knife applications” because they try to do everything (you may know one!). People expected that in a web GIS they should be able to toggle layer visibility, reorder layers, change layer symbols on the fly, and do everything else that they were accustomed to doing on the desktop. Ironically, this mindset prevailed at a time when web technology was least suited to accommodate it.

In the mid-2000s, after Google Maps, Microsoft Virtual Earth (now Bing Maps), and other popular mapping applications hit the web, people started to realize that maybe they didn't need the ability to tinker with the properties of every single layer. These providers had started fusing their vector layers together in a single rasterized image that was divided into 256 x 256 pixel images, or tiles. These tiles were pregenerated and stored on disk for rapid distribution to clients. This was done out of necessity to support hundreds or thousands of simultaneous users, a burden too great for drawing the maps on the fly.

The figure below shows how a tiled map consists of a "pyramid" of images covering the extent of the map across various scales. Tiled maps typically come with a level, row, and column numbering scheme that can be shared across caches to make sure that tile boundaries match up if you are overlaying two tile sets.

Figure 5.1 Tiled web maps take the form of a pyramid where the map is drawn at a progressive series of scale levels, with the smallest (zoomed out) scales using fewer tiles.

Cartographers loved the tiled maps because now they could invest all the tools of their trade into making an aesthetically pleasing web map without worrying about performance. Once you had created the tiles, you just had a set of images sitting on disk, and the server could retrieve a beautiful image just as fast as it could retrieve an ugly one. And because the tiled map images could be distributed so quickly by a web server, Google, and others were able to employ asynchronous JavaScript and XML (AJAX) programming techniques to retrieve the tiles with no page blink as people panned.

This was revolutionary. Which would you rather have: a slippy map with stunning cartography and no layer control, or a clunky and ugly map with the ability to reorder layers and adjust the color of a school? Some longtime GIS geeks had to stop to think about this, but for the common web user, the choice was a no-brainer.

Within a year or two of Google Maps' release, commercial GIS software began offering the ability to build map tiles. For many, ArcGIS Server was desirable because the map could be authored using the mature map authoring tools in ArcMap; however, cost was a concern for some. Arc2Earth was another commercial alternative. The free and open source Mapnik library could also build tiles, but it wasn't until recent years that projects like TileMill wrapped a user-friendly GUI around Mapnik. QGIS now also offers the option to create tiles, as you will see in the second walkthrough of this lesson.

Figure 5.2

Credit: Tiles from OpenStreetMap data, rendered by MapQuest

Tiled maps were the only model that could reasonably work for serving complex web maps to thousands of simultaneous users. However, they eliminated the ability for users to change layer order or symbols. People started working around this by serving out their general-purpose basemap layers as tiles and then overlaying a separate layer with thematic information. The general-purpose basemap tiles could be re-used in many applications. The thematic layers could also be tiled if the data didn't change too quickly or cover too broad an area at large scales. For example, you saw in Lesson 2 that the Microsoft Bing Maps traffic layer is served as PNG tiles.

If you want faster navigation of your basemap, or you feel that more than a couple of users will be requesting maps simultaneously from your server, you should create a tile cache of your basemap. You may also choose to cache thematic layers if their features are not constantly changing attributes or position.

In both cases, be aware that the tile cache represents a snapshot of your data at the time the cache was created. To put it more bluntly, your tiles are “dumb images” that will not automatically update themselves when the back end data changes. You are responsible for periodically creating new tiles in order to update the map.

With large caches, sometimes server administrators target the cache updates at just the changed areas, rather than rebuilding tiles for the entire map. This requires keeping some kind of log about which places were edited, or comparing “before” and “after” versions of a dataset. Later in this course, you will see that OpenStreetMap.org works in this way. If you make an edit to OpenStreetMap, you'll see your changes appear within a few minutes (at least at the largest scales). This is not because the entire cache was rebuilt, but because an area of change was detected, thereby triggering a partial rebuild of the cache in just the modified area.

Is there an existing tile cache that meets your needs?

Building a tiled basemap requires lots of rich data, high-end map authoring software, cartographic skills, and potentially enormous amounts of time and disk space. You may still need to do it at one point or another, which is why you will get a taste of this experience in Geog 585. However, because of these challenges, general-purpose web mashups often use tiles made by somebody else. OpenStreetMap is an attractive option if you want free tiles with no restrictions. If you want to use Google's, Microsoft's, or Esri's tiles, you may be able to use them for free, or you may have to pay, depending on the nature of your map (commercial or not-for-profit) and how many people use your app. Other companies, such as CloudMade and MapBox have marketed their own versions of tiles using OpenStreetMap data.

If you are going to build your own basemap, it's helpful to have an experienced cartographer on staff who is experienced with designing at multiple scales. The symbols, colors, and details must be adjusted appropriately at each of the scales where tiles are created. Tiled basemaps can quickly get complicated with layer and labeling scale suppression rules. Cartographers may also need to design one set of tiles to stand on its own and another set of tiles to overlay remotely sensed imagery, a task that requires very different colors and symbols.

Projection

If you are going to overlay your tiles with any of the tiles from OpenStreetMap, Google, Microsoft, or Esri (or even just attempt to look like them), you must warp your most precious GIS data into a modified spherical Mercator projection that was created solely for the convenience of fitting the world onto a set of square tiles. GIS purists balked at this idea and predicted it would fail to achieve mass uptake, but many people (at least at mid-latitudes) now hold their nose and move forward with it.

Be aware that Esri, Google, and other organizations have used other code numbers and variations of this projection in the past: things can get very confusing if you are using older software or APIs. In the past couple of years, people seem to have standardized on EPSG:3857 although even the subparameters of this projection can be interpreted in diverse ways that lead to offsets. As a side note, can you figure out the humorous reason that Google once used the code 900913 for this projection?

Even when you display your maps in EPSG:3857, you do not ever want to perform measurements in this projection. The results will be largely skewed even at mid-latitudes. It's best to make sure that geometries are projected into a more local coordinate system before performing any measurements. Most popular mapping tools such as Google Earth and the ArcGIS.com map viewer do this behind the scenes when you use their measuring tools, but it doesn't happen automatically if you're building your own solution.

Scales

Not only must you match the projection in order to overlay, you must also match the scales. These are not nice, well-rounded numbers; rather, they were derived mathematically from the starting point of putting the whole world on a 2x2 grid of tiles. For example, one of the scales is 1:36,111.98 and as you zoom in, the next one is 1:18,055.99. So much for your simple USGS 1:24000 series! Partly for this reason (and partly because many laypeople don't understand map scales), the common web map scale set is often referred to with simple numbers such as “Level 14,” “Level 15,” etc., that increase as you zoom in. You just have to get a feel for which levels correspond to national scale, provincial scale, city scale, neighborhood scale, etc. The table in the Bing Maps Tile System article is helpful for this.

Map tiles often have a simple folder structure, which makes them easy to serve. However, they can become complex to manage due to their sheer size and number. There are several different ways people have devised to serve map tiles:

• Put them in a folder structure on your web server, and let people pull them directly. In this approach, you just organize the tiles as individual image files under a folder structure representing the scale level, row, and column. Many mapping APIs can use tiles when presented with a URL structure with variables representing the consistent position of the level, row, and column. For example, when using a tiled layer in the Leaflet API, you must provide a URL in the form http://{s}.somedomain.com/blabla/{z}/{x}/{y}.png, where z is the zoom level and x and y are the column and row.
• Make the tiles available through a web service. In this approach, the web service exposes parameters for you to request a tile based on a particular level, row, and column; however, the back-end infrastructure is a little more hidden. This approach has slightly more overhead (and therefore time lag) than exposing the tiles directly through a folder. A variation of the WMS specification called WMTS is designed for serving tiles and works in this manner. You can also see this pattern at work when you navigate around Google Maps using the network tool of your browser. The following tile URL for Google Maps reveals nothing about the back-end folder structure for the tiles, although you can see the level, row, and column variables at work if you look hard enough: https://mts0.google.com/vt/lyrs=m@241289412&hl=en&src=app&x=74&y=96&z=8&s=Galile

Strategies for creating tiles

At large (zoomed in) scales, the number of tiles to cache can be overwhelming, especially if you are covering a broad area such as a state or country. The irony is that at such large scales, many tiles will convey very little information. Zooming in to a neighborhood block scale at 1:2250 may show plenty of interesting features, but pan out into the desert or ocean at the same scale, and the tile may be completely blank. Do you want to spend the hours creating these tiles and gigabytes of disk space to store them?

In these situations, you may want to find software that can create tiles on demand, meaning at the time they are first visited by a user. The first person to navigate to a region will need to wait for the server to create the tiles, but subsequent visitors will enjoy the full benefits of created tiles. The most popular areas will fill up with tiles, but you won't spend resources creating and storing tiles that are never visited. Obviously, this approach varies in effectiveness based on how fast the server can actually draw the tiles on demand.

An alternative is to use a “Data not available” tile to denote areas where tiles have not been created. Web map administrators are sometimes loathe to do this, but it is often a common enough practice that lay users blame themselves when they see the tile (“Oops, I zoomed in too far!”) rather than the administrator (“Why didn't they make the map available at this scale?”).

The best approach may be to strategically create a subset of the most interesting tiles and leave the less interesting tiles to be created on demand (or returned using a “Data not available” tile). As a geographer, it may hurt your soul to call any place “less interesting,” but the bitter truth is that not all tiles of the map experience an equal number of views. Fisher (2007) showed how early visitors to Microsoft Virtual Earth (now Bing Maps) stuck mostly to the big cities, coastlines, and transportation corridors. Quinn and Gahegan (2010) built models taking into account these patterns, showing how the majority of map requests could likely be satisfied by creating a fraction of the full number of tiles that it would ordinarily take to cover the full rectangular extent of the map. Their models were cobbled together using datasets like buffered roads, coastlines, and points of interests, but more recent feeds from social media such as geotagged tweets and Flickr photos may prove even more accurate in revealing the most interesting regions of the map for the majority of users. Note that some types of specialized maps (for mineral exploration, or wilderness conservation) may have very different usage patterns than general-use basemaps.

The ability to selectively cache a subset of tiles like this depends on the tile creation software's flexibility in allowing the administrator to designate custom regions for caching. Most software just allows the submission of a rectangular bounding box for tile creation; however, interesting areas of the map such as cities and coastlines are usually not shaped like rectangles. If you identify an irregularly-shaped region where you want to create tiles, you may have to abstract it into a series of rectangles and run multiple tile creation jobs, using each rectangle as an input. If you are luckier, your tile creation software will accept a spatial dataset (such as a shapefile) as a bounding region.

Creating tiles with FOSS

Creating tiled web maps is a common task that has been addressed by various FOSS packages. The most accessible one for you at this point is GeoWebCache because it's integrated directly into GeoServer. Others include TileCache and TileStache.

The Mapnik library is a FOSS tile creation library that binds to Python and other languages. It allows a lot of advanced drawing options not found in your typical WMS layer. Working with Mapnik typically required some Linux knowledge and some trial and error; however, the for-profit company Mapbox released an open source program called TileMill that can run on Mac and Windows and puts a nice GUI around Mapnik. We have been using TileMill in this class for a long time, but a couple of years ago, Mapbox stopped developing it. TileMill has been continued as a community-driven project on GitHub, but the development has slowed and problems have become more frequent recently. At the same time, tile creation support has been integrated into QGIS, so at some point, we made the decision to use QGIS for this part of the class as well. In the second walkthrough in this lesson, you'll use QGIS to create a basemap of Philadelphia using some of the layers you processed earlier.

Although the rasterized tile sets we have discussed in this lesson are able to deliver nice-looking maps in a relatively rapid format, they can be cumbersome to keep updated, and they require enormous amounts of computing resources at large map scales. To work around these challenges, a data storage format called "vector tiles" has gained popularity in the past several years. Mapbox has led development efforts on vector tiles and has shared a vector tiles specification under a Creative Commons license.

Vector tiles are exactly what you would guess: they store chunks of vector data instead of storing a map image. The idea behind vector tiles is that it is more efficient to keep data styling separate from the data coordinates and attributes. The client can use a predefined set of styling rules to draw tiles of raw vector coordinate and attribute data sent by the server. This allows the restyling of data on the fly, which is another serious limitation of rasterized tiles. Think about it: If you want to change the shade of green used to draw parks with your rasterized tiles, you must rebuild every tile containing a park. If you want to do the same thing with vector tiles, you just update your styling instructions in one place and the tiles themselves stay the same. Other display operations such as rotating the map also become easier to implement with vector tiles.

Vector tiles are designed to be small on disk, and employ a number of optimization approaches designed to reduce the amount of characters needed to store the geographic data and attributes, some of which are described in this video by Mapbox engineer Dane Springmeyer. He also introduces a product called Mapbox Studio, which works with vector tiles only and is being promoted by Mapbox as a replacement for Tile Mill. The .mbtiles file format, which originally stored rasterized tiles, now only stores vector tiles when exported from Mapbox Studio.

In reality, there continue to be use cases for vector and rasterized tile formats, although it is likely that a number of organizations will see performance benefits from rebuilding some of their originally rasterized tile sets as vector tiles in the future. This is even more likely as popular commercial software packages such as ArcGIS introduce tools to work with the Mapbox vector tile specification, a strategic decision that Esri announced in a 2015 blog post. On the open source side of things, GeoServer added support for vector tiles in version 2.14, as detailed in these instructions.

Open source clients are also recognizing the staying power of vector tiles, exemplified by the VectorTile layer format built into OpenLayers 3 and plugin support for Mapbox vector tiles in Leaflet. At the time of this writing, QGIS does not natively support viewing vector tiles, although there is a relatively new plugin for this purpose.

Suppose you're satisfied with the layers and symbols in your WMS, but you want it to draw faster and be available to many simultaneous users. In this situation, it might make sense to use GeoWebCache to create your tiles, because GeoWebCache is built directly into GeoServer. In this walkthrough, you'll use GeoWeb Cache to create a tile cache for the Philadelphia NeighborhoodMap group layer WMS that you published in the previous lesson.

1. Start GeoServer and open the GeoServer Web Admin page.
2. Use the Layer Preview link to preview your geog585:NeighborhoodMap group layer (the one with the green icon). Use the OpenLayers preview so that you can zoom and pan around. Take note of the performance and appearance of the map. You'll see the labels repositioned on each pan, a sure indication that tiles are not being used yet.
3. In the GeoServer Web Admin page, click the Tile Layers link.
4. Click the geog585:NeighborhoodMap link, and then select the Tile Caching tab. This is where you can set up parameters for caching of your layer. If you stopped here, your cache would be created on demand. In our case, we actually want to pregenerate tiles for most of the levels so that they won't need to be built on demand. GeoWebCache calls this "seeding" the cache.

5. Click Tile Layers again and in the row for geog585:NeighborhoodMap, click Seed/Truncate.

   

   Figure 5.4 Tile Layers in GeoServer

   Credit: © Penn State is licensed under CC BY-NC-SA 4.0

6. Scroll down, and fill out the form to create a new task as shown in the image below. Notice that your tile creation task will create PNG images using the Google Maps tiling scheme (900913).

   

   Figure 5.5 Tile Layers GUI in GeoServer

   Credit: © Penn State is licensed under CC BY-NC-SA 4.0
7. At the bottom of the form, click Submit. Although you can't see it, your computer is busy drawing your map over and over at different scales. This can tax the computer's memory and CPU resources, and you may see a performance impact on other applications while the caching is occurring. If you want to see your processor at work, open Windows Task Manager and look at the CPU and memory resources being used by java.exe.

8. Every 30 seconds or so, click the Refresh List link to see the progress of your cache. When you click this link and the task status disappears, it means your cache is complete.

   

   Figure 5.6 Tile Layers progress window in GeoServer

   Credit: © Penn State is licensed under CC BY-NC-SA 4.0

9. In the GeoServer Web Admin page, click Tile Layers and use the dropdown list to preview your cache in EPSG:900913 using the PNG format (see image below). This time when you pan around, you should see the map appearing instantly. You can verify that the tile cache is being used if the labels do not change position as you pan.

   

   Figure 5.7 Tile Layers GUI for previewing the results in GeoServer

   Credit: © Penn State is licensed under CC BY-NC-SA 4.0

   
   Caution: Make sure you are using the Tile Layers preview and not the Layer Preview preview. The Tile Layers preview uses a slightly different URL for the layer that indicates the tile cache should be used. Also, do not worry if the cache is reported on the Tile Layers page as N/A or 0.0 B in size. This seems to be normal, even though you have now built the cache. 

Although performance is improved with the tile cache, you may notice some duplicate labels appearing. This is a difficult problem to avoid with map tiling, because each tile does not "know" about the labels on the adjacent tiles. To mitigate this problem, tile caching software typically draws an area much larger than a tile and then cuts it up into individual tiles. GeoWebCache calls this large area a "metatile" (Esri calls it a "supertile"). If you like, you can experiment with adjusting the metatile size; although duplicate labels can still appear at the metatile boundaries, the duplicates will be fewer and farther between. You may also find that the settings and options in the next walkthrough with QGIS make it easier to get the labeling you want.

In this walkthrough, you will use QGIS to create a general-use basemap of Philadelphia. The idea is that you will be able to overlay thematic layers on top of this map in lessons to come. The data for this walkthrough is the base data for Philadelphia that you preprocessed in Lesson 3. If you followed all instructions, you should have this data in a folder called c:\data\Philadelphia or something similar. It should be using the mercator projection EPSG:3857.

Designing the map using QGIS

1. Open QGIS, start a new project, and add the city limits shapefile. The projection of the project shown at the bottom right should automatically change to EPSG:3857.
2. Open the Layer Styling panel, and change the symbol layer type from Simple Fill, to Outline: Simple Line.
3. Change the Stroke width to 3 Points.
4. Click on the current apllied color, and change the HTML notation (the hex code of a color) to #88789e.
5. Hit Apply and close the Layer Styling Panel.
6. Add the waterways shapefile to the project.
7. Following the same process as above, change the color of the default SImple Line symbology to #89aceb.
8. Change the width to 1 Point.
9. Hit Apply and close the Layer Styling Panel.
10. Add the natural features shapefile to the project and change the order of layers such that waterways is above natural features.
11. Open the Layer Styling panel for natural features, and change the type from Single Symbol to Categorized.
12. Use the drop down box below to assign the category value as type.
13. Click the Classify button below to see all the available values of type.

14. Change the colors for parks and forests to #ae8, and for reverbanks and water to #89aceb. After hitting apply, your map should look similar to this:

    

    Figure 5.8 Progress so far.

    Credit: © Penn State is licensed under CC BY-NC-SA 4.0
15. Add the Neighborhoods shapefile to the project.
16. Open the Layer Styling panel.
17. Click on Simple Fill, and change both FIll color and Stroke color to tansparent. You can also turn the opacity all the way down to 0% to achieve the same result. This is because all we want from this layer are labels with the names of the neighborhoods.
18. Hit Apply, and select the labels tab .
19. Change the No Labels option to Single Labels.
20. Verify that the Value is set to NAME.
21. On the first tab below called Text, change the font to Arial Black, size 12, color: #88789e.
22. Select the furthest to the right tab called Rendering .
23. Enable the Scale Dependent Visibility, and set the minimum scale (the one with the minus on the magnifying glass) to 1:100,000.

24. Hit Apply and close the Layer Styling panel. The result should look similar to the image below and the labels should disappear when you zoom out.

    

    Figure 5.9. Neighborhood labels.

    Credit: © Penn State is licensed under CC BY-NC-SA 4.0
25. Add the roads shapefile to the project and rename it to Major Roads. The idea here is that we want minor roads to only show up at higher zoom levels. We will achieve this by setting up this new layer, to only show major roads and then later in step 30, add the roads layer to the project a second time to show all roads but with scale-specific rendering rules.
26. Open the Layer Styling panel, and change the symbology to Categorized.
27. Change the value to Type, and click Classify.
28. Unselect all types but the following and set the color and width attributes as listed:
    • motorway. Color: #606060, width: 3 Points.
    • trunk. Color: #606060, width: 3 Points.
    • primary. Color: #838383, width 2 Points.
29. Click Apply and close the panel.
30. Add the roads shapefile again and move it to a position directly below the Major roads layer in the Layers list so that major roads will be rendered on top.
31. Open the Layer Styling panel.
32. Set the color to #b6b6b6 and the width to 1 Point.
33. Click on the labels tab, select Single Labels, and make sure that value is set to name.
34. Change the font to Arial, size 11.
35. Change the color to #b6b6b6.
36. Go to the rendering tab, and enable the scale visibility.
37. Set the minimum visibility to 1:25,000.
38. Click apply and close the panel.
39. The scale visibility in step 37 only applies to the labels, so we need to make the roads themselves only show up at a certain scale: Right click on the name of the layer, and select Set Layer Scale Visibility.

40. Set the minimum visibility to 1:100,000. After clicking Ok, try out how the features and labels shown in the map change while you zoom in and out. Notice how minor roads only show up at higher zoom levels, and road labels are only starting to appear at even higher levels. The two images below give you an idea of how things should look like at low and high zoom levels.
    
    One thing to note is that QGIS, in addition to applying the minimum scale condition for the labels, also by default applies a label layouting algorithm that will suppress labels of features that are very small at the current zoom level or would lead to overlap between labels. As a result, you will see even more road labels show up when you zoom in further. The technique of adding the same layer multiple times but with different settings is very helpful for implementing scale-specific rendering rules in the context of creating basemap tiles and in other application domains as well. We could have easily created a visual hierarchy with many more levels of features and labels showing up at different zoom levels but wanted to keep things relatively simple in this walkthrough. We also did not make much use of the more advanced approaches that QGIS provides for symbolizing vector layers, some of which you saw in the Lesson 1 walkthrough.

    

    Figure 5.10 Map so far zoomed out.

    Credit: © Penn State is licensed under CC BY-NC-SA 4.0
    

    Figure 5.11 Map so far zoomed in.

    Credit: © Penn State is licensed under CC BY-NC-SA 4.0
41. Lastly, add the Railways shapefile to the project.
42. Open the Layer Styling Panel.
43. Change the color to #d2bcb0, and set the width to 1 Point.
44. Before moving on, make sure the layers are arranged in the following order:
    • Neighborhood
    • City Limits
    • Major Roads
    • Roads
    • Railways
    • Waterways
    • Natural

It would also be a good idea to save the project now!

Exporting the tiles

Once you've finished the map design phase and your map looks good at each scale, you can start thinking about generating the tiles. For large maps, plan for this to take some time, taking into account some of the factors discussed earlier in the lesson, such as the shape of the map and the scale levels you choose to generate.

This section of the walkthrough explains how to generate tiles of your Philadelphia basemap, and host them on the web. You will then test them out in a web page.

1. Click the Toolbox icon on the top ribbon .
2. Search for xyz in the Processing Toolbox window that opens up.

3. Select the tool named Generate XYZ tiles (Directory).

   

   Figure 5.12 Generate tiles tool.

   Credit: © Penn State is licensed under CC BY-NC-SA 4.0

4. In the new window that opens, select the following options:

   • Extent: click the button next to the box and select Calculate from Layer and select the City Limits layer (This determines the area for which tiles will be created. Some of the other options here can also be very useful, e.g., in cases where you don't have a suitable layer for deriving the extent from.)
   • Minimum zoom: 0 (this is the lowest zoom level for which tiles will be created).
   • Maximum zoom: 17 (highest zoom level for which tiles will be created).
   • Tile format: PNG.
   • Scroll down and select your desired Output directory by picking the Save to Directory... option and selecting a new folder that you created for storing the tiles called PhillyBasemap.

   We are setting the minimum zoom level to zero here, so that tiles will be provided all the way up to the top of the pyramid in the tiles hierarchy. However, often when working with a relatively small project area like this (e.g., in your term projects), your plan is to limit how far out the user can zoom to avoid that the displayed area becomes tiny or (almost) invisible. In that case, it makes sense to set the minimum zoom option accordingly and use Javascript programming (see next few lessons) to restrict the possible zoom levels. That said, in terms of storage space, you won't save much by excluding the lower zoom levels because almost all storage will always be taken up by the highest levels.

5. Press Run to create the tile images. Once this is done, check out the content of the output folder.

Hosting and testing the tiles

Setting up S3 storage and uploading tiles

If you navigate around your folder of tiles, you'll notice that the images are extracted into a highly organized structure of level\column\row. This structure is understood by various mapping programs and APIs, so all you have to do at this point is put your tiles onto a web-facing server. If you already have acces to some web space, you could simply upload the folder with the tiles to that web server. What we will instead do in this course is use Amazon's cloud services, namely the S3 service for scalable storage, as a convenient place to host the tiles on the web. We will place the tiles there and then test them in a web map. However, this will require you to set up an Amazon AWS account first, including registering a credit card. Uploading and accessing the tiles over the web will then cause some costs that your credit card will be billed for at the end of each month but these will be very small, < $10 most likely, if you just use the tiles for this class and follow the instructions below. The steps below will include descriptions of how to monitor the costs and how to set up a billing warning if the accumulated monthly costs go above a certain level. Please note that the workflow below is very specific to S3. If we would instead use a normal web server to host the tiles, we would most likely use FTP or SCP to upload the tiles to some folder in our personal web space on that server.

1. Go to Amazon Web Services and sign up.
2. Create a Root user account using your PSU email. (If you already have created an AWS account for your PSU email, e.g. for another class, you can instead log into that account and continue with Step 3.)
   You can give your account any name you want. You will then have to verify your email address and define a Root user password. After that, you have to enter your contact information and confirm your identify by entering a verification code sent to you via SMS or an automatic call to the phone number you provide. Lastly, you have to select a support plan; please choose the free "Basic support plan". When done setting up your account, you can click the button that leads to the "AWS Management Console".

3. Now, in the AWS Management Console, go to the search bar at the top of the screen and search for Billing . Click on the link (the name of the service at the top) to get to the "AWS Billing Dashboard".

   

   Figure 5.13 Amazon services search bar

   Credit: © Penn State is licensed under CC BY-NC-SA 4.0

4. The Billing Dashboard will show you both a cost estimate for where you will end up at the end of the month and the current costs accumulated so far this month ("Current MTD balance"). Further down below, there is a link called "View your bill" which will lead you to a page where you can explore the costs accumulated so far by unfolding the items at the bottom under "Amazon Web Services EMEA SARL charge by service".

   

   Figure 5.14 Amazon services billing information

   Credit: © Penn State is licensed under CC BY-NC-SA 4.0
5. Now type in the term Budget in the search bar at the top and select the sevice called "AWS Budgets" by clicking the name. We now want to set up a budget so that you will get alerted by email if the estimated or accumulated costs for a month pass a certain threshold.

6. Click on the orange button "Create a budget" at the top. On the following page, select the options "Use a template (simplified)" and "Monthly cost budget", then fill out the details for the budget as shown in the image below but using your own email address to which the warning should be sent. The $5 is just a suggestion. Feel free to use a somewhat higher or lower number. You can also change the name of the budget but the default name is fine. Lastly, click the orange "Create budget" at the bottom.

   

   Figure 5.15 AWS creating the budget

   Credit: © Penn State is licensed under CC BY-NC-SA 4.0

   As the information text said, you will now receive messages if any one of the following conditions is true: 1) your actual spend reaches 85% of the budget you defined, 2) your actual spend reached 100%, or 3) your forecasted spend is expected to reach 100% of the budget. You can alway go back to the AWS Budgets page to select and modify you budget there, including changing the threshold value. However, here is one important final note regarding the Billing and Budgets dashboards: For some reason, AWS does not update the accumulated costs in real-time. It can actually take multiple hours until the numbers get updated. Nevertheless, this should still help you keep track of how the costs are slowly increasing while you are using the data in your S3 bucket.

7. In the AWS Management Console, go to the search bar at the top of the screen again and now search for S3.
8. Click at the first result that is indeed the S3 service.
9. Once in the S3 service, click the orange button to create a bucket (basically a storage container inside your S3 cloud storage).

10. Under "General configuration", give your bucket a unique name, we recommend starting with "geog585-" followed by some random string of character and/or numbers that you pick yourself. We here will use the name geog585-a1rt15.

    

    Figure 5.16 S3 bucket create

    Credit: © Penn State is licensed under CC BY-NC-SA 4.0
11. If you want you can select the region that is closest to you, or in case of a real "global" project the region closest to your clients.
12. Scroll down to where it says Block Public Access settings for this bucket.
    • Usually we want to protect our data, but in this case, the map tiles have to be fully accessible by everyone, so uncheck the first tick to allow access.

    • Tick the box where you acknowledge that the bucket will become public.

      

      Figure 5.17 Bucket public access settings

      Credit: © Penn State is licensed under CC BY-NC-SA 4.0
13. Leave the rest as it is, and scroll down to click Create Bucket.

14. Once back to the main page, you should see your new bucket in the bucket list. Click on the name to open it up.

    

    Figure 5.18 S3 Bucket List

    Credit: © Penn State is licensed under CC BY-NC-SA 4.0

15. You should now see the page for your bucket, currently still not containing any objects. Click the tab called Permissions at the top.

    

    Figure 5.19 Bucket Page

    Credit: © Penn State is licensed under CC BY-NC-SA 4.0
16. Scroll down to Bucket Policy and click Edit. Amazon really does not want us to create a public bucket unintentionally, so we need to do one more thing.

17. Copy and paste the following in the available space:

    {
    "Version": "2012-10-17",
    "Statement": [
    {
    "Sid": "PublicReadGetObject",
    "Effect": "Allow",
    "Principal": "*",
    "Action": "s3:GetObject",
    "Resource": "arn:aws:s3:::geog585-a1rt15/*"
    }
    ]
    }
    

    Figure 5.20 Bucket policy

    Credit: © Penn State is licensed under CC BY-NC-SA 4.0
18. Above the text field and its title "Policy", you can find the ARN name of your bucket. Click the small icon left to it, to copy this name. Then go to the last line of the code you just pasted to change the name of the bucket there. Paste the ARN name you copied to replace the string arn:aws:s3:::geog585-a1rt15 there. Be careful to keep the initial double quote and the /*" after the bucket name.
19. Hit the orange button Save Changes.

20. Now we are almost read to upload our map tiles. If you only want to upload a couple of files (for instance, to upload and share the web map you will create in your Lesson 7 homework assignment), you can do so directly on the AWS page for your bucket by selecting the "Objects" tab at the top (you can think of "objects" as meaning "files" here) and then clicking the Upload button. This will show an area to which you can the drag and drop folder and files from the Windows File Explorer. The AWS will then scan the files and afterwards show you another Upload button that you can press to start the actual upload. However, this approach of using the web page won't work well for basemap tile sets that tend to consist of a huge amount of small files. The Philadelphia tile set, for instance, only is 200MB large in total but it consists of roughly 21,000 files. In our tests, uplading this amount of files via the web interface took around 6 hours.

    Therefore, we are going to do the upload via another piece of software, the AWS Command Line Interface (CLI), which is much more efficient for small files and should be able to upload the Philadelphia tile set in just a few minutes. To set this up, we need to do two things: (1) Create a so-called Access Key that will allow us to access our S3 bucket via CLI and (2) install the CLI software itself. Let's start with creating a Root access key...

21. On the AWS page, click on your account name at the very top right of the page and then select "Security credentials". Scroll to the area called "Access keys". Then click the button called "Create access key".

    

    Figure 5.21 Creating access key

    Credit: © Penn State is licensed under CC BY-NC-SA 4.0

    
    AWS will warn you that creating a Root access key is not the best practice. However, we want to keep things simple here, and you can always delete the access key immediately after you have successfully uploaded the files if you want. So please go ahead and enable the checkbox saying that you understand that this is not best practice, then click the orange button saying "Create access key".

22. AWS will now show you a window called "Retrieve access key" with your new access key in plain text and the corresponding secret key hidden by default. You can think of the access key as a login name and the secret key as the password. You will need both for using CLI, so please copy both of them. Important: This will be the only time you will be able to see/copy the secret key. If you loose it, you will have to go back and create a new access key. Once you have the keys stored somewhere else, click the "Done" button at the bottom. To delete or create further access keys, you can always go back to the Access Key area under Security Credentials as in Step 21.

    

    Figure 5.22 Access key and secret key

    Credit: © Penn State is licensed under CC BY-NC-SA 4.0
23. Now, it is time to install the AWS CLI tools: Click on this link https://awscli.amazonaws.com/AWSCLIV2.msi to download the software.
24. Double-click the downloaded file called AWSCLIV2.msi to start the installation. Then follow the instructions given by the installer.
25. Now open the Windows Command Shell cmd by going to the Windows start menu and typing cmd . This should open the black shell window (see next screenshot below in Step 27).
26. Let's first check if CLI has been installed correctly. Type in the following command and then press ENTER: aws --version
    The output should look as shown in the next screenshot below starting with "aws-cli" followed by some version number.

27. Next, we have to give CLI some information including the access key and secret key. Type in the following command and then press ENTER: aws configure
    You will then be asked for access key, secret key, default region, and default format. For the access key and secret key, copy and paste the keys that you copied from the AWS page in Step 22. For the other two items, you can simply press ENTER without typing in antyhing.

    

    Figure 5.23 Configuring CLI

    Credit: © Penn State is licensed under CC BY-NC-SA 4.0
28. Now we are ready to upload the files to our bucket. Use the cd command to move to the folder that contains the PhillyBasemap folder to which you exported the tiles in Steps 4+5 from the Section "Exporting the tiles" (see next screenshot from Step 29).

29. Now type in the following command but with the name in bold replaced by the bucket name you used and then press ENTER:

    aws s3 cp PhillyBasemap s3://geog585-a1rt15/PhillyBasemap --recursive

    

    Figure 5.24 Starting the tile upload

    Credit: © Penn State is licensed under CC BY-NC-SA 4.0

30. You should now see how the tiles are uploading. If instead you get some odd error messages. Try with running the command a second time; somehow this seems to happen sometimes on the first attempt.

    

    Figure 5.25 Tiles uploading

    Credit: © Penn State is licensed under CC BY-NC-SA 4.0

31. Once everything is uploaded, you should see a prompt in the cmd shell window. Now switch back to the AWS web page and refresh the "Objects" tab on the page for your bucket. Navigate to the PhillyBasemp folder and then further into the subfolders to one random png tile.

    

    Figure 5.26 Object overview

    Credit: © Penn State is licensed under CC BY-NC-SA 4.0
32. On the screen that opens, find the Object URL and copy it. The URL is made up of the base URL (https://geog585-a1rt15.s3.us-east-2.amazonaws.com/PhillyBasemap) followed by the folders and file name corresponding to the zoom level and x, y indices of this particular tile image. If you want, you can open a new tab in your browser and copy the URL into the address bar to open this one tile. You should see the tile image but keep in mind that you have picked an empty tile so that you will just get an image that is fully white. To later use your basemap tiles in a web map you, you will have to take the base URL and then replace the concrete numbers at the end with a string of variabels to create a template string, for instance looking like this: https://geog585-a1rt15.s3.us-east-2.amazonaws.com/PhillyBasemap/{z}/{x}/{y}.png . We will make use of this option in the final part of this walkthrough in which we will test out our tiles in an ArcGIS Online map.
33. GOOD JOB! You are done!!

Testing the tile set in an ArcGIS Online map

1. Open a browser to ArcGIS and click the Map link at the top of the screen. You'll see an Esri-provided map, which we will switch out for our Philadelphia basemap. You can log in with your PSU credentials if you want, but it is not required.
2. Click Add > Add Layer from URL.

3. Populate the dialog box as shown below but using the base URL for the tiles in your own bucket from Step 32 in the previous section. Make sure to keep the /{level}/{col}/{row}.png part though. Note that ArcGIS Online uses different variable names (level, col, row) compared to the example given in Step 28 and what you will see in Leaflet in the next lesson. Also note that to set the extent, you must click Set Tile Coverage and draw a box around Philadelphia. You don't need to get the exact same coordinates shown below. The window might look different due to a recent update, but the logic is exactly the same. 

   

   Figure 5.27 Add Layer from Web GUI in GeoServer

   Credit: © Penn State is licensed under CC BY-NC-SA 4.0

4. Click Add Layer. If nothing happens for a long time, repeat and make sure there is no error in the URL field. Navigate around your Philadelphia map to test it out. If you turn on the network utility of your browser's developer tools, you should be able to see the tiles being brought into the map.

   

   Figure 5.28 Sample map output

   Credit: © Penn State is licensed under CC BY-NC-SA 4.0

Optionally, you could add other web layers on top, but, for the sake of time, this will not be described here. In future lessons, you'll learn how to build this type of web map programmatically without using ArcGIS.com.

Now that you've gone through a walkthrough and built your own tiled map, hopefully your appreciation has increased for the amount of cartographic design and effort required for producing a web basemap. In this week's assignment, you'll look at some existing tiled maps, then get some practice building your own.

Do the following:

• Find four online maps that use tiled maps in the background. These can be either rasterized image tiles or vector tiles. For each map, comment on where the tiles are coming from and how they were produced, if you can figure it out. It's okay if some of your tile providers are repeated (for example, two of the four might use Google Maps), but try to find at least one web map that uses a "homegrown" tiled basemap in the background. Simple local map viewers produced by city and county GIS websites are good places for this.
• Create a tiled map of rasterized images using some or all of your term project data. Depending on the nature of your data, this can be a basemap or a map with several thematic layers that is designed to overlay a basemap.
  
  You should design this map using QGIS and upload it to your AWS S3 bucket, following the same procedures we used in the 2nd walkthrough. The map should be designed using sound cartographic principles, including aesthetically pleasing colors, an intuitive visual hierarchy, and a useful labeling scheme. Take advantage of scale-dependent rendering settings on your layers and the option of working with multiple copies of the same data set, so that the user is not overwhelmed with information at any particular scale. Zooming into the map should reveal progressively more information.
  
  Important: Be careful with how many levels of tiles you build for this assignment, in particular if your project area is very large. Our recommendation is to keep the total size of the tile set that you are going to upload to S3 below 250MB, definitely below 500MB, and below 30,000 tiles in total (you can remove the subfolders for higher zoom levels after you produced the tiles to reduce the total size). As you saw in the walkthrough, if the project area is very small like a large city, a maximum zoom level of 17 should be ok. If your project area is an entire state in the U.S., try out what total sizes you get if you for maximum zoom levels between 12 and 14. If your project area is the entire U.S. (or a similar area), try out maximum zoom levels between 10 and 12. Also, if your tiled map consists of thematic layers, make sure the tile background is transparent, so that the tiles can be overlaid on top of other layers.

Please produce a report containing all the information requested in the first bullet above, as well as a URL to your tileset in your AWS S3 bucket so that I can test in a web map as shown in the walkthrough, together with a few words on how you designed your tiles so that I know what to look for. Then submit the report to the Lesson 5 drop box on Canvas.