A time cartogram visualizes travelling-times between locations. It replaces the geographic distance by time distance and distorts the underlying map accordingly. Two types of time cartograms exist: centered and non-centered ones. A centered time cartogram visualizes travelling-times from a fixed starting location to other destinations in a region [1–4], while a non-centered time cartogramvisualizes travelling-times between all pairs of locations [5–7].
The use of cartograms is growing due to their captivating design  and also due to the availability of methods to create them automatically [e.g. 7, 9–12]. The simple reason is because they show what researchers need to see and researchers outside of cartography do not find these hard to understand. The United Kingdom’s Chief Medical Officer used cartograms throughout her annual report to show disease patterns across the UK  – whereas a normal map would be much less useful for such projections. Dent  described cartograms as “innovative,” “interesting,” and “stylish”. According to Dorling [15, p. 53], “Whatever you choose to use cartograms for, from studying participation in elections, to the spread of a disease, or the social structure of a country, the very different perspectives they show are likely to alter the way you imagine the processes behind these patterns to be operating.” Kocmoud and House [16, p. 236] stated that “The cartogram is a useful tool for visualizing the geographical distribution of ‘routine’ data in a variety of disciplines, including politics, social demographics, epidemiology, and business.” Shimizu and Inoue  reported that cartograms are a highly effective method for visually representing statistical data. Similarly, Wu and Hung  found cartograms very useful for creating strong visual impacts.
Despite the many positive claims, some researchers are doubtful of the effectiveness of cartograms because of their distortion in shape [e.g. 19–22]. In the literature, we find only a few studies on the usability of area cartograms [e.g. 23–25], and only limited research has been reported on the usability of time cartograms [26, 27]. In recent years, various methods have been developed for constructing time cartograms [e.g. 1–3, 5, 17, 18, 28–36], however, the practical value of such cartograms has not yet been established. This motivated us to conduct a detailed and systematic usability evaluation of centered time cartograms (hereafter abbreviated as CTC) to investigate whether they are indeed useful.
Several CTCs were created to answer spatio-temporal questions related to the Dutch railway network. To create these cartograms, we used the method proposed by Ullah and Kraak . Two user studies were conducted: a laboratory test and an online survey. In the laboratory test, we used eye-tracking, thinking aloud, and videorecording to compare four alternative designs of CTCs (i.e. with railroads, without railroads, with emphasized railroads, and with emphasized time-circles – see Figure 1) to test whether railroads add to the readability of CTCs (or provide a false impression of time) and to examine the impact of visual hierarchy on the readability of CTCs. These designs are abbreviated WR, NR, ER, and EC, respectively. Four groups of participants were asked to perform spatiotemporal tasks using these designs and report which one of these provided an effective and efficient visual representation and best satisfied their requirements. Forty responseswere collected and analyzed. In the online survey, CTCs were evaluated against geographic and schematic maps by asking spatio-temporal questions to determine whether they can answer some questions better than the other two solutions. Eighty-eight persons participated in the survey.
The remainder of this paper is organized as follows: Section 2 discusses the design, implementation, results, and conclusions of the laboratory test. Section 3 reports on the design, implementation, results, and conclusions of the online survey. Section 4 draws final conclusions and looks at possibilities for future work.
2 The laboratory test
2.1 Objective, design, and implementation
In this research, a laboratory test was conducted applying eye-tracking, thinking aloud, and video-recording to compare four different CTC designs (see Figure 1). The test objectives comprised:
To find out which alternative CTC design (or combination) performs better when answering spatiotemporal tasks
To discover which usability issues are involved when test participants are working with alternative CTC designs
To investigate whether railroads add to the readability of CTCs (or provide a false impression of time)
To determine whether the distortion in CTCs indeed affects their readability
For the laboratory evaluation, CTCs were constructed for the railway network in the Dutch province of Overijssel. This network has 33 train stations. To prevent test results being biased by a particular CTC shape and to investigate whether a distortion in CTC shape affects readability, six stations (i.e. Steenwijk, Zwolle, Gramsbergen, Enschede, Raalte, and Deventer) were selected as starting stations (see Figure 2). The four different CTC designs were presented to test participants (TPs) in four groups. The first group worked with CTCs with railroads, the second with CTCs without railroads, the third with CTCs with emphasized railroads, and the fourth with CTCs with emphasized time-circles. Each group performed eight different spatio-temporal tasks (listed in Table 1). The taskswere designed to be real-world to allow testing of the CTCs from a real-world approach and provide an avenue to understand the practical value of these cartograms. The first six tasks were different (but with similar complexity) for each starting station, while the seventh and eighth tasks were the same for all TPs. To avoid potential bias, the tasks and corresponding maps were presented in random order. Each test was then followed by a questionnaire where the TPs ranked each design on a Likert scale from ‘strongly agree’ to ‘strongly disagree’, considering three usability aspects: (a) pleasant to see, (b) easy to understand, and (c) use without confusion. The questionnaire aimed to capture the feelings of the TPs about each design which could then be interpreted in combination with their suggestions during the process of thinking aloud.
Potential TPs were invited to participate through an email. Theywere asked to complete an online survey about their profile and whether they were willing to participate in the actual user test. Profile information was used to allocate participants to particular test groups (see Figure 3 below). Afterwards, a separate invitation was sent to all willing participants to participate in the actual user test. Test completion time for each TP was on average 40 minutes (10 minutes for introduction + 5 minutes for warm-up tasks + 25 minutes for the actual test).
In total, 48 persons indicated their willingness to participate in the user test by completing the profile survey, however, among those 48 respondents 8 failed to arrive for testing. Among those final 40 test participants, 11 listed Bachelor, 23 Master and 6 PhD as their highest educational degree. The participants, from 25 different nationalities, were postgraduate students and staff members of the Faculty of Geo-Information Science and Earth Observation at the University of Twente in the Netherlands and were thus knowledgeable about maps. Participants had varied academic backgrounds, with the majority from GIS and Remote Sensing fields and most were occasional users of Dutch trains. Approximately half of the participants knew about cartograms in general, but had limited or no knowledge about CTCs.
Participants play a key role in usability studies [37, 38]. Accordingly, one of the pre-test preparations for the laboratory test was the collection of the participants’ profiles. We used thirteen criteria to assign participants to particular test groups with the intention to make the groups as similar as possible in their composition. The criteria included – given in order of decreasing priority – experience of travelling using Dutch trains, knowledge about the Dutch railway network and the geography of the Netherlands, duration of stay in the Netherlands, map-use and map-making experience, knowledge about cartograms, usual method of train trip planning, department or course, educational level and background, country of origin, age, and gender. Highest priority was given to the user characteristics that can influence the test results the most. Using these criteria TPs were divided into four groups (as illustrated in Figure 3) using a technique called “matrice ordonnable” .
A Tobii X60 eye-tracker (with peripherals) and Tobii Studio software 2.2 was used to prepare and present the four alternative design solutions of the CTC from six different starting stations in random order. Tobii Studio is a platform that is able to achieve integrative recording of eye-tracking, video, sound, and screen logging and these outcomes may be analyzed in various (statistical and visual) ways. The user tests were conducted in the ITC usability laboratory which is a well-equipped laboratory designed for user research (Figure 4). The testing environment was programmed for uninterrupted sound (for the thinking aloud process) and video-recording of the whole testing procedure.
Three pilot tests were conducted before the actual user tests to assess the test scenarios for any potential problems. Three experts in usability testing attended the pilot tests to evaluate the presentation of the CTCs and the overall test set up. Their comments were used to detect and remedy any potential test set up problems before the final test execution.
2.2 Results and discussion
From the huge test records of the Tobii software that included visualizations such as gaze plots, heat maps, and statistical graphics for area of interests (AOIs), the test results of the research were organized and prepared for analysis. Result gathering and preparation were complemented by the verbal and action protocol data of the thinking aloud process.
The processing of test results was primarily directed towards the analysis of usability aspects such as correctness, response time, and satisfaction. During task execution in the experiment, TPs were asked to either choose an answer from the given choices or to write their response if they felt their answer was not listed. These data were augmented with the help of the video-recording, reasoning, and eye-movements. The results for each task (see Table 1) are discussed in detail below.
The results for temporal task T1 are presented in Figure 5. In this task the TPs had to state the departure time from a source station to reach a destination at 11:00 am in the shortest time possible. To perform this task, TPs did not require the railroads and only needed the time-circles to calculate travelling-time from source to destination. Accordingly, for this task NR was expected to perform better than the other cartograms. The results were consistent with our hypothesis: NR showed higher correctness. In contrast, the correctness of ER was lower while WR and EC performed equally well. However, the TPs who worked with NR took more time to complete the task. The thinking aloud data showed that the reason for a longer response time was that the TPs spent a greater deal of time locating the route fromsource to destination. On the other hand the TPs who worked with WR, ER, and EC took less time on average to perform the task as the railroads helped TPs to quickly locate the route and time-circles to calculate the travelling-time from source to destination. A majority of the TPs liked all four alternative designs for this task.
The results for temporal task T2 are displayed in Figure 6. In this task the TPs had to list all stations that are reachable within 20 minutes by train from a source station. Again, the railroads were not needed for this task as TPs were only required to list all stations located inside the 20-minute circle. For this task, NR was again hypothesized to perform better than the rest. Accordingly, the correctness of NR was higher. The three other cartograms performed equally well as two TPs only answered the question partially. In those instances, the TPs either did not list all stations reachable within 20 minutes or listed additional incorrect stations (i.e. requiring more than 20 minutes of travelling-time). The average response time of NR was longer, followed by ER and then WR and EC in order. However, the average response times for all four cartograms were quite comparable. Most TPs ranked all four designs as ‘good’ and ‘very good’.
Figure 7 depicts the results for temporal task T3. In this task the TPs were required to find the station that was closer in time to the source station among two given stations. To perform this task, TPs again only needed the time-circles to calculate travelling-times to the two stations. For this task, we expected NR to perform better than the other cartograms. In CTCs, travelling-times are indicated by concentric time-circles while the railroads only show connectivity between stations. This task was specifically developed to determine whether the railroads provided a false impression of travelling-times because tracks do not always follow a direct time from circle to circle. For this purpose a pair of stations was carefully selected for each CTC where the railroads could be misleading. For example, we chose Oldenzaal and Almelo for the CTC from Enschede. In this instance if the railroad is followed, Almelo seems closer to Enschede than Oldenzaal. But this is not correct. Oldenzaal is closer to the 20-minute circle than Almelo. As evident from the correctness graph however, the railroads did not cause confusion. All participants performed the task easily, particularly using WR, NR, and ER. On average, the response times of all four groups were similar. The majority of TPs found all four designs satisfactory for this task.
The results for spatio-temporal task T4 are summarized in Figure 8. In this task the TPs had to determine the path which takes the longest distance to reach the destination from the source. To perform this task, TPs required both the railroads (to find the routes from source to destination) and the time-circles (to calculate the longest route). For this task, ER and EC were expected to perform better than WR and NR. CTCs visualize travelling-times from the starting station to all other stations by replacing the geographic distance with time distance. Distance in CTCs always means time (time distance). Here, we purposely asked TPs to determine the path which takes the longest distance to investigate their understanding of the CTC and to assess whether they found the term ‘distance’ confusing. Since most TPs were frequent users of maps in general but were not knowledgeable about CTCs, they found the task confusing. In each group, many TPs either stated that the task cannot be completed with the provided CTC or performed the task wrongly by misinterpreting the distance for physical distance. Only TPs who were already knowledgeable about CTCs understood the task well and provided clear answers. Due to the confusion all four cartograms performed poorly for this task, however the correctness of ER and EC was slightly better compared to the two other CTCs. The TPs who worked with NR took more time to perform the task, as they spent a considerable time in identifying the paths from source to destination. The TPs in the other three groups took less time to complete the task. The satisfaction results for this task were aligned with the results of correctness and accordingly most TPs ranked the cartograms between ‘very poor’ and ‘average’.
Figure 9 shows the results for spatial task T5. In this task the TPs had to identify the number of all possible paths (without back-tracking) from a source to a destination. To perform this task, TPs only needed the railroads to see the connectivity between stations in order to locate the paths from source to destination. For this task, WR, ER, and EC were hypothesized to perform better. The majority of TPs understood the task and provided answers without confusion and the results were aligned with our hypothesis: WR, ER, and EC performed better. NR performed the worst because it was not possible to perform this spatial task without railroads. However, due to their familiarity with the Overijssel railway network, two persons were still able to perform the task correctly using NR. The average response times of WR, ER, and EC were comparable. However, one TP in WR spent a considerable length of time to complete the task successfully. As apparent from the satisfaction graph, the TPs were quite satisfied with WR and less satisfied with NR.
The results for spatio-temporal task T6 are illustrated in Figure 10. In this task the TPs were required to list all intermediate stations while travelling (via the shortest path in time) from a source station to a destination. In this task, TPs needed both the railroads and time-circles to find the shortest path from source to destination and list all stations along it. Unlike in T4, we specifically asked TPs to list the intermediate stations along the shortest path in time to see how they reacted to the term ‘time’. In contrast to T4, TPs did not face any difficulty and understood the task very well. For this task, we expected ER and EC to perform better and the results were in line with our hypothesis as the correctness of ER and ECwas higher and the correctness of the other two cartograms (WR and NR) lower. Again, this task could not be performed by TPs using NR. However, some TPs again performed the task correctly due to their familiarity with the Overijssel railway network, while others used their instinct. On average, the response times of all four groups were comparable with the exception of two TPs (one in ER and one in EC) who took an atypical length of time to complete the task successfully. In line with the correctness results, the majority of TPs ranked ER and EC either ‘good’ or ‘very good’.
The results for spatio-temporal task T7 are given in Figure 11. In this task the TPs were asked to identify a station (or stations) which was evenly close time-wise to both Enschede and Zwolle. This was considered the most difficult task because it required two different cartograms (fromEnschede and Zwolle) to complete the task successfully. To perform this task, TPs required both railroads and time-circles: the railroads to locate the paths between Enschede and Zwolle and the time-circles to find out the stations which are in the middle time-wise. Four stations – Wierden, Rijssen, Holten, and Daarlerveen – are evenly close time-wise to both Enschede and Zwolle. For this task, WR, ER, and EC were expected to perform better than NR and this hypothesis was confirmed by the testing. A majority of the TPs who worked with these cartograms performed the task correctly by listing all four stations. Here, the railroads helped to quickly identify the routes between Enschede and Zwolle and the time-circles assisted to find the required stations. This is also the reason that those TPs took less time to perform the task. On the other hand, NR performed reasonably well but while most TPs performed the task correctly they but did not list all four correct stations. Due to the missing railroads, they were not able to identify all possible routes between Enschede and Zwolle and thus answered the task only partially. They also took a marginally longer than the other cartograms because of the time spent in locating the routes. However, the TPs seem to have found this visualization satisfying, which is not really strange because of a problem with thewording of this task.We did not state explicitly (but obviously should have done) that TPs had to list all possible stations which are evenly close time-wise to both Enschede and Zwolle. This may have explained why many of the TPs in NR answered the task only partially correctly but ranked the design the highest.
In the spatio-temporal task T8, the TPs in each group were provided with all four alternative designs of the CTC. They were asked to perform the given task using all four designs and choose the one that best satisfied their requirements. The task required TPs to list all intermediate stations while travelling (via the shortest path in time) from Raalte to Holten. To avoid potential bias, the presentation order of the cartograms was different in each group (as illustrated in Figures 12(a)–12(d)). Note that here, the heat maps show eye fixation in each group during the task execution and not the preference for a particular design . At the end of the task, TPs were asked to indicate their CTC preference (Figure 12(e)). The majority of TPs (27 out of 40) opted for ER, followed by EC (7 out of 40) and WR (6 out of 40). Not a single TP selected NR.
At the end of test, the TPs in each group were provided with a questionnaire where they rated the design according to three different categories: (a) pleasant to see, (b) easy to understand, and (c) use without confusion. The results are summarized in Figure 13. Overall, the satisfaction of WR, ER, and EC was more on the positive side for the three usability aspects. In the case of NR, satisfaction was more on the negative side for the three usability aspects, particularly ‘easy to understand’ and ‘use without confusion’.
Overall, the results of the laboratory test suggested that among various designs, ER is the most preferred design and NR is the least preferred. In general however, the effectiveness, efficiency, and satisfaction of the CTCs significantly depend on the nature and complexity of the task. Prior knowledge about CTCs also plays a role in this regard. Overall, no design was found to be equally suited to all tasks and there are apparent differences in the suitability of certain designs for answering specific tasks. We found that while WR, ER, and EC are better suited for spatial and spatio-temporal tasks, NR gives better performance for temporal tasks. We propose that optimal results could be achieved by presenting these cartograms in an interactive visualization environment where users can select a particular design and turn different layers on/off depending upon the nature and complexity of the task.
2.3 Further analysis based on thinking aloud, eye-tracking, and profile data
In this section, thinking aloud and eye-tracking results are discussed along with the responses data to identify the problem areas in the four alternative CTC designs. During the test execution, what TPs say or believe they do, is not always the same as what they actually do. We used video-recordings and thinking aloud data from the test to analyze the TPs’ behavior while performing the tasks. While thinking aloud can help assess TPs’ responses and feelings when they execute tasks, eye-tracking also provides additional unobtrusive evidence of TPs’ behaviour. We considered the TPs’ visual behavior as typical if the fixation points were placed on the area of interest and considered as atypical if otherwise. We defined the area of interest as the corridor where we anticipated the TPs to look at while working with the CTC during task execution.
Group 1 worked with CTCs with railroads (WR). TP7 and TP10 required more time for all tasks, particularly for T1 (350 seconds and 211 seconds respectively versus the average time of 121 seconds). TP7 was familiar with cartograms and the geography of the Netherlands. In addition, she had map-making and map-use experience. She completed T1 successfully but took very long. The gaze plot in Figure 14(a) compares the visual behaviors of TP7 versus TP3 (who was also familiar with cartograms and took only 68 seconds to perform the task correctly) for T1.Unlike TP7, the fixation points of TP3 are only placed on the area of interest. TP10 was not knowledgeable about cartograms, did not have any map-making experience and had very little map-use experience. She performed T1 successfully but consumed more time to understand the task and familiarize herself with the representation. This is evident from the gaze plot (see Figure 14(b)) and also supported by the thinking aloud data. Figure 14(c) compares the visual behaviors of TP1 and TP5 for T2. Both participants were knowledgeable about cartograms and showed similar visual behaviors to complete the task successfully in comparable times (81 and 121 seconds respectively). TP8, being a frequent user of the Dutch trains, performed T3 successfully in only 43 seconds. In that instance, the gaze fixation points were placed on the desired stations (as shown in Figure 14(d)). TP10 stated that T4 cannot be answered with the provided cartogram, which was incorrect. The scattered fixation points of TP10 show that the participant did not understand the task well and faced difficulty in reading the cartogram (see Figure 14(e)). A comparison of the gaze plots of TP1 and TP5 for T6 is presented in Figure 14(f). TP1 completed the task correctly, while TP5 answered the task wrongly. The fixation points of TP1 are placed on the area of interest whereas those of TP5 are scattered. This participant was seemingly confused while performing the task.
Group 2 worked with CTCs without railroads (NR). The participants in this group consumed more time overall for all tasks and particularly for spatial and spatio-temporal tasks. TP7 was a frequent traveler of the Dutch trains but did not know about cartograms. He completed T1 successfully but took very long (337 seconds versus the average time of 185 seconds) to understand the task and familiarize himself with the CTC as depicted in Figure 15(a)). The gaze plots in Figure 15(b) compare the visual behaviors for T2 of TP3 (who answered the task incorrectly and took 87 seconds versus the average time of 109 seconds) and TP5 (who completed the task successfully but required 208 seconds). TP3 was a frequent user of maps but was not familiar with cartograms whilst, TP5 was an occasional user of maps but was knowledgeable about cartograms. Both participants showed different visual behavior. In the case of TP3, the fixation points are not targeted on the area of interest. This suggests that the participant was confused and faced difficulty in interpreting the cartogram. The fixation points of TP5 are also scattered, meaning that the participant likely did not understand the task well and was confused. This is likely the reason why TP5 consumed more time to complete the task. TP6 and TP9 (both familiar with cartograms) showed similar visual behaviors to T3 and completed the task successfully (Figure 15(c)). Both participants took 52 seconds for the task. TP7 incorrectly stated that T4 cannot be answered with the provided cartogram and took 308 seconds to decide. The gaze plot in Figure 15(d) agrees with these results, showing that the participant was not able to identify the different paths. Figure 15(e) illustrates the gaze plot of TP7 for T5. The participant answered the task partially because he failed to define all possible paths. Additionally, he did not understand the task well and also faced difficulty in reading the cartogram. The gaze plots of TP6 and TP9 for T6 are shown in Figure 15(f). TP6 answered the task incorrectly and TP9 completed the task successfully. The gaze plot of TP6 shows that the participant did not understand the task well.
Group 3 worked with CTCs with emphasized railroads (ER). The gaze plot in Figure 16(a) shows the visual behaviors of TP2 and TP10 (both familiar with cartograms and with equal map-use experience) for T1. TP2 answered the task wrongly and took 104 seconds while TP10 performed the task correctly and required 79 seconds. The fixation points of TP2 are not placed on the area of interest whereas the fixation points of TP10 are perfectly placed on the area of interest. Figure 16(b) shows the gaze plot of TP5 for T2. The participant did not know about cartograms and completed the task successfully but consumed more time (115 seconds versus the average time of 84 seconds). The gaze plot indicates that this participant took a substantial length of time to understand the cartogram. Figure 16(c) illustrates the gaze plots of TP3 and TP6 for T3. Both participants had different profiles and showed different visual behaviors to perform the task. TP3 answered the task wrongly, requiring 41 seconds, whilst TP6 performed the task successfully in 25 seconds. Unlike TP3, TP6’s fixation points are placed on the area of interest. A comparison of the gaze plots of TP1 and TP4 for T4 is given in Figure 16(d). TP1 completed the task successfully and had gaze points concentrated on the area of interest. TP4 performed the task incorrectly and illustrated scattered gaze points, showing that the participant likely did not understand the task. Figure 16(e) and Figure 16(f) depict the gaze points of TP5 for T5 and T6 respectively and in both cases the fixation points were targeted on the area of interest and the tasks were completed correctly.
Group 4 worked with CTCs with emphasized timecircles (EC). The gaze plots of TP1 for T1 and T2 are shown in Figure 17(a) and Figure 17(b) respectively. The participant, being knowledgeable about cartograms, performed the tasks correctly and the gaze points are perfectly placed on the area of interest in both cases. Figure 17(c) depicts the gaze plot of TP9 for T3. Here the participant answered the task wrongly and the scattered gaze plot verifies that result. A comparison of the gaze plots of TP4 and TP7 for T4 is provided in Figure 17(d). TP4, having no knowledge of cartograms, incorrectly stated that the task could not be answered with the provided cartogram. Accordingly, the fixation points are not targeted on the area of interest and this illustrates the participant likely did not understand the task well. In contrast, TP7, who also had no knowledgeable of cartograms, completed the task successfully and illustrated gaze points which were placed on the area of interest. TP8 answered T5 partially as he did not define all possible correct paths (as can be seen in Figure 17(e)). The gaze plots in Figure 17(f) compare the visual behaviors of TP2 and TP10 for T6. Both participants performed the task correctly in 596 seconds and 63 seconds respectively. TP2 seemingly took long to understand the task and familiarize herself with the cartogram whereas with TP10, the fixation points were perfectly placed on the area of interest. This is the likely reason TP10 consumed less time.
Various TPs worked with the CTCs from Enschede, Zwolle, Deventer, and Steenwijk (less distorted) to perform different tasks. Several other TPs performed similar tasks using the CTCs from Raalte and Gramsbergen (both highly distorted and far from reality). Because of the severe distortion in the geography, we expected poor performance in the latter case. However, the responses, thinking aloud and eye-tracking data showed that the distortions of CTCs did not greatly affect the readability. For instance, TP3 in group 1 performed T1 using the CTC from Enschede. TP10 in group 3 worked with the CTC from Raalte to perform T1. Both participants were familiar with cartograms and showed similar visual behaviors to complete the task successfully in comparable times (68 seconds and 79 seconds respectively). See Figure 14(a) and Figure 16(a). TP1 in group 4 performed T1 and T2 correctly using the CTCs from Gramsbergen and Raalte respectively. The distorted nature of the cartograms did not influence the visual behavior of TP1 as the gaze points were perfectly placed on the area interest in both cases (compare Figure 17(a) versus Figure 17(b)).
2.4 Conclusions from the laboratory test
The following can be concluded from the laboratory based usability evaluation:
Overall, the results suggested that ER is the most preferred design and NR the least preferred. However, the effectiveness, efficiency, and satisfaction of the CTCs were highly dependent on the nature and complexity of the task: WR, ER, and EC being better suited to spatial and spatio-temporal tasks and NR better suited to temporal tasks.
The thinking aloud data illustrated a number of design issues. Some participants stated that the legendwas not clear enough and suggested to provide a clearer legend.Most participants commented on the labeling, color, and point size of the stations, and color and line width of the railroads. All these problems were corrected for the online survey based usability evaluation.
Overall the results (particularly from task T3) indicated that the inclusion of railroads adds to the readability of the CTCs and does not provide a false impression of time. However, in some cases and particularly for non-experienced users, the railroads could be misleading. Users therefore need to be educated about the role of railroads in CTCs.
Overall, the responses, thinking aloud and eye-tracking data inferred that the distortions of CTCs do not greatly affect the readability.
3 The online survey
3.1 Objective, design, and implementation
This experiment was aimed to explore the usability of a CTC in comparison with two other kinds of equivalent representations (i.e. a geographic and schematic map).The laboratory test showed that the CTC with emphasized railroads (ER) was the most preferred design. However, the participants suggested a number of improvements to the design particularly relating to the legend which was not clear enough. A modified legend was then provided to assist participants in knowing what each of the elements used in the cartograms represent. Many participants also commented on the labeling of the stations. For better visibility and to allow a stronger reflections and evaluations on the actual tasks, and unlike in the laboratory evaluation, we therefore only labeled the stations of interest. Color and point size of the stations and color and line width of the railroads were also adjusted as per participants’ suggestions. Compare Figure 1(c) versus Figure 18(a).We then investigated whether the improved CTC was preferable over the other two map types in performing spatio-temporal tasks.
In this part, CTCs were constructed for the whole network of the Dutch railway. Seventy-three train stations, well-spread over the country, were used to construct the cartograms. Four stations (i.e. Eindhoven, Maastricht, Utrecht, and Schiphol) were selected as starting stations (see Figure 18). The alternative geographic and schematic maps of the Dutch railway are shown in Figure 19. To make sure that the tests were fair, the maps were designed so that they contained exactly the same amount of information. In the geographic and schematic maps, time was encoded using labels along the network segments.
The test contained four spatio-temporal tasks (listed in Table 2) and consisted of two parts. In the first part, the participants were asked to perform the four tasks using one of the three representations (selected at random). To select a representation at random, three option buttons (placed in a row and without a label to prevent any bias) were provided. The first option button selected the geographic map, the second button selected the CTC, and the third button offered the schematic map. In the second part of the test, the participants performed the same four tasks using all three representations and ranked each on a Likert scale from ‘very good’ to ‘very poor’ based on how well each type assisted them in performing the tasks. The actual test was preceded by a questionnaire in which participants were asked to answer questions about their profile.
The test was implemented through an online survey (created using the Lime Survey tool – www.limesur vey.org). Lime Survey was chosen because it provides time statistics for responses. The survey was distributed to different networks by sending email invitations. In total, 140 responses were collected; 88 responses were complete, and the other 52 were incomplete. We considered the complete responses only. The responses were different for each representation; 55 participants worked with the CTC, 12 with the geographic map, and 21 participants performed the tasks using the schematic map. The difference in the number of respondents may be due to the order of the option buttons. With the CTC button being in the middle, many participants performed the tasks using the CTC.
Among the 88 participants who completed the survey, 1 listed intermediate vocational education, 1 high school, 28 Bachelor, 43 Master and 15 PhD as their highest educational degree. The participants had different academic backgrounds, with the majority from Geo-Information Science and Earth Observation. They came from 32 different nationalities, with 67 currently living in the Netherlands and the other 21 in the rest of the world. A majority of the respondents were frequent train travelers and also frequent users of maps. About half of the participants knew about cartograms in general, but had limited or no knowledge about CTCs.
3.2 Results and discussion
The participants were asked to perform four tasks related to spatio-temporal aspects (see Table 2). Firstly, they were requested to perform the tasks using one of the three different representations. We collected participants’ correctness and response times. Secondly, they performed the same four tasks using all three representations, and gave a rank to each representation on a Likert scale from ‘very good’ to ‘very poor’ considering how effective each representation was to perform the tasks. This part was aimed to measure participants’ satisfaction. Below we discuss the results in detail first for each task.
For each task, we calculated correctness, response time, and satisfaction for all three representations. We expected the CTC to perform better than the other two solutions for task T1 and task T2, and poorly for task T3 and task T4: as T1 and T2 are temporal tasks and T3 and T4 spatiotemporal tasks. T3 was likely to be difficult for the CTC, as it uses two different cartograms (i.e. one from Eindhoven and the other from Maastricht) to complete the task successfully.
The results for temporal task T1 are displayed in Figure 20. In this task, participants had to report the departure time from Eindhoven to reach Zwolle at 11:00 am in the shortest time possible. For this task, the average correctness of the CTC was higher than that of the schematic map but only marginally lower than that of the geographic map. Similarly the average response time of the CTC was shorter than both the geographic map and the schematic map. In the CTC, the participants spent less time performing the task because they seemingly quickly calculated the travelling-time from Eindhoven to Zwolle by counting the number of circles. Alternately, participants who worked with the geographic and schematic map took longer because they had to add the travelling-times along the network segments in order to ascertain the time distance from Eindhoven to Zwolle.Most participantswere satisfied with the CTC. They ranked the CTC as ‘good’ and ‘very good’ and the other two maps as ‘average’ and ‘good’ (Figure 24(a)).
Figure 21 displays the results for temporal task T2. In this task, participants had to list all stations that were reachable within 30 minutes by train from Utrecht. This task was easy to perform using the CTC but difficult to perform using the two other maps. Using the CTC, all one needed to do was to record all of the stations located inside the first (i.e. 30-minute) circle. Using the other two maps however, participants had to add travelling-times along the network segments to ensure they were less than 30 minutes. That extra arithmetic was obviously more time consuming. The results of T2 were in line with our hypothesis. The CTC was both effective and efficient for this task. Test participants also found the CTC more visually interesting and satisfying compared to the two other maps. The satisfaction score of the CTC was more on the positive side compared to the geographic and schematic map type where scores were more on the negative side (Figure 24(b)).
The results for spatio-temporal task T3 are given in Figure 22. In this task participants had to identify a station (or stations) which was time-wise evenly close to both Eindhoven and Maastricht. Here, the participants needed two different cartograms to perform the task successfully; from Eindhoven and from Maastricht. Both cartograms were provided to the participants. This task was considered to be difficult for the participants who worked with the CTC. We also expected the participants to require more time using the CTC in comparison with the other two maps. However the CTC performed equally well versus the other map types. The correctness of the CTC was higher and the average response time comparable compared to the two other map types. For this task, a majority of the participants ranked the CTC between ‘good’ and ‘very good’ and the other two representations between ‘average’ and ‘good’ (Figure 24(c)).
The results for spatio-temporal task T4 are illustrated in Figure 23. In this task participants had to list the intermediate stations while travelling from Schiphol to Enschede via the shortest path in time. This task required participants to first find the shortest path from Schiphol to Enschede and then list all stations along it. For this task, the correctness of the CTC was higher than the geographic and schematic map. The average response time of the CTC was also shorter than the other two maps. Using the geographic and schematic map, the participants had to add the travelling-times along all possible paths from the source to destination to know the shortest path. That was obviously time consuming and is evident from the results. For this task, the participants ranked all three maps between ‘average’ and ‘good’, with the ranking of CTC slightly more on the positive side (Figure 24(d)).
Moreover, we calculated two statistical measures – mean and mode – to rank the three representations. In order to compute the mean ranking, numbers from 1 to 5 were used to indicate the range from ‘very poor’ to ‘very good’. The mean ranks are listed in Table 3. The ranking order based on the mean rank value is: CTC (3.89); geographic map (3.46); schematic map (3.37). Mode is the value that occurs most often in a set of data. We counted the first and second frequently occurring rank for the three maps. The results are shown in Table 4. The ranking order based on the mode scores is CTC>geographic map>schematic map, which is in agreement with the ranking order based on the mean rank value.
3.3 Conclusions from the online survey
The online survey based usability evaluation illustrated the following:
Overall, CTCs performed better than the schematic and geographic map in answering spatio-temporal tasks, particularly when the task had a dominant time-related component.
CTCs are appealing, as most participants ranked them as ‘very good’ and ‘good’.
4 Final conclusions and future work
This study suggested centered time cartograms (CTCs) as an intriguing, but as yet untested, possibility for representing space-time phenomena and presented a detailed and systematic user evaluation of these cartograms. Two user tests were conducted: a laboratory test and an online survey. In the laboratory test, we used eye-tracking, thinking aloud, and video-recording to compare four different designs of centered time cartograms to find out which design (or combination) of these performs better in answering spatio-temporal questions. In the online survey, we evaluated centered time cartograms against a geographic and schematic map to determine whether they better facilitated the answering of certain questions compared to geographic and schematic maps. Based on our results, the following conclusions can be drawn.
Among the four alternative designs, the CTC with emphasized railroads was the most preferred design and the CTC without railroads the least preferred. However, the effectiveness, efficiency, and satisfaction of the CTCs were greatly dependent on the nature and complexity of the task. The CTCs with railroads, with emphasized railroads, and with emphasized time-circles were better suited for spatial and spatio-temporal tasks while the CTC without railroads was better suited for temporal tasks.
The inclusion of railroads to these CTCs generally increases their readability and does not provide a false impression of time. However in some cases, and particularly for non-experienced users, the railroads were misleading. Users therefore need to be educated about the role of railroads in CTCs.
Distortions in CTCs do not greatly affect their readability and visual behavior.
CTCs were effective and also more efficient than geographic and schematic maps in answering spatiotemporal tasks, particularly when the task had a dominant time-related component.
CTCs provide an alternative method to visualize travelling-times and are appealing due to their captivating design. We suggest this method as an intriguing possibility for representing space-time phenomena and believe that this visualization method deserves greater attention. We hope that our work will encourage the design and use of CTCs by cartographers and information visualization researchers.
In general, the laboratory test and the online survey illustrated that no single visualization was suitable for all tasks and that certain representations are better suited to answering specific tasks: one representation may be better to answer one task and another representation may be better to answer another task. We suggest that optimal results would be obtained if these representations could be embedded in an interactive visualization environment where users can select a particular representation and also switch on/off different layers depending upon the nature and complexity of the task. Additionally, for transport networks in which segments and interconnections are dense and numerous, there may be readability issues. To address this problem, the interactive environment should have zoom and pan functionalities to allow visualization of the network at different resolutions. Moreover, the environment could permit users to choose line width, line color, point size, and point color for increased visibility.
Bies S, van Kreveld M. Time-spacemaps from triangulations. In: Didimo W, Patrignani M, editors. Graph Drawing. Volume 7704, Lecture Notes in Computer Science: Springer Berlin Heidelberg; 2013. p 511–516. Google Scholar
Ullah R, Kraak M-J. An alternative method to constructing time cartograms for the visual representation of scheduled movement data. Journal of Maps 2014; 11(4):674–687. Google Scholar
van Campenhout M. Travel time maps [MSc thesis]: Technical University Eindhoven; 2010. Google Scholar
Angel S, Hyman GM. Transformations and Geographic Theory. Geographical Analysis 1972; 4(4):350–367. Google Scholar
Buchin K, van Goethem A, Hoffmann M, van Kreveld M, Speckmann B. Travel-time maps: linear cartograms with fixed vertex locations. In: Duckham M, Pebesma E, Stewart K, Frank A, editors. Geographic Information Science. Volume 8728, Lecture Notes in Computer Science: Springer International Publishing; 2014. p 18–33. Google Scholar
Dorling D. Cartogram. In: Monmonier M, Collier P, Cook K, Kimerling J, Morrison J, editors. History of Cartography: Cartography in the Twentieth Century. Volume6. Chicago: Chicago University Press; 2015. p 1960.Google Scholar
Inoue R. A new construction method for circle cartograms. Cartography and Geographic Information Science 2011; 38(2):146– 152. Google Scholar
Bhatt LM. Investigating the appropriateness of Gastner- Newman’s cartogram versus conventional maps in visual representation and modeling of health data [MSc thesis]: Southern Illinois University at Carbondale; 2006. Google Scholar
Gastner MT, Newman MEJ. Diffusion-based method for producing density-equalizing maps. Proceedings of the National Academy of Sciences of the United States of America 2004; 101(20):7499–7504. Google Scholar
Keim DA, North SC, Panse C. CartoDraw: a fast algorithm for generating contiguous cartograms. Visualization and Computer Graphics, IEEE Transactions on 2004; 10(1):95–110.Google Scholar
Sagar BSD. Cartograms viamathematical morphology. Information Visualization 2014; 13(1):42–58.Google Scholar
van Kreveld M, Speckmann B. On rectangular cartograms. Computational Geometry 2007; 37(3):175–187. Google Scholar
Davies S. Annual Report of the Chief Medical Officer, Volume One, 2011, On the State of the Public’s Health. London: Department of Health 2012:11. Google Scholar
Dent BD. Communication aspects of value-by-area cartograms. The American Cartographer 1975; 2(2):154–168. Google Scholar
Dorling D. Area cartograms: their use and creation (concepts and techniques in modern geography). Norfolk, UK: Environmental Publications 1996. Google Scholar
Kocmoud C, House D. A constraint-based approach to constructing continuous cartograms. In: Poiker T, Chrisman N, editors. Proceedings of the 8th International Symposiumon Spatial Data Handling. Vancouver, BC: Taylor and Francis; 1998. p 236–246. Google Scholar
Shimizu E, Inoue R. A new algorithm for distance cartogram construction. International Journal of Geographical Information Science 2009; 23(11):1453–1470. Google Scholar
Wu Y-H, Hung M-C. Non-connective linear cartograms for mapping traflc conditions. Cartographic Perspectives 2012(65):33– 50. Google Scholar
FotheringhamAS, Brunsdon C, Charlton M. Quantitative geography: perspectives on spatial data analysis. SAGE Publications; 2000. Google Scholar
Tobler W. Thirty five years of computer cartograms. Annals of the Association of American Geographers 2004; 94(1):58–73.Google Scholar
Yau N. Alternative to cartograms using transparency. Flowing- Data2008.Google Scholar
Roth RE, Woodruff AW, Johnson ZF. Value-by-alpha maps: an alternative technique to the cartogram. The Cartographic Journal 2010; 47(2):130. Google Scholar
Sun H, Li Z. Effectiveness of cartogram for the representation of spatial data. The Cartographic Journal 2010; 47(1):12–21. Google Scholar
Kaspar S, Fabrikant S, Freckmann P. Empirical study of cartograms. 2011; Paris, France.Google Scholar
Reyes Nuñez JJ, Juhász B. Hungarian survey on the use of cartograms in school cartography. International Journal of Cartography 2015; 1(1):5–17. Google Scholar
Hong S, Kim Y-S, Yoon J-C, Aragon CR. Traflgram: distortion for clarification via isochronal cartography. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Toronto, Ontario, Canada: ACM; 2014. p 907–916. Google Scholar
Ullah R, Kraak M-J, van Elzakker C. Using cartograms to explore temporal data: do they work? GeoViz 2013. Hamburg2013. Google Scholar
Goedvolk A. De nieuwe relatieve afstand voor het openbaar vervoer. Nieuwe Geografenkrant 1988; 10:6–7. Google Scholar
Spiekermann K, Wegener M. The shrinking continent: new timespace maps of Europe. Environment and Planning B: Planning and Design 1994; 21(6):653–673.Google Scholar
Cauvin C. A systemic approach to transport accessibility. A methodology developed in Strasbourg: 1982–2002. Cybergeo: European Journal of Geography 2005.Google Scholar
Ahmed N, Miller HJ. Time-space transformations of geographic space for exploring, analyzing and visualizing transportation systems. Journal of Transport Geography 2007; 15(1):2–17.Google Scholar
Axhausen KW, Dolci C, Fröhlich P, Scherer M, Carosio A. Constructing time-scaled maps: Switzerland from 1950 to 2000. Transport Reviews 2008; 28(3):391–413.Google Scholar
Kaiser C, Walsh F, Farmer CQ, Pozdnoukhov A. User-centric timedistance representation of road networks. In: Fabrikant S, Reichenbacher T, Kreveld M, Schlieder C, editors. Geographic Information Science. Volume 6292, Lecture Notes in Computer Science: Springer Berlin Heidelberg; 2010. p 85–99.Google Scholar
Chen X. Seeing differently: cartography for subjective maps based on dynamic urban data [MSc thesis]: Massachusetts Institute of Technology; 2011.Google Scholar
Ramaer L. De vervaardiging van temporele kartogrammen. 100 Jaar veranderingen in de reistijd per trein in beeld. Geo-Info 2011;2011-10/11:11-13.Google Scholar
Denain J-C, Langlois P. Cartographie en anamorphose. Mappemonde 1998; 49(1):16–19. Google Scholar
Nielsen J. The use and misuse of focus groups. Software, IEEE 1997; 14(1):94–95. Google Scholar
Nielsen J. Estimating the number of subjects needed for a thinking aloud test. International Journal of Human-Computer Studies 1994; 41(3):385–397. Google Scholar
Bertin J. La graphique. Communications 1970; 15:169–185. Google Scholar
About the article
Published Online: 2016-06-08
Published in Print: 2016-06-01
Citation Information: Open Geosciences, Volume 8, Issue 1, Pages 337–359, ISSN (Online) 2391-5447, DOI: https://doi.org/10.1515/geo-2016-0035.
© 2016 R. Ullah et al., published by De Gruyter Open. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License. BY-NC-ND 3.0