In the early days of stylometry research, scholars were hampered by the lack of a standard or accepted methodology. This is evident in Hallberg’s studies, where ad hoc methods tend to be developed for each new attribution problem, for the most part without any particular theoretical justification. Another problem typical of early stylometry is the lack of a control corpus. A control corpus is a collection of texts by known authors which can be used to verify that a proposed stylometry method is actually effective at authorship attribution. In the works of Hallberg and West, only very limited steps are taken to verify that the methods used are empirically effective.

By mentioning these limitations, I am by no means claiming that Hallberg’s contributions to stylistic research are worthless. On the contrary, I think any reader of Hallberg will come to appreciate both his keen eye for stylistic details and his almost superhuman patience for collecting data. Nevertheless, there is clearly a lot of room for further work.

In the present study I apply a tried and tested stylometry method to the problem of Egils saga and Heimskringla. In the last decade, a method proposed by Burrows (2002) has been shown to be effective for a variety of corpora in different languages and gained widespread acceptance (Fotis et al., 2015; Eder and Rybicki, 2013). It is particularly valuable and convenient that implementations of Burrows’ method, with several variations, are freely available.

I am not aware of any previous testing of Burrows’ method for Icelandic texts so it will be valuable to work through a control corpus. This will also give me the opportunity to explain the working of the method.

In the following I use Hoover’s (2005–2015) implementation of Burrow’s method. The method is, for the most part, intuitive and does not require a heavily technical background to understand. The basic idea is that each author has a characteristic word frequency pattern. By comparing word frequency in two different texts we can establish delta, a statistical measure of the difference between the texts. When a pair of texts has a low delta this can, in the right context, indicate that they have the same author.

For a control corpus we need a collection of substantial texts by known authors working in the same time period. The texts should be similar in kind, not e. g. a mix of fiction, personal letters and spiritual literature. Since most Old Icelandic texts are anonymous, there is little prospect of an Old Icelandic control corpus. Instead I turn to modern Icelandic texts, specifically fiction from the period 1850–1920, where a number of out-of-copyright texts are available in digital form from Netútgáfan. My primary sample consists of works by five authors:

I have obtained digital copies of those texts from Netútgáfan and removed some material extraneous to the prose fiction under examination: chapter headings, poetry quotes and non-narrative prologues and epilogues.

To analyze the primary sample we compile tables showing the number of occurrences of each word form in each text. Various tools exist for this purpose, I have used Linux command line utilities but applications with graphical user interfaces are also available. The results for Halla, in abbreviated form, are as follows:

When we have tabulated word form occurrences for each text in the primary sample, we can assemble a table with frequencies. For example, the lexeme og occurs 2697 times in Halla, which has a total of 53,410 words – thus there are about 505 occurrences per 10,000 words.

Table 3 shows the frequency of the five most common words in the primary corpus. The table also shows the standard deviation for each lexeme, a measure of the variation in the data. The frequency of the lexeme að varies substantially more in our texts than that of the lexeme í, thus the former has a much higher standard deviation.

Before proceeding further we may wonder if we should exclude some words from the comparison. Research on English texts has shown that results can sometimes be improved by removing pronouns. This is helpful to attenuate the difference between first and third person narratives and the effect of the gender of the main characters. I have experimented with removing pronouns from the analysis but this did not have a substantial effect so I have included them in the results published here.

Another issue is the presence of words primarily or exclusively found in one of the primary texts. Table 4 shows some examples:

Words which are frequent in one text but rare or non-existent in the others are typically names or other very context-specific words. It is moderately helpful to remove these from consideration. In the following analysis I have, following Hoover (2005–2015), removed words where one of the primary texts is responsible for more than 70 % of the occurrences.

A final question which needs to be answered is how many words are to be included in the analysis. Burrows’ original demonstration used 150 words but subsequent research has shown that using more words improves accuracy, up to a point (Hoover, 2005–2015). The ideal number of words depends on the length of the texts. For the following analysis I have chosen to use the 1000 most frequent words, which is a reasonable compromise for texts of varying length. For good measure I also include runs with the 500 and 2000 most frequent words.

After this preparatory work we can get down to the business of evaluating sample texts. I use the following novels, novellas and short stories by the authors from the primary corpus. I am in no way cherry-picking texts, my corpus consists of all the texts available to me:

We now proceed to tabulate the frequency of word forms in each sample text and compare the result with the samples in the primary set. The results for the top five words are as follows in a comparison between Leysing and the Jón Trausti primary sample (Halla):

We obtain a z-score, also known as a standard score, by dividing the difference in frequency with the standard deviation. To finally obtain a Burrows’ delta score we add up the z-score of all 1000 words. When we do this we get the following results for Leysing:

We see that delta is lowest between Leysing and Jón Trausti (Halla), which is consistent with the fact that Jón Trausti is the actual author of the text. The method, then, has scored a success but we may be interested in some measure of confidence in a given result. We see from the table that the difference between the best and the second best match is 227 while the difference between scores 2 and 5 is merely 144. The best match is clearly separated from the rest of the pack, which is consistent with high confidence in the result. As a convenient metric we can use the percentage increase in delta between the best and the second best match (i. e. the difference between the best two scores divided by the best score). In this case there is an increase of 25 %.

We can now show the results for all the test texts:

The delta test identifies the correct author in 27 cases out of 28. Considering that many of the texts are quite short, this is an astonishingly successful run. In eight cases the algorithm identifies the correct author only by the skin of its teeth, with a difference score of 2.5 % or less. This suggests that we have been lucky and indeed it turns out that a small change in parameters yields less accurate results. If we run the test with the 500 most frequent words we get four erroneous attributions:

The average size of the incorrectly attributed texts is 14,257 words, this is consistent with the principle that longer texts typically benefit from taking more words into account. If we run the test with the 2000 most frequent words we get nine errors:

In this case the short texts run into difficulties, the average text has a length of 2,108 words. Consistent with previous research, short texts are most profitably analyzed with a relatively low number of words.

It is important to note that we do not get any erroneous attributions with a high difference score. All attributions with a score of 2 % or more are correct in the data examined so far.

In the previous section we were in the happy position of dealing with texts which we knew to be by one of the authors in the primary set. A more difficult situation, and one more realistic in the context of our ultimate goals here, is when the text whose authorship is in question may not be by any of the authors whose stylistic fingerprint we are comparing with. To test the method against this possibility I have compared the primary corpus from the previous section with a test corpus consisting of prose by other authors. Since the supply of available texts is limited, this corpus includes some translations. The texts are as follows (all from Netútgáfan except Umhverfis jörðina á 80 dögum and Fanginn í Zenda, which were obtained from Rafbókavefurinn):

The results of the delta test for those texts is as follows:

The difference scores are in the range 0.0 %–5.6 % and the average is 2.0 %. This is a bit higher than the results for the erroneously attributed texts in test corpus 1. But compared with the correctly attributed texts, this is not very high. Of the 27 correctly attributed texts in Table 8, 12 have a difference score higher than 5.6 %. This includes all four texts longer than 25,000 words.

The preceding sections have allowed me to showcase Burrows’ method and to establish that it is effective in identifying the authors of texts in Icelandic. I now turn to the main object of inquiry, the putative stylistic connection between Heimskringla and Egils saga.

In the control corpus, I used texts available in digital form and only exerted minimum effort to normalize their presentation. But when dealing with Old Icelandic texts, more care is needed. It would, for example, be absurd to compare editions with modern Icelandic orthography directly to editions with normalized Old Norse orthography. Since most digital texts available to me use modern orthography, I have decided to use this as a standard. More specifically, I have followed the conventions used at Netútgáfan for the Sagas of Icelanders but striven for more consistency.

Even in a framework of normalized spelling, editors may choose to retain some archaic forms or to follow the main manuscript on its choice between variant word forms. But since such variation is not likely to stem from the original author, I have sought to normalize it away. Some representative examples of variation which I have done away with are as follows:

As in the previous test, I have removed words where one primary text is responsible for more than 70 % of occurrences. I have also manually removed all remaining proper names.

We can now move on to defining a primary corpus. For the first study the question to be examined is whether one of the large sagas of Icelanders is significantly more similar than the others to Heimskringla. This is the same question Hallberg originally studied. The primary corpus is as follows:

The work to be tested is Heimskringla which I will handle in two parts. There is good evidence that Óláfs saga helga (“Heimskringla A”) was written separately from the rest (“Heimskringla B”). Indeed, Jonna Louis-Jensen (2009, 2013; see also 1997) argues that the two parts have different authors.

We can now compare our test corpus with our primary corpus. For the 1000 most frequent words, the results are as follows:

For both parts of Heimskringla, Egils saga is the most similar text, by a healthy margin. It matters little whether the analysis uses the 500, 1000 or 2000 most frequent words:

These results are consistent with the theory that Egils saga and (both parts of) Heimskringla have the same author. But this is not the only conceivable explanation. One alternative possibility is that Egils saga has stylistic affinity with the kings’ sagas in general rather than Heimskringla in particular. To check for this effect, comparison should be made with more kings’ sagas.

Another alternative is that the similarity between Egils saga and Heimskringla is a consequence of their being composed around the same time while the other major sagas of Icelanders might be younger (Elín Bára Magnúsdóttir 2015, 274). In Vésteinn Ólason’s categorization of the sagas of Icelanders, the sagas are divided into three chronological groups. Egils saga belongs to the oldest group (1200–1280) while Grettis saga is placed in the youngest group (1300–1450) and Eyrbyggja saga, Njáls saga and Laxdæla saga are placed in the middle group (1240–1310) (Vésteinn Ólason 1993, 42). To test against this idea, comparison should be made with works believed to be close in age to Egils saga and Heimskringla.

For a more challenging test of the connection between Egils saga and Heimskringla I have prepared a second primary corpus to address the concerns raised in the previous section. In this case we are using a variety of historical texts for comparison:

Jómsvíkinga saga may have been composed in the early 13th century. It is a sui generis text with affinity to the kings’ sagas (Finlay, 2014). Knýtlinga saga is one of the kings’ sagas – seemingly composed in conscious imitation of Heimskringla. There are reasons to suppose that it was composed by Óláfr Þórðarson (d. 1259), Snorri Sturluson’s nephew (Bjarni Guðnason 1982, clxxxix–clxxxiv).³ Íslendinga saga deals with events in Iceland in the 13th century. It is attributed to Sturla Þórðarson (1214–1284), another nephew of Snorri Sturluson. Morkinskinna is one of the kings’ sagas, believed to have been composed shortly before Heimskringla. It is one of Heimskringla’s sources.

Jómsvíkinga saga was obtained from Netútgáfan, Knýtlinga saga was obtained from the Heimskringla text collection, Íslendinga saga was specially provided by the Árni Magnússon Institute from Íslenskt textasafn, the Morkinskinna text was typed up from the Íslenzk fornrit edition (Ármann Jakobsson and Þórður Ingi Guðjónsson 2002) – I limited myself to the first 30,000 words of the text preserved in the oldest manuscript (GKS 1009 fol.). As in the previous case, I took pains to normalize all the texts to the same standard.

The results of a test with the 1000 most frequent words yields the following results:

The results are largely consistent with different parameters:

These results demonstrate that the stylistic affinity between Heimskringla and Egils saga is not merely the consequence of a closeness in age or a closeness between Egils saga and the kings’ sagas. Consistent with previous research, Óláfs saga helga is found to be even more similar to Egils saga than the other parts of Heimskringla are.

This is not the first attempt to determine the degree of stylistic affinity between Egils saga and Heimskringla and it is natural to ask how it compares with previous research. The principal difference between this method and Hallberg’s (1962) influential pair word investigation is that here we are dealing with common words while the pair words belong to the rare part of the vocabulary. There is, thus, essentially no overlap between these two studies – they complement each other.

Hallberg’s pair words method was criticized on various methodological grounds by Marina Meier (1963). In particular, she pointed out Hallberg’s lack of statistical sophistication in compensating for the problem that the texts he studied were of various lengths (a problem returned to in Leoni 1970 and Louis-Jensen 2009). Hallberg replied (1964), clarifying his methods and arguing that his simple methods were adequate to the task. In my view, his reply is largely satisfactory – more sophisticated methods are unlikely to yield significantly different results.

In addition to his pair word research, Hallberg also conducted investigations into a few particular words and collocations. Already in his first study, he noted that the verb kveðask has an unusually low frequency both in Heimskringla and in Egils saga (1962, 52–56). This was followed up by various similar observations. One of the traits studied was the frequency of sentence initial ok er versus en er (both meaning “when”) (Hallberg 1963, 10–11; Hallberg 1968, 200–202), where a high frequency of en er is the putative Snorri trait. In this case, the stability of the manuscript transmission has been investigated (Haukur Þorgeirsson 2014).

The problem with studies of individual words is that the investigator is open to the charge of opportunism. Why did we choose to focus on kveðask and en er rather than a hundred other common words or phrases? In comparing any two given texts, it is surely possible to find some shared trait if we allow ourselves to pick any convenient target, an issue sometimes referred to as the multiple comparisons problem. While this does not negate the value of studying individual traits, it highlights the advantage of methods like Hallberg’s pair word method and Burrow’s delta in which the investigator proceeds according to a pre-defined plan.

Following in Hallberg’s footsteps, Jonna Louis-Jensen has investigated certain stylistic traits in Heimskringla and Egils saga which may have diachronic implications. She correctly points out that Óláfs saga helga and Egils saga have a preference for til þess er over þar til er (both meaning ‘until’) and a preference for hitta over finna (in the sense ‘meet’) (Louis-Jensen, 2009, 108–110). In the remainder of Heimskringla, there is a balance between til þess er and þar til er and a preference for finna over hitta. According to Hallberg’s diachronic studies, this suggests that “Heimskringla B” is a younger work than Egils saga and Óláfs saga helga. Louis-Jensen further backs this up with observations on some archaic features in Egils saga which are not present in Heimskringla (Louis-Jensen, 2013, 142–145).

Louis-Jensen’s evidence does suggest that Óláfs saga helga may be an older text than the rest of Heimskringla. This should not be a surprising result since it has been commonly accepted since Sigurður Nordal 1914 that this was the order in which these texts were written.⁴ The style of a given author changes throughout his or her life Stamou (2007) and such changes can reflect broader trends in the language community (Can/Patton, 2004, 2010). Thus, the chronological development which Louis-Jensen points to does not constitute strong evidence against common authorship. As pointed out by Wright (2015, 9–11), all the archaic features she identifies fit comfortably within Snorri’s working lifetime.

This article confirms that Burrows’ delta is an effective tool for authorship attribution of Icelandic texts, performing well on the test case of 19th century prose fiction. When this test is then applied to Heimskringla and the sagas of Icelanders, Egils saga turns out to have the lowest delta by a large margin.

In a more challenging test, Heimskringla is compared with Egils saga, Jómsvíkinga saga, Morkinskinna, Íslendinga saga and Knýtlinga saga. Even here, Egils saga has the lowest delta for both parts of Heimskringla. These results constitute substantial support for the theory of common authorship of Egils saga and Heimskringla.