Jean-Sébastien Coron ORCID logo and Agnese Gini

Improved cryptanalysis of the AJPS Mersenne based cryptosystem

De Gruyter | Published online: July 21, 2020

Abstract

At Crypto 2018, Aggarwal, Joux, Prakash and Santha (AJPS) described a new public-key encryption scheme based on Mersenne numbers. Shortly after the publication of the cryptosystem, Beunardeau et al. described an attack with complexity 𝓞(22h). In this paper, we describe an improved attack with complexity 𝓞(21.75h).

MSC 2010: 94A60

1 Introduction

The AJPS public-key encryption scheme

At Crypto 2018, Aggarwal, Joux, Prakash and Santha (AJPS) described a new public-key encryption scheme based on arithmetic modulo Mersenne numbers [2]. A Mersenne prime is a prime integer p of the form p = 2n − 1, where n is a prime. The arithmetic modulo p has good properties, and one can establish a correspondence between integers modulo p and binary strings of length n, up to 0n ∼ 1n. In particular, one can define the Hamming weight of a number as the Hamming weight of the unique binary string associated to it, i.e. the number of ones in its binary representation. In the earliest version of their work, the authors presented a public-key encryption scheme (AJPS-1) somewhat similar to the NTRU cryptosystem, but based on a new assumption, the Mersenne low Hamming ratio assumption. Its security relies on the following assumption: given H = F/G mod p, where the binary representation of F and G modulo p has low Hamming weight, then H looks pseudorandom; namely, it is hard to distinguish H from a random integer modulo p.

The Beunardeau et al. attack

Even though the authors claimed that the known lattice attacks against NTRU would not apply, very soon, Beunardeau et al. [3] described a lattice-based attack against the first AJPS proposal. The attack complexity is 𝓞(22h), where h is the Hamming weight of F and G. The attack was further analyzed in [4]; the authors also described a meet-in-the-middle attack against AJPS-1 based on locality-sensitive hash functions to obtain collisions; they showed that the lattice attack from [3] is more efficient.

Since AJPS-1 allows to encrypt only a single bit at a time, it is not very efficient. However, in a later version of the article, published at Crypto 2018 [2], Aggarwal et al. described a variant (AJPS-2) that encrypts many bits at a time, with much larger security parameters to prevent the lattice attack.

Our contribution

In this paper, we describe a variant of the Beunardeau et al. attack against AJPS-2, with improved complexity 𝓞(21.75h) instead of 𝓞(22h). Instead of recovering the private key, our attack only breaks the indistinguishability of ciphertexts.

2 The AJPS cryptosystems

In this section, we recall the two versions of the AJPS cryptosystems; see [2] for further details.

AJPS-1: bit-by-bit encryption

Let p = 2n − 1 be a Mersenne prime, where n itself is prime. Let h be an integer. Let F and G be two random integers modulo p with Hamming weight h such that 4h2 < n ≤ 16h2. Then the public key is pk = H = F/G mod p and the private key is sk = G. To encrypt, choose two random integers A and B of Hamming weight h. Encrypt the bit b as

C = ( 1 ) b ( A H + B ) .

To decrypt, compute d = Ham(CG). Output 0 if d ≤ 2h2; otherwise, output 1.

Decryption works because

C G = ( 1 ) b ( A H G + B G ) = ( 1 ) b ( A F + B G )

which has Hamming weight at most 2h2 if b = 0, and at least n − 2h2 if b = 1. Namely, for any number x of Hamming weight h, the integer x ⋅ 2z mod p for z ≥ 0 is a cyclic shift of x, and therefore its Hamming weight remains unchanged. Therefore, the Hamming weight of AF is at most h2, and the Hamming weight of BG is also at most h2; therefore, the Hamming weight of CG is at most 2h2 for b = 0.

AJPS-2: error correcting codes

Let n be a positive integer such that p = 2n − 1 be a Mersenne prime. Let h ∈ ℕ be such that 10h2 < n ≤ 16h2. Let F, G be two random integers modulo p with Hamming weight h, and let R be a random integer modulo p. Set

p k = ( R , F R + G ) = ( R , T ) and s k = F .

To encrypt a message m ∈ {0, 1}h, first generate three random integers A, B1, B2 modulo p, with Hamming weight h. Then, using the encoding algorithm 𝓔 : {0, 1}h → {0, 1}n of an error correcting code (𝓔, 𝓓), compute the ciphertext

( C 1 , C 2 ) = ( A R + B 1 , ( A T + B 2 ) E ( m ) ) .

To decrypt, compute 𝓓((FC1) ⊕ C2), where 𝓓 is the corresponding decoding algorithm.

Decryption works because

F C 1 = A F R + F B 1 = A ( T G ) + F B 1 = ( A T + B 2 ) A G B 2 + B 1 F ,

and therefore the Hamming distance between AT + B2 and FC1 is expected to be low, which enables to recover m with good probability.

3 The Beunardeau et al. attack

Basic attack

Beunardeau et al. described an attack against AJPS-1 in [3] that recovers the private key from the public key. More precisely, they consider the following problem.

Definition 3.1

(Mersenne low Hamming ratio search problem (MLHSP)). Let p = 2n − 1 be an n-bit Mersenne prime and h an integer. Let F, G be two n-bit random strings with Hamming weight h. Given H = F/G mod p, recover F and G.

Their basic attack is based on the following observation. With probability 2−2h, we have both F < p and G < p , and therefore, given H = F/G mod p, one can recover F and G by applying LLL in dimension 2. In the original proposal [1], it was recommended to take h = 17 for λ = 120 bits of security. However, here we have an attack that recovers the private key from the public key with probability 2−34; see also [4] for a detailed analysis.

More precisely, one considers the lattice 𝓛 generated by the rows of the matrix

1 H 0 p .

We have that det 𝓛 = p; hence, by the Gaussian heuristic, it contains a vector of norm ( det L ) 1 2 = p . Moreover, (G, F) is a short vector of the lattice. Therefore, if both F < p and G < p , we can recover F and G; since F and G have Hamming weight h, this happens with probability 2−2h.

We note that a similar attack can also be applied to the encryption equation C = (−1)b ⋅ (AH + B). Namely, if both A < p and B < p , then we can recover A and B by applying LLL in dimension 3, hence the plaintext bit b. Indeed, we have that only one between (H, C) and (−H, C) is an instance of the following problem.

Definition 3.2

(Mersenne low Hamming combination search problem (MLHCSP)). Let p = 2n − 1 be an n-bit Mersenne prime, h an integer, R a uniformly random n-bit string, and let F, G have Hamming weight h. Given the pair (R, FR + G mod p), find F, G.

Given R and T = FR + G mod p, a variant attack recovers F, G with probability 2−2h. More precisely, the attack works by considering the lattice 𝓛 of row vectors

2 n 2 0 T 0 1 R 0 0 p .

We have that ( 2 n 2 , F, G) belongs to the lattice 𝓛. Moreover, det L = 2 n 2 p 2 3 n 2 . Hence, by the Gaussian heuristic, the lattice 𝓛 contains a vector of norm ≃ 2 n 2 . Therefore, if both F < p and G < p , we can recover F and G by applying LLL to the lattice 𝓛.

Extension with random partitions

The basic attack from [3] is only a weak-key attack that recovers the private key from the public key with probability 2−2h over the set of possible public keys. Similarly, the above variant attack against the encryption equation can only decrypt a fraction 2−2h of the ciphertexts. Therefore, the authors extended their attack by considering random partitions, with higher-dimensional lattices. In that case, the attack can recover the private key from any public key, solving MLHSP, with complexity 𝓞(22h). The same partition strategy can be used for the MLHCSP with the same complexity. In our improved attack in the next section, we will also use random partitions.

4 Our new attack

We describe our new attack against AJPS-2. We consider the previous encryption equation

( C 1 , C 2 ) = ( A R + B 1 , ( A T + B 2 ) E ( m ) )

Given the public key (R, T) and a ciphertext (C1, C2), our attack can distinguish between m = 0 and m ≠ 0. Assume that m = 0 and 𝓔(m) = 0. In that case, we have

C 1 = A R + B 1 , C 2 = A T + B 2 .

We claim that if A, B1 and B2 are less than 2 2 n 3 , then we can recover A, B1 and B2 with LLL. Namely, we consider the lattice of row vectors

2 2 3 n 0 C 1 C 2 0 1 R T 0 0 p 0 0 0 0 p .

We have that det L = 2 2 n 3 p 2 2 8 n 3 . Therefore, by the Gaussian heuristic, the lattice 𝓛 contains vectors of norm ≃ 2 2 n 3 . Moreover, the lattice 𝓛 contains the vector ( 2 2 n 3 , A, B1, B2). Therefore if A, B1 and B2 are less than 2 2 n 3 , we can recover A, B1 and B2 by applying LLL to 𝓛.

Since A has Hamming weight h, the probability that A < 2 2 n 3 is ( 2 3 ) h ; the same holds for B1 and B2. The success probability of the attack is therefore

( 2 3 ) 3 h 2 1.75 h

which gives a slightly better success probability than the original attack with 2−2h. Therefore, using the same partition technique as in [3], the attack complexity to break the indistinguishability of any ciphertext is 𝓞(21.75h) instead of 𝓞(22h).

4.1 Working with random partitions

We show that, using the same random partition technique as in [3], we can break the indistinguishability property of any ciphertext (C1, C2), whereas the basic attack above only works when A, B1 and B2 are less than 2 2 n 3 , which only happens with probability ( 2 3 ) 3 h .

We consider the set [n] = {0, 1, …, n − 1}. We say that P = { P i } i = 1 k is an interval-like partition if it is a partition of [n] such that the sets are of the form Pi = {y : cyd or Pi = {d, d + 1, …, 0, …, c − 1, c for cd ∈ [n]. We define pi as the least element of Pi, namely as c if the interval is of the first type and as d if it is of the second type. We can use a partition to represent a number E modulo p by a sequence of smaller integers. More precisely, letting en−1e0 be the binary representation of e, we can divide it by the partition

e p 1 1 e p k e p k 1 e p k 1 e p 2 1 e p 1 ,

and letting di the number represented by epi−1epi−1, we obtain

E = i = 1 k d i 2 p i .

Consider P, Q, S three interval-like partitions of [n] of cardinality k, and j, respectively. Let R, T, C1, C2, A, B1, B2 be as in AJPS-2. We define a family of embedded lattices parameterized with respect to β,P, Q, S as

L β , P , Q , S = { ( α β , x , y , z ) Z × Z k × Z × Z j : α C 1 R i = 1 k x i 2 p i + i = 1 y i 2 q i mod p , α C 2 T i = 1 k x i 2 p i + i = 1 j z i 2 s i mod p }

for some scaling factorβ ∈ ℤ. The dimension of 𝓛β,P,Q,S is d = k + + j + 1, and a basis of this lattice is given by rows of the matrix

M β , P , Q , S = β 0 0 0 0 0 C 1 2 q 1 0 0 C 2 2 s 1 0 1 0 0 0 0 R 2 p k q 1 0 0 T 2 p k s 1 0 0 1 0 0 0 R 2 p k 1 q 1 0 0 T 2 p k 1 s 1 0 0 0 R 2 p 2 q 1 0 0 T 2 p 2 s 1 0 0 0 1 0 0 R 2 p 1 q 1 0 0 T 2 p 1 s 1 0 0 0 0 1 0 2 q q 1 0 0 0 0 0 0 0 2 q i q 1 0 0 0 0 0 0 0 0 1 2 q 2 q 1 0 0 0 0 0 0 0 0 0 p 0 0 0 0 0 0 0 0 0 0 1 0 2 s j s 1 0 0 0 0 0 0 0 2 s i s 1 0 0 0 0 0 0 0 0 1 2 s 2 s 1 0 0 0 0 0 0 0 0 0 p .

We claim that we can recover A, B1, B2 by using a lattice of the family {𝓛β,P,Q,S}. We define the secret vector to be

s := ( β , a 1 , , a k , b 1 ( 1 ) , , b ( 1 ) , b 1 ( 2 ) , , b j ( 2 ) ) L β , P , Q , S ,

where 0 a i < 2 p i 0 b i ( 1 ) < 2 q i 0 b i ( 2 ) < 2 s i and

A = i = 1 k a i 2 p i , B 1 = i = 1 b i ( 1 ) 2 q i , B 2 = i = 1 j b i ( 2 ) 2 s i .

We will use the notations a = (a1, …, ak), b ( 1 ) = ( b 1 ( 1 ) , , b ( 1 ) ) , b ( 2 ) = ( b 1 ( 2 ) , , b j ( 2 ) ) , e = (a, b(1), b(2)) and s = (β, e).

In the following, we determine under which conditions the secret vector s is the unique shortest vector of the lattice 𝓛β,P,Q,S. Given A, B1, B2, we say that the triple (P, Q, S) of partitions of [n] is a lucky triple if there exists a scaling factor β ∈ ℕ such that the secret vector s is the unique shortest vector of 𝓛β,P,Q,S. In that case, 𝓛β,P,Q,S will be said to be a lucky lattice respect to A, B1, B2. In other words, we aim to establish sufficient conditions under which a lattice 𝓛β,P,Q,S is lucky given a ciphertext C = (C1, C2) such that 𝓔(m) = 0.

The volume of 𝓛β,P,Q,S is

vol ( L β , P , Q , S ) = det ( M ) = p 2 β .

We write β = 2tn; thus we have vol(𝓛β,P,Q,S) ≃ 2(2+t)n. By the Gaussian heuristic, we obtain the following estimate of the length of the shortest vector of 𝓛β,P,Q,S:

d 2 π e vol ( L β , P , Q , S ) 1 d = d 2 π e 2 ( 2 + t ) n d . (4.1)

Since the Hamming weight of A, B1, B2 is the same, we take k = j = . We note that the lattice 𝓛β,P,Q,S contains intrinsic short vectors u = (0, …, 0, 2g, −1, 0, …, 0) whose norm is ≃ 2g when g is of the form pipi−1 or qiqi−1 or sisi−1. If we consider partitions with intervals of similar length, we obtain u 2 n k . Therefore, we have to ensure that such vectors are not shorter than our target secret vector.

In low dimensions, we can assume that LLL recovers the shortest vector s of the lattice. From (4.1), we must therefore ensure

s d 2 π e 2 ( 2 + t ) n d ,

where d = 3k + 1 is the lattice dimension. We expect the entries of the secret vector to be about of the same size for a lucky triple; hence we take the scaling factor β such that β=2tn ≃ ∥e∥. Then we have approximately

2 t n + 1 2 2 ( 2 + t ) n 3 k + 1

which gives t 2 3 k 3 k + 1 6 k n . Therefore, we have the approximative condition to have a lucky triple (P, Q, S) of partitions

e < 2 2 n 3 k . (4.2)

It remains to evaluate the probability to find a lucky triple of partitions (P, Q, S). It is actually easier to assume that the partitions (P, Q, S) are fixed and the ciphertext C = (C1, C2) is random. In that case, from the bound (4.2), each of the h bits from the integers A, B1 and B2 must land in one of the subintervals of length 2 n 3 k of the k partition intervals. For a single bit, this happens with probability roughly k 2 n 3 k 1 n = 2 3 . Therefore, as in the basic attack, the success probability is roughly ( 2 3 ) 3 h ≃ 2−1.75h. Therefore, the number of partitions to try before finding a lucky one is approximately 𝓞(21.75h) instead of 𝓞(22h) in the original attack from [3].

Security parameter selection

In the latest version of the paper, the authors recommended to take for λ bit of security h = λ in order to prevent possible improvements of the Beunardeau et al. attack. Then our attack does not affect the choice of parameter proposed in [2].

4.2 Practical experiments

We have performed some practical experiments for various values of bitsize n and Hamming weight h of AJPS-2 in order to compare our new attack with the original Beunardeau et al. attack. For both attacks, since we do not know a priori the optimal size of the partition k to recover the secret, we perform a repeated loop over all possible 1 ≤ kh. We summarize our results in Table 1, showing that our attack indeed requires fewer partitions than the original attack.

Table 1

Average number ȳ of partitions required to recover the secret values A, B1, B2, compared to the average number Ȳ required for the original attack. We used 70 samples for h = 3, 6, 7 and 9 samples for h = 9.

h n log2(ȳ) log2(Ȳ)
3 127 6.5 7.4
6 521 13.0 14.5
7 607 14.6 16.5
9 1279 14.9 16.4

References

[1] D. Aggarwal, A. Joux, A. Prakash and M. Santha, A new public-key cryptosystem via Mersenne numbers, Cryptology ePrint Archive (2017), https://eprint.iacr.org/2017/481. Search in Google Scholar

[2] D. Aggarwal, A. Joux, A. Prakash and M. Santha, A new public-key cryptosystem via Mersenne numbers, in: Advances in CryptologyCRYPTO 2018, Lecture Notes in Comput. Sci. 10993, Springer, Berlin (2018), 459–482. Search in Google Scholar

[3] M. Beunardeau, A. Connolly, R. Géraud and D. Naccache, On the hardness of the Mersenne low Hamming ratio assumption, Cryptology ePrint Archive (2017), https://eprint.iacr.org/2017/522. Search in Google Scholar

[4] K. de Boer, L. Ducas, S. Jeffery and R. de Wolf, Attacks on the AJPS Mersenne-based cryptosystem, in: Post-Quantum CryptographyPQCrypto 2018, Lecture Notes in Comput. Sci. 10786, Springer, Cham (2018), 101–120. Search in Google Scholar

Received: 2019-07-13
Accepted: 2019-09-23
Published Online: 2020-07-21

© 2020 Jean-Sébastien Coron, Agnese Gini, published by De Gruyter

This work is licensed under the Creative Commons Attribution 4.0 International License.