CVE-2019-10099

Spark 2.3.3 Pre-Encryption Data Leakage

• CVSS 7.5
• CWE-312 Cleartext Storage of Sensitive Information
• Cryptographic
• Local

Prior to Spark 2.3.3, in certain situations Spark would write user data to local disk unencrypted, even if spark.io.encryption.enabled=true. This includes cached blocks that are fetched to disk (controlled by spark.maxRemoteBlockSizeFetchToMem); in SparkR, using parallelize; in Pyspark, using broadcast and parallelize; and use of python udfs.

CVE

CVE-2019-10099

CWE

CWE-312 Cleartext Storage of Sensitive Information

CVSS base score

7.5

Published

2019-08-07

OWASP

A08 Software and Data Integrity Failures

Orthogonal defect classification

Function

Code defect classification

Incorrect Functionality

Category

Cryptographic

Subcategory

Unencrypted communication

Accessibility scope

Local

Impact

Information Disclosure

Fixed by upgrading

Yes

Solution

Upgrade to Spark 2.3.3 or higher.

from pyspark import SparkConf, SparkContext

conf = SparkConf()
conf.set("spark.io.encryption.enabled", "true")
conf.set("spark.maxRemoteBlockSizeFetchToMem", "64m")

sc = SparkContext(conf=conf)

data = [1, 2, 3, 4, 5]

rdd = sc.parallelize(data)

broadcast_var = sc.broadcast(rdd.collect())

def multiply_by_two(x):
    return x * 2

result = rdd.map(multiply_by_two).collect()

print(result)

sc.stop()

from pyspark import SparkConf, SparkContext

conf = SparkConf()
conf.set("spark.io.encryption.enabled", "true")
conf.set("spark.authenticate", "true")
conf.set("spark.authenticate.secret", "your-strong-shared-secret")
conf.set("spark.network.crypto.enabled", "true")
conf.set("spark.network.crypto.keyLength", "128")
conf.set("spark.network.crypto.keyFactoryAlgorithm", "PBKDF2WithHmacSHA1")
conf.set("spark.broadcast.compress", "true")
conf.set("spark.maxRemoteBlockSizeFetchToMem", "64m")

sc = SparkContext(conf=conf)

data = [1, 2, 3, 4, 5]

rdd = sc.parallelize(data)

broadcast_data = [int(x) for x in rdd.collect() if isinstance(x, int)]
broadcast_var = sc.broadcast(broadcast_data)

def multiply_by_two(x):
    return x * 2

result = rdd.map(multiply_by_two).collect()

print(result)

sc.stop()

@misc{vaitp:cve201910099,
  title        = {{Spark 2.3.3 Pre-Encryption Data Leakage}},
  author       = {Bogaerts, Fr\'ed\'eric and Ivaki, Naghmeh and Fonseca, Jos\'e},
  year         = {2019},
  note         = {VAITP Python Vulnerability Dataset, entry CVE-2019-10099},
  howpublished = {\url{https://netpack.pt/vaitp/vulnerability/CVE-2019-10099/}}
}