Spark DataSet - Data Type

About

Same as Spark SQL - Data Type

Articles Related

Coercion

Python

from pyspark.sql.functions import col , column
dataFrame = dataFrame.withColumn("price", col("price").cast("double"))
dataFrame = dataFrame.withColumn("time_at", col("time_at").cast("timestamp"))

List

Supported types are:

• string,
• boolean,
• byte,
• short,
• int,
• long,
• float,
• double,
• decimal,
• date,
• timestamp