PySpark - Version

About

version in PySpark.

All information about PySpark version and its components.

Articles Related

List

• The version of PySpark on which Python version it's support can be found in the Release History: https://pypi.org/project/pyspark/#history
• In a Spark Distribution, in the version file: “spark-2.2.0-bin-hadoop2.7\python\pyspark\version.py”

Dependency

• At its core PySpark depends on Py4J, but some additional sub-packages have their own extra requirements for some features (including numpy, pandas, and pyarrow).