Introduction to PartitioningΒΆ
Let us get an overview of partitioning of Spark Metastore tables.
It is similar to list partitioning where each partition is equal to a particular value for a given column.
Spark Metastore does not support range partitioning and bucketing. Bucketing is supported in Hive which is similar to Hash Partitioning.
Once the table is created, we can add static partitions and then load or insert data into it.
Spark Metastore also support creation of partitions dynamically, where partitions will be created based up on the partition column value.
A Partitioned table can be managed or external.