当前位置:网站首页>org. apache. parquet. schema. InvalidSchemaException: A group type can not be empty. Parquet does not su
org. apache. parquet. schema. InvalidSchemaException: A group type can not be empty. Parquet does not su
2022-04-23 16:43:00 【A hundred nights】
1. Read cvs File data write parquet file
val spark =SparkSession.builder().appName("test").master("local").getOrCreate()
val file = "hdfs://clusters/test/demo.csv"
val frame = spark.read.option("header","true").csv(file)
frame.printSchema()
val newfile = file.split("\\.")(0)+".parquet"
frame.write.parquet(newfile)
Wrong presentation :
org.apache.parquet.schema.InvalidSchemaException: A group type can not be empty. Parquet does not support empty group without leaves. Empty group: spark_schema
I found a lot of information on the Internet , The general explanation is that dataframe Of schema Structural problems , This is not the question of this article .
after N Attempts to , Finally, the problem is .parquet(newfile). Code program query , Found this parquet Method called parquet There are two packages , The first package is used by default , However, this bag is not spark-sql Under the parquet package .
then , I deleted the version with a try mentality 1.8.1 Of parquet package , Stay and spark Dependent parquet package , Rerun code , press forward to the enemy's capital .

版权声明
本文为[A hundred nights]所创,转载请带上原文链接,感谢
https://yzsam.com/2022/04/202204231401520905.html
边栏推荐
- Day 10 abnormal mechanism
- The solution of not displaying a whole line when the total value needs to be set to 0 in sail software
- Calculate pie chart percentage
- Do you really understand the principle of code scanning login?
- 昆腾全双工数字无线收发芯片KT1605/KT1606/KT1607/KT1608适用对讲机方案
- Gartner 發布新興技術研究:深入洞悉元宇宙
- loggie 源码分析 source file 模块主干分析
- 05 Lua control structure
- Creation of RAID disk array and RAID5
- Custom implementation of Baidu image recognition (instead of aipocr)
猜你喜欢

英语 | Day15、16 x 句句真研每日一句(从句断开、修饰)

Ali developed three sides, and the interviewer's set of combined punches made me confused on the spot

Pytorch: the pit between train mode and eval mode

JMeter setting environment variable supports direct startup by entering JMeter in any terminal directory

LVM and disk quota

The font of the soft cell changes color

MySql主从复制

ByteVCharts可视化图表库,你想要的我都有

Deepinv20 installation MariaDB

安装及管理程序
随机推荐
◰ GL shader handler encapsulation
The most detailed knapsack problem!!!
关于 background-image 渐变gradient()那些事!
Sail soft segmentation solution: take only one character (required field) of a string
Sail soft implements a radio button, which can uniformly set the selection status of other radio buttons
文件操作详解(2)
NVIDIA graphics card driver error
织梦DEDECMS安全设置指南
File upload and download of robot framework
LVM与磁盘配额
Selenium IDE and XPath installation of chrome plug-in
MySql主从复制
Disk management and file system
Phpstudy V8, a commonly used software for station construction 1 graphic installation tutorial (Windows version) super detailed
◰GL-着色器处理程序封装
阿里研发三面,面试官一套组合拳让我当场懵逼
Detailed explanation of UWA pipeline function | visual configuration automatic test
TypeError: set_figure_params() got an unexpected keyword argument ‘figsize‘
On the security of key passing and digital signature
Deepinv20 installation MariaDB