当前位置:网站首页>org. apache. parquet. schema. InvalidSchemaException: A group type can not be empty. Parquet does not su
org. apache. parquet. schema. InvalidSchemaException: A group type can not be empty. Parquet does not su
2022-04-23 16:43:00 【A hundred nights】
1. Read cvs File data write parquet file
val spark =SparkSession.builder().appName("test").master("local").getOrCreate()
val file = "hdfs://clusters/test/demo.csv"
val frame = spark.read.option("header","true").csv(file)
frame.printSchema()
val newfile = file.split("\\.")(0)+".parquet"
frame.write.parquet(newfile)
Wrong presentation :
org.apache.parquet.schema.InvalidSchemaException: A group type can not be empty. Parquet does not support empty group without leaves. Empty group: spark_schema
I found a lot of information on the Internet , The general explanation is that dataframe Of schema Structural problems , This is not the question of this article .
after N Attempts to , Finally, the problem is .parquet(newfile). Code program query , Found this parquet Method called parquet There are two packages , The first package is used by default , However, this bag is not spark-sql Under the parquet package .
then , I deleted the version with a try mentality 1.8.1 Of parquet package , Stay and spark Dependent parquet package , Rerun code , press forward to the enemy's capital .
版权声明
本文为[A hundred nights]所创,转载请带上原文链接,感谢
https://yzsam.com/2022/04/202204231401520905.html
边栏推荐
- 杂文 谈谈古典的《拆掉思维里的墙》
- Execution plan calculation for different time types
- PHP 零基础入门笔记(13):数组相关函数
- Sail soft segmentation solution: take only one character (required field) of a string
- PyMySQL
- 05 Lua control structure
- Use case labeling mechanism of robot framework
- Detailed explanation of gzip and gunzip decompression parameters
- English | day15, 16 x sentence true research daily sentence (clause disconnection, modification)
- Custom implementation of Baidu image recognition (instead of aipocr)
猜你喜欢
磁盘管理与文件系统
Gartner 发布新兴技术研究:深入洞悉元宇宙
深度学习100例 | 第41天-卷积神经网络(CNN):UrbanSound8K音频分类(语音识别)
PyMySQL
NVIDIA graphics card driver error
英语 | Day15、16 x 句句真研每日一句(从句断开、修饰)
Sail soft implements a radio button, which can uniformly set the selection status of other radio buttons
DDT + Excel for interface test
JMeter setting environment variable supports direct startup by entering JMeter in any terminal directory
Dlib of face recognition framework
随机推荐
NVIDIA graphics card driver error
Introduction to how to set up LAN
聊一聊浏览器缓存控制
TypeError: set_figure_params() got an unexpected keyword argument ‘figsize‘
Zhongang Mining: Fluorite Flotation Process
Modify the test case name generated by DDT
NVIDIA显卡驱动报错
The font of the soft cell changes color
Derivation of Σ GL perspective projection matrix
◰GL-阴影贴图核心步骤
欣旺达:HEV和BEV超快充拳头产品大规模出货
Detailed explanation of gzip and gunzip decompression parameters
Custom implementation of Baidu image recognition (instead of aipocr)
Detailed explanation of UWA pipeline function | visual configuration automatic test
面试百分百问到的进程,你究竟了解多少
博士申请 | 厦门大学信息学院郭诗辉老师团队招收全奖博士/博后/实习生
Kunteng full duplex digital wireless transceiver chip kt1605 / kt1606 / kt1607 / kt1608 is suitable for interphone scheme
Deepinv20 installation MariaDB
How much do you know about the process of the interview
Use case labeling mechanism of robot framework