当前位置:网站首页>org. apache. parquet. schema. InvalidSchemaException: A group type can not be empty. Parquet does not su
org. apache. parquet. schema. InvalidSchemaException: A group type can not be empty. Parquet does not su
2022-04-23 16:43:00 【A hundred nights】
1. Read cvs File data write parquet file
val spark =SparkSession.builder().appName("test").master("local").getOrCreate()
val file = "hdfs://clusters/test/demo.csv"
val frame = spark.read.option("header","true").csv(file)
frame.printSchema()
val newfile = file.split("\\.")(0)+".parquet"
frame.write.parquet(newfile)
Wrong presentation :
org.apache.parquet.schema.InvalidSchemaException: A group type can not be empty. Parquet does not support empty group without leaves. Empty group: spark_schema
I found a lot of information on the Internet , The general explanation is that dataframe Of schema Structural problems , This is not the question of this article .
after N Attempts to , Finally, the problem is .parquet(newfile). Code program query , Found this parquet Method called parquet There are two packages , The first package is used by default , However, this bag is not spark-sql Under the parquet package .
then , I deleted the version with a try mentality 1.8.1 Of parquet package , Stay and spark Dependent parquet package , Rerun code , press forward to the enemy's capital .

版权声明
本文为[A hundred nights]所创,转载请带上原文链接,感谢
https://yzsam.com/2022/04/202204231401520905.html
边栏推荐
- PyMySQL
- 磁盘管理与文件系统
- Gartner 发布新兴技术研究:深入洞悉元宇宙
- STM32__03—初识定时器
- Dlib of face recognition framework
- JSP learning 2
- Force buckle-746 Climb stairs with minimum cost
- True math problems in 1959 college entrance examination
- Loggie source code analysis source file module backbone analysis
- Database dbvisualizer Pro reported file error, resulting in data connection failure
猜你喜欢

Do you really understand the principle of code scanning login?

G008-HWY-CC-ESTOR-04 华为 Dorado V6 存储仿真器配置

Use if else to judge in sail software - use the title condition to judge

【Pygame小游戏】10年前风靡全球的手游《愤怒的小鸟》,是如何霸榜的?经典回归......
JIRA screenshot

How magical is the unsafe class used by all major frameworks?

How to choose the wireless gooseneck anchor microphone and handheld microphone scheme

Pytorch: the pit between train mode and eval mode

loggie 源码分析 source file 模块主干分析

The solution of not displaying a whole line when the total value needs to be set to 0 in sail software
随机推荐
Change the icon size of PLSQL toolbar
MySql主从复制
Solution of garbled code on idea console
建站常用软件PhpStudy V8.1图文安装教程(Windows版)超详细
On the value, breaking and harvest of NFT project
文件系统读写性能测试实战
蓝桥杯省一之路06——第十二届省赛真题第二场
ByteVCharts可视化图表库,你想要的我都有
欣旺达:HEV和BEV超快充拳头产品大规模出货
Phpstudy V8, a commonly used software for station construction 1 graphic installation tutorial (Windows version) super detailed
How does flash cache data in memory?
Introduction to how to set up LAN
Pycham connects to the remote server and realizes remote debugging
Use if else to judge in sail software - use the title condition to judge
Set the color change of interlaced lines in cells in the sail software and the font becomes larger and red when the number is greater than 100
OMNeT学习之新建工程
Real time operation of vim editor
loggie 源码分析 source file 模块主干分析
Nacos 详解,有点东西
ACL 2022 | dialogved: a pre trained implicit variable encoding decoding model for dialogue reply generation