当前位置:网站首页>Sqoop imports data from Mysql to HDFS using lzop compression format and reports NullPointerException
Sqoop imports data from Mysql to HDFS using lzop compression format and reports NullPointerException
2022-04-23 20:11:00 【My brother is not strong enough to fight】
The specific errors are as follows :
Error: java.lang.NullPointerException
at com.hadoop.mapreduce.LzoSplitRecordReader.initialize(LzoSplitRecordReader.java:63)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:560)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:798)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
2021-12-21 11:14:28,253 INFO mapreduce.Job: Task Id : attempt_1639967851440_0006_m_000000_1, Status : FAILED
Error: java.lang.NullPointerException
at com.hadoop.mapreduce.LzoSplitRecordReader.initialize(LzoSplitRecordReader.java:63)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:560)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:798)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
2021-12-21 11:14:31,265 INFO mapreduce.Job: Task Id : attempt_1639967851440_0006_m_000000_2, Status : FAILED
Error: java.lang.NullPointerException
at com.hadoop.mapreduce.LzoSplitRecordReader.initialize(LzoSplitRecordReader.java:63)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:560)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:798)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
2021-12-21 11:14:36,283 INFO mapreduce.Job: map 100% reduce 0%
2021-12-21 11:14:37,292 INFO mapreduce.Job: Job job_1639967851440_0006 failed with state FAILED due to: Task failed task_1639967851440_0006_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0
2021-12-21 11:14:37,344 INFO mapreduce.Job: Counters: 9
Job Counters
Failed map tasks=4
Launched map tasks=4
Other local map tasks=3
Data-local map tasks=1
Total time spent by all maps in occupied slots (ms)=11344
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=5672
Total vcore-milliseconds taken by all map tasks=5672
Total megabyte-milliseconds taken by all map tasks=5808128
2021-12-21 11:14:37,345 ERROR lzo.DistributedLzoIndexer: DistributedIndexer job job_1639967851440_0006 failed.
resolvent :
stay core-site.xml Add configuration support LZO Compression configuration is enough .
<configuration>
<property>
<name>io.compression.codecs</name>
<value>
org.apache.hadoop.io.compress.GzipCodec,
org.apache.hadoop.io.compress.DefaultCodec,
org.apache.hadoop.io.compress.BZip2Codec,
org.apache.hadoop.io.compress.SnappyCodec,
com.hadoop.compression.lzo.LzoCodec,
com.hadoop.compression.lzo.LzopCodec
</value>
</property>
<property>
<name>io.compression.codec.lzo.class</name>
<value>com.hadoop.compression.lzo.LzoCodec</value>
</property>
</configuration>
版权声明
本文为[My brother is not strong enough to fight]所创,转载请带上原文链接,感谢
https://yzsam.com/2022/04/202204210556214387.html
边栏推荐
- [numerical prediction case] (3) LSTM time series electricity quantity prediction, with tensorflow complete code attached
- DTMF dual tone multi frequency signal simulation demonstration system
- PCL点云处理之基于PCA的几何形状特征计算(五十二)
- R语言使用econocharts包创建微观经济或宏观经济图、indifference函数可视化无差异曲线、自定义计算交叉点、自定义配置indifference函数的参数丰富可视化效果
- Mysql database - single table query (III)
- Computing the intersection of two planes in PCL point cloud processing (51)
- 【webrtc】Add x264 encoder for CEF/Chromium
- Project training of Software College of Shandong University - Innovation Training - network security shooting range experimental platform (VII)
- CVPR 2022 | QueryDet:使用级联稀疏query加速高分辨率下的小目标检测
- [text classification cases] (4) RNN and LSTM film evaluation Tendency Classification, with tensorflow complete code attached
猜你喜欢
Redis cache penetration, cache breakdown, cache avalanche
STM32 Basics
Kubernetes entry to mastery - bare metal loadbalance 80 443 port exposure precautions
aqs的学习
山东大学软件学院项目实训-创新实训-网络安全靶场实验平台(七)
Comment créer un pass BEP - 20 sur la chaîne BNB
山东大学软件学院项目实训-创新实训-网络安全靶场实验平台(八)
Possible root causes include a too low setting for -Xss and illegal cyclic inheritance dependencies
Building googlenet neural network based on pytorch for flower recognition
[numerical prediction case] (3) LSTM time series electricity quantity prediction, with tensorflow complete code attached
随机推荐
程序设计语言基础(2)
渤海期货这家公司怎么样。期货开户办理安全?
Fundamentals of programming language (2)
Still using listview? Use animatedlist to make list elements move
C语言的十六进制printf为何输出有时候输出带0xFF有时没有
Database query - course selection system
山东大学软件学院项目实训-创新实训-网络安全靶场实验平台(八)
R language ggplot2 visualization: ggplot2 visualizes the scatter diagram and uses geom_ mark_ The ellipse function adds ellipses around data points of data clusters or data groups for annotation
nc基础用法1
selenium.common.exceptions.WebDriverException: Message: ‘chromedriver‘ executable needs to be in PAT
如何在BNB链上创建BEP-20通证
Kubernetes introduction to mastery - ktconnect (full name: kubernetes toolkit connect) is a small tool based on kubernetes environment to improve the efficiency of local test joint debugging.
STM32 Basics
Physical meaning of FFT: 1024 point FFT is 1024 real numbers. The actual input to FFT is 1024 complex numbers (imaginary part is 0), and the output is also 1024 complex numbers. The effective data is
Electron入门教程4 —— 切换应用的主题
山东大学软件学院项目实训-创新实训-网络安全靶场实验平台(六)
DTMF双音多频信号仿真演示系统
[2022] regard 3D target detection as sequence prediction - point2seq: detecting 3D objects as sequences
高效的串口循环Buffer接收处理思路及代码2
LeetCode异或运算