当前位置:网站首页>Sqoop imports data from Mysql to HDFS using lzop compression format and reports NullPointerException
Sqoop imports data from Mysql to HDFS using lzop compression format and reports NullPointerException
2022-04-23 20:11:00 【My brother is not strong enough to fight】
The specific errors are as follows :
Error: java.lang.NullPointerException
at com.hadoop.mapreduce.LzoSplitRecordReader.initialize(LzoSplitRecordReader.java:63)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:560)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:798)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
2021-12-21 11:14:28,253 INFO mapreduce.Job: Task Id : attempt_1639967851440_0006_m_000000_1, Status : FAILED
Error: java.lang.NullPointerException
at com.hadoop.mapreduce.LzoSplitRecordReader.initialize(LzoSplitRecordReader.java:63)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:560)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:798)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
2021-12-21 11:14:31,265 INFO mapreduce.Job: Task Id : attempt_1639967851440_0006_m_000000_2, Status : FAILED
Error: java.lang.NullPointerException
at com.hadoop.mapreduce.LzoSplitRecordReader.initialize(LzoSplitRecordReader.java:63)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:560)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:798)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
2021-12-21 11:14:36,283 INFO mapreduce.Job: map 100% reduce 0%
2021-12-21 11:14:37,292 INFO mapreduce.Job: Job job_1639967851440_0006 failed with state FAILED due to: Task failed task_1639967851440_0006_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0
2021-12-21 11:14:37,344 INFO mapreduce.Job: Counters: 9
Job Counters
Failed map tasks=4
Launched map tasks=4
Other local map tasks=3
Data-local map tasks=1
Total time spent by all maps in occupied slots (ms)=11344
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=5672
Total vcore-milliseconds taken by all map tasks=5672
Total megabyte-milliseconds taken by all map tasks=5808128
2021-12-21 11:14:37,345 ERROR lzo.DistributedLzoIndexer: DistributedIndexer job job_1639967851440_0006 failed.
resolvent :
stay core-site.xml Add configuration support LZO Compression configuration is enough .
<configuration>
<property>
<name>io.compression.codecs</name>
<value>
org.apache.hadoop.io.compress.GzipCodec,
org.apache.hadoop.io.compress.DefaultCodec,
org.apache.hadoop.io.compress.BZip2Codec,
org.apache.hadoop.io.compress.SnappyCodec,
com.hadoop.compression.lzo.LzoCodec,
com.hadoop.compression.lzo.LzopCodec
</value>
</property>
<property>
<name>io.compression.codec.lzo.class</name>
<value>com.hadoop.compression.lzo.LzoCodec</value>
</property>
</configuration>
版权声明
本文为[My brother is not strong enough to fight]所创,转载请带上原文链接,感谢
https://yzsam.com/2022/04/202204210556214387.html
边栏推荐
- How about Bohai futures. Is it safe to open futures accounts?
- 网络通信基础(局域网、广域网、IP地址、端口号、协议、封装、分用)
- Building googlenet neural network based on pytorch for flower recognition
- Mfcc: Mel frequency cepstrum coefficient calculation of perceived frequency and actual frequency conversion
- Mysql database - single table query (II)
- Command - sudo
- 【文本分类案例】(4) RNN、LSTM 电影评价倾向分类,附TensorFlow完整代码
- C语言的十六进制printf为何输出有时候输出带0xFF有时没有
- Unity general steps for creating a hyper realistic 3D scene
- How to do product innovation—— Exploration of product innovation methodology I
猜你喜欢
[text classification cases] (4) RNN and LSTM film evaluation Tendency Classification, with tensorflow complete code attached
Leetcode dynamic planning training camp (1-5 days)
aqs的学习
[webrtc] add x264 encoder for CEF / Chromium
【webrtc】Add x264 encoder for CEF/Chromium
深度分析数据恢复原理——那些数据可以恢复那些不可以数据恢复软件
Grafana shares links with variable parameters
[numerical prediction case] (3) LSTM time series electricity quantity prediction, with tensorflow complete code attached
Deep learning -- Summary of Feature Engineering
Project training of Software College of Shandong University - Innovation Training - network security shooting range experimental platform (VII)
随机推荐
Software College of Shandong University Project Training - Innovation Training - network security shooting range experimental platform (8)
NC basic usage 4
aqs的学习
MySQL数据库 - 数据库和表的基本操作(二)
VeraCrypt文件硬盘加密使用教程
Kubernetes introduction to mastery - ktconnect (full name: kubernetes toolkit connect) is a small tool based on kubernetes environment to improve the efficiency of local test joint debugging.
Command - sudo
selenium.common.exceptions.WebDriverException: Message: ‘chromedriver‘ executable needs to be in PAT
Design of library management database system
IIS数据转换问题16bit转24bit
NC basic usage
Kubernetes getting started to proficient - install openelb on kubernetes
The textarea cursor cannot be controlled by the keyboard due to antd dropdown + modal + textarea
PCL点云处理之直线与平面的交点计算(五十三)
nc基础用法4
nc基础用法1
[H264] hevc H264 parsing and frame rate setting of the old version of libvlc
波场DAO新物种下场,USDD如何破局稳定币市场?
[numerical prediction case] (3) LSTM time series electricity quantity prediction, with tensorflow complete code attached
MySQL数据库 - 单表查询(一)