当前位置:网站首页>Sqoop imports data from Mysql to HDFS using lzop compression format and reports NullPointerException
Sqoop imports data from Mysql to HDFS using lzop compression format and reports NullPointerException
2022-04-23 20:11:00 【My brother is not strong enough to fight】
The specific errors are as follows :
Error: java.lang.NullPointerException
at com.hadoop.mapreduce.LzoSplitRecordReader.initialize(LzoSplitRecordReader.java:63)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:560)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:798)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
2021-12-21 11:14:28,253 INFO mapreduce.Job: Task Id : attempt_1639967851440_0006_m_000000_1, Status : FAILED
Error: java.lang.NullPointerException
at com.hadoop.mapreduce.LzoSplitRecordReader.initialize(LzoSplitRecordReader.java:63)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:560)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:798)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
2021-12-21 11:14:31,265 INFO mapreduce.Job: Task Id : attempt_1639967851440_0006_m_000000_2, Status : FAILED
Error: java.lang.NullPointerException
at com.hadoop.mapreduce.LzoSplitRecordReader.initialize(LzoSplitRecordReader.java:63)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:560)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:798)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
2021-12-21 11:14:36,283 INFO mapreduce.Job: map 100% reduce 0%
2021-12-21 11:14:37,292 INFO mapreduce.Job: Job job_1639967851440_0006 failed with state FAILED due to: Task failed task_1639967851440_0006_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0
2021-12-21 11:14:37,344 INFO mapreduce.Job: Counters: 9
Job Counters
Failed map tasks=4
Launched map tasks=4
Other local map tasks=3
Data-local map tasks=1
Total time spent by all maps in occupied slots (ms)=11344
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=5672
Total vcore-milliseconds taken by all map tasks=5672
Total megabyte-milliseconds taken by all map tasks=5808128
2021-12-21 11:14:37,345 ERROR lzo.DistributedLzoIndexer: DistributedIndexer job job_1639967851440_0006 failed.
resolvent :
stay core-site.xml Add configuration support LZO Compression configuration is enough .
<configuration>
<property>
<name>io.compression.codecs</name>
<value>
org.apache.hadoop.io.compress.GzipCodec,
org.apache.hadoop.io.compress.DefaultCodec,
org.apache.hadoop.io.compress.BZip2Codec,
org.apache.hadoop.io.compress.SnappyCodec,
com.hadoop.compression.lzo.LzoCodec,
com.hadoop.compression.lzo.LzopCodec
</value>
</property>
<property>
<name>io.compression.codec.lzo.class</name>
<value>com.hadoop.compression.lzo.LzoCodec</value>
</property>
</configuration>
版权声明
本文为[My brother is not strong enough to fight]所创,转载请带上原文链接,感谢
https://yzsam.com/2022/04/202204210556214387.html
边栏推荐
- DTMF双音多频信号仿真演示系统
- How to create bep-20 pass on BNB chain
- R language ggplot2 visual facet_wrap, and use the lineheight parameter to customize the height of the facet icon tab (gray label bar)
- Computing the intersection of two planes in PCL point cloud processing (51)
- 渤海期货这家公司怎么样。期货开户办理安全?
- MySQL 进阶 锁 -- MySQL锁概述、MySQL锁的分类:全局锁(数据备份)、表级锁(表共享读锁、表独占写锁、元数据锁、意向锁)、行级锁(行锁、间隙锁、临键锁)
- IIS data conversion problem: 16bit to 24bit
- Possible root causes include a too low setting for -Xss and illegal cyclic inheritance dependencies
- Mfcc: Mel frequency cepstrum coefficient calculation of perceived frequency and actual frequency conversion
- MySQL数据库 - 单表查询(二)
猜你喜欢
Compact CUDA tutorial - CUDA driver API
Understanding various team patterns in scrum patterns
Openharmony open source developer growth plan, looking for new open source forces that change the world!
aqs的学习
PCL点云处理之计算两平面交线(五十一)
Esp8266 - beginner level Chapter 1
网络通信基础(局域网、广域网、IP地址、端口号、协议、封装、分用)
Notes of Tang Shu's grammar class in postgraduate entrance examination English
Deep learning -- Summary of Feature Engineering
MySQL advanced lock - overview of MySQL locks and classification of MySQL locks: global lock (data backup), table level lock (table shared read lock, table exclusive write lock, metadata lock and inte
随机推荐
nc基础用法
Project training of Software College of Shandong University - Innovation Training - network security shooting range experimental platform (6)
Cadence Orcad Capture 批量更改元件封装功能介绍图文教程及视频演示
R语言ggplot2可视化分面图(facet_wrap)、使用lineheight参数自定义设置分面图标签栏(灰色标签栏)的高度
渤海期货这家公司怎么样。期货开户办理安全?
Mysql database - single table query (I)
使用 WPAD/PAC 和 JScript在win11中进行远程代码执行
Why is the hexadecimal printf output of C language sometimes with 0xff and sometimes not
R language survival package coxph function to build Cox regression model, ggrisk package ggrisk function and two_ Scatter function visualizes the risk score map of Cox regression, interprets the risk
R语言survival包coxph函数构建cox回归模型、ggrisk包ggrisk函数和two_scatter函数可视化Cox回归的风险评分图、解读风险评分图、基于LIRI数据集(基因数据集)
R语言ggplot2可视化:ggplot2可视化散点图并使用geom_mark_ellipse函数在数据簇或数据分组的数据点周围添加椭圆进行注释
@MapperScan与@Mapper
基于pytorch搭建GoogleNet神经网络用于花类识别
Vericrypt file hard disk encryption tutorial
[target tracking] pedestrian attitude recognition based on frame difference method combined with Kalman filter, with matlab code
高效的串口循环Buffer接收处理思路及代码2
R language uses timeroc package to calculate the multi time AUC value of survival data under competitive risk, uses Cox model and adds covariates, and R language uses the plotauccurve function of time
如何在BNB链上创建BEP-20通证
[numerical prediction case] (3) LSTM time series electricity quantity prediction, with tensorflow complete code attached
Shanda Wangan shooting range experimental platform project - personal record (IV)