当前位置:网站首页>The spark operator - repartition operator
The spark operator - repartition operator
2022-08-05 06:11:00 【zdaiqing】
Source code
def repartition(numPartitions: Int)(implicit ord: Ordering[T] = null): RDD[T] = withScope {coalesce(numPartitions, shuffle = true)}- The bottom layer calls coalesce() to implement the repartitioning operation
- Mandatory shuffle operation
Repartition and coalesce relationship
- All operators are used to reset the number of partitions
- The bottom layer of repartition calls coalesce() to implement the repartition function
- repartition forces a shuffle operation
- coalesce decides whether to perform the shuffle operation according to the parameters
References
边栏推荐
- Unity huatuo 革命性热更系列1.2 huatuo热更环境安装与示例项目
- 【3D模型教程】ZBrush如何表现皮肤纹理?
- URP渲染管线实战教程系列 之URP渲染管线实战解密(一)
- SSL证书提示过期或者无效,该怎么处理呢?
- 每日一题-二分法
- Remembering my first CCF-A conference paper | After six rejections, my paper is finally accepted, yay!
- 入门文档03 区分开发与生产环境(生产环境才执行‘热更新’)
- Cocos Creator开发中的事件响应
- Image compression failure problem
- spark源码-任务提交流程之-1-sparkSubmit
猜你喜欢

IP数据包格式(ICMP协议与ARP协议)

Image compression failure problem

Getting Started 11 Automatically add version numbers

链表章6道easy总结(leetcode)

Getting Started Document 01 series in order

入门文档04 一个任务依赖另外一个任务时,需要按顺序执行

【Day8】使用LVM扩容所涉及的命令
![[Day1] (Super detailed steps) Build a soft RAID disk array](/img/40/cda8e5522c2795e03c0d47e8a689f8.png)
[Day1] (Super detailed steps) Build a soft RAID disk array

【UiPath2022+C#】UiPath 练习和解决方案-变量、数据类型和控制流程

入门文档09 独立的watch
随机推荐
快问快答—腾讯云服务器常见问题解答
每日一题-有效的括号-0719
spark源码-任务提交流程之-2-YarnClusterApplication
spark源码-RPC通信机制
每日一题-正则表达式匹配-0715
Getting Started Documentation 12 webserve + Hot Updates
腾讯云云函数SCF—入门须知
D39_向量
“元宇宙”是个啥?都有哪些大招?
每日一题-最长回文子串-0714
【Day8】磁盘及磁盘的分区有关知识
dsf5.0 弹框点确定没有返回值的问题
OpenCV3.0 兼容VS2010与VS2013的问题
Blender软件介绍与使用心得
来来来,一文让你读懂Cocos Creator如何读写JSON文件
2020年手机上最好的25种免费游戏
入门文档04 一个任务依赖另外一个任务时,需要按顺序执行
Account and Permission Management
你要找的cocos面试答案都在这里了!
腾讯内部技术:《轩辕传奇》服务器架构演变