当前位置:网站首页>August 14, 2020: what are the execution engines for data tasks?
August 14, 2020: what are the execution engines for data tasks?
2020-11-06 21:55:00 【Fuda Dajia architect's daily question】
Fogo's answer 2020-08-14:
The most basic thing is crontab Timing task .
Considering the distributed partition timing task and high availability , It can be used elastic-job Such a framework . But such a framework cannot handle complex task dependencies .
Big data processing frameworks generally use DAG( Directed acyclic graph ) Workflow scheduling system , adopt DAG The graph describes the complex dependencies between tasks . stay Spark In job scheduling system , Is based on DAG describe rdd Dependencies between .
版权声明
本文为[Fuda Dajia architect's daily question]所创,转载请带上原文链接,感谢
边栏推荐
- Nodejs中使用jsonwebtoken(JWT)生成token的场景使用
- Count the number of project code lines
- Understanding formatting principles
- 【涂鸦物联网足迹】涂鸦云平台全景介绍
- win7 APPCRASH(解决方法)(转)
- Can you do it with only six characters?
- Summary of front-end performance optimization that every front-end engineer should understand:
- Summary of common SQL statements
- The method of local search port number occupation in Windows system
- [byte jumps, autumn recruitment Posts open] ohayoo! Don't leave after school, I want to ask you to play games!!!
猜你喜欢
win7 APPCRASH(解决方法)(转)
Metersphere developer's Manual
Utility class functions (continuous update)
Code generator plug-in and creator preform file analysis
C calls SendMessage to refresh the taskbar icon (the icon does not disappear at the end of forcing)
Small program introduction to proficient (2): understand the four important files of small program development
ES6 learning notes (4): easy to understand the new grammar of ES6
Jenkins installation and deployment process
Introduction to Huawei cloud micro certification examination
轻量型 GPU 应用首选 京东智联云推出 NVIDIA vGPU 实例
随机推荐
How does cglib implement multiple agents?
Vue communication and cross component listening state Vue communication
An article taught you to download cool dog music using Python web crawler
JVM memory allocation - xms128m - xmx512m - XX: permsize = 128M - XX: maxpermsize = 512M
ES6 learning notes (2): teach you to play with class inheritance and class objects
应用层软件开发教父教你如何重构,资深程序员必备专业技能
Road to simple HTML + JS to achieve the most simple game Tetris
2020-08-29:进程线程的区别,除了包含关系之外的一些区别,底层详细信息?
打工人好物——磨炼钢铁意志就要这样高效的电脑
Stickinengine architecture 12 communication protocol
The 4th China BIM (digital construction) manager Summit Forum will be held in Hangzhou in 2020
Junit测试出现 empty test suite
Countdown | 2020 PostgreSQL Asia Conference - agenda arrangement of Chinese sub Forum
2020-09-09:裸写算法:两个线程轮流打印数字1-100。
Unexpected element.. required element
An article takes you to understand CSS3 picture border
消防器材RFID固定资产管理系统
2020-08-18:介绍下MR过程?
C calls SendMessage to refresh the taskbar icon (the icon does not disappear at the end of forcing)
How to play sortable JS vuedraggable to realize nested drag function of forms