当前位置：网站首页>August 14, 2020: what are the execution engines for data tasks?

August 14, 2020: what are the execution engines for data tasks?

2020-11-06 21:55:00 【Fuda Dajia architect's daily question】

5 Line code , Can let wechat small program on the shelf to their own APP in | Register and send it to Dajiang 、 Huawei 、 Cherry keyboard ！>>>

Fogo's answer 2020-08-14：

Know the answer

The most basic thing is crontab Timing task .
Considering the distributed partition timing task and high availability , It can be used elastic-job Such a framework . But such a framework cannot handle complex task dependencies .
Big data processing frameworks generally use DAG（ Directed acyclic graph ） Workflow scheduling system , adopt DAG The graph describes the complex dependencies between tasks . stay Spark In job scheduling system , Is based on DAG describe rdd Dependencies between .

版权声明
本文为[Fuda Dajia architect's daily question]所创，转载请带上原文链接，感谢

边栏推荐

猜你喜欢

随机推荐