Data Lake simple record

Simply record some pictures , Make an impression on yourself , Do not do a lot of specific interpretation

Data Lake

Concept

 Data lake is a design idea 

Data Lake reading mode / Data warehouse write mode

There are three ways to realize data Lake

Comparison between data lake and data warehouse

advantage

Design principles

1  Separation of data and business ： Only consider data , Not related to business 
2  Separation of storage and computing ： Tradition hadoop colony , Storage and computing are generally a cluster node , Computing expansion generally means storage expansion . So it's best to design separation 
	 Amazon, for example aws For storage only , It has nothing to do with all computing nodes 
3 Lambda framework  vs Kappa framework  vs Iota framework 
5  The importance of managing services and choosing the right tools 

Lambda/Kappa framework

lambda above

Kappa The core idea of Architecture ：

1  Simplified deleted Lambda Offline batch double counting , Only through real-time calculation 
2  Save the data source for a few more days , Recalculate historical data only when necessary , Recalculate with the same code using the saved data source 

Practical application of data Lake Delta Lake

 You can continue to look at it in detail when you use it in the future