Wu Jiesheng: today's application architecture is at an unpredictable stage

In recent years ,“ Observability ” This word is widely used in China IT The circle suddenly became popular , Ali 、 Baidu 、 byte 、 Tencent and other large factories have followed up the observability construction , More companies based on observable technology have been established , Financing in the field of observability is hot . This one is called “ Observability ” Fire , Even from the back-end field , Extended to the big front end , Some mobile development teams want to introduce “ Observability ” The concept solves deeper application architecture problems , Improve performance and business experience . Based on this topic , With this scene “ Cloud table sect ” talk face-to-face with sb. . This article is authorized to be reproduced from InfoQ.

Talk to the guests

Wu Jiesheng ： Alibaba senior researcher , Head of Alibaba cloud intelligent storage

Huo Taiwen ： Founder and CEO,InfoQ Founder of China

“ Observability ” It's not a new word , To find out why it turned red ,InfoQ Interviewed Alibaba senior researcher 、 Wu Jiesheng, head of Alibaba cloud intelligent storage （Jason Wu）.Jason The team in charge , Some work is also being done for observability development .

This time , For observability ,Jason Talked freely about their understanding , Here are his four core ideas ：

• Point of view ： Today's Application Architecture , Is at an unpredictable stage .
• Viewpoint two ： Monitoring is after the system has observable ability , An engineering capability provided by the upper layer .
• Point three ： Observability always exists , Now it's just turning red .
• Viewpoint 4 ： The age of cloud Nativity , Products need to be observable .

Observability, Observability or Observability ？

Before talking about the present and future of observability , Let's talk about the word itself . The concept of observability comes from abroad Observability The word" , But if we open all kinds of translation software , Input “Observability”, The translation results are often “ Observability ”. therefore , Why not call “ Observability ”, But to mention “ Observability ”？ We found that , Many technical teams today mix these two words , They think there's no difference , You may even find it annoying that you are engaged in writing work , Bite the words here , I'm really full . But translation is actually a very serious and important thing , Good translation can make people understand at a glance . Take a classic counterexample , There is a word in the computer field called robustness , If you don't check the relevant information , It may be hard for you to understand what this feature is . therefore , It's choice “ Observability ” still “ Observability ”, There is still some small discussion value in this matter . If we cite existing domestic literature , You will find that these two words have related meanings .

Observability is observability 2014 Terms of general medicine and community health published by the national science and technology terminology examination and Approval Committee in .

Observability is 1990 Automatic science and technology terms published by the national science and technology terms examination and Approval Committee in .
look , Following the name of observability , It will fit better . So from a technical point of view , What's the situation ？ For this question ,Jason Expressed his views ：“ I really haven't thought about this before , But if you search , It will be found that the observable frequency is higher . I tend to be observable and personal , Why? ？ because , Many of our systems , Or say , Many application architectures , Today is actually in an unpredictable stage . Since it's unpredictable , So the corresponding , Observability may fit a little better .”

Is observability a new concept ？

In the discussion of observability , Also mixed with some critical voices ： This is actually monitoring /Monitoring It's just a makeover , But some people claim that this is a new technology , It's all a supplier conspiracy . There are two issues that need to be discussed here , First of all , Is observability a new technology ？ second , Is observability the packaging hype of monitoring ？

The connotation of observability has always existed

First , The connotation of observability always exists . If you recall carefully , You'll find large distributed systems in the past , Including parallel computing , All emphasize observability . Another example ,Linux There are many observable things in it , Open some FLAG, It will collect a lot of performance indicators or logs for you . When compiling programs , Open some compilation options , You can have a lot Debug Information about , These are all manifestations of observability .

Jason Think ,** In a sense , Observability is a capability of a system or product .** Your product and system have the basic attributes of function and performance , It also needs to be observable , It's a manifestation of ability . What benefits will this ability bring ？ You can put the software 、 product 、 service , From black box to white box , Help you better understand them , Help you find problems at a higher level 、 Location problem 、 solve the problem .

So why is observability raised on a large scale today , It has also been raised to a new height ？ Because of the development of Cloud Computing .

Containers 、 Microservices 、 The emergence of Distributed Systems , Make the system more and more complex , In this environment , Everyone's design of the current new system 、 Product design , Some new demands have been derived , And this appeal is becoming more and more obvious .

For example , today , Many of our businesses have adopted microservice architecture , There are thousands of containers , These containers are interdependent , Communication is very complicated . if , Each container does not have its own observable ability , How are you going to troubleshoot ？

Another example , Maybe your system uses a variety of technology products , Some businesses are your own , Some are Alibaba cloud products , Some are provided by third parties , These are intertwined . If your system is not observable , How are you going to troubleshoot ？

Sum up , yes Two reasons have driven the popularity of the concept of observability , One is the demand for the system from black box to white box , Second, the demand for the observability of resources to business .

The first point is easy to understand , Second, we need another example to illustrate . If you're running a web service , You find the customer experience very slow , Then you need to find out where it's slow , At this time, the front and rear ends need to check the problems . Similar to this, there are more and more opportunities to solve problems from a business perspective , The requirement of observability is higher and higher . Another example , As a business party , You use a lot of container microservices , Using a lot of cloud vendors' products , If you care about business stability , You need to know what's going on in your system , What happened in the products of those cloud manufacturers , How do these things connect . This is the observability from resources to business .

Observable flourishes in primary clouds , How it will work on cloud primitives ？Jaosn Express , It makes people realize , Whether it's a cloud native service or , Yunyuan's products are also good , All need to have the ability to observe , This is a basic requirement .“ We need to provide some auxiliary tools , To help customers understand and gain insight into the behavior of their business , So there's something like APM Some of these tools and products , It can automatically insert some code into the customer's Application , To help them do performance monitoring , Or something else .”

Observability vs monitor

The second question is , Whether observability is monitoring ？

On this question , There are several views in the industry , Some people think that monitoring is observable , Some people think there is no difference between the two, but the previous monitoring was wrong , Some people also think that monitoring is a subset of observability , Others think it's not the same thing at all .

Here we need to mention the second point , Supporters of this view say ：“ Monitoring is not data acquisition 、 Data storage and data display , That's the same as ordinary CRUD What's the difference? ？ Monitoring should have Topo Of , If a Metric Declined , Alert , You can't tell me why , Is it human change or machine failure , What's your name, surveillance ？”

Back to our question ,Jason Think , The monitoring system is after the system has the ability to be observed , The engineering capabilities we provide above . You must first be observable , To be monitored . When a software 、 After a module is observable , You can get Traces track 、Loggings journal 、Metrics Performance and other index data , And then build a monitoring engineering platform on it . Thanks to this , You may also monitor more dimensions .

The wonderful combination of observability and storage

Data is the core of observability , When it comes to data, it must be inseparable from storage . As the head of Alibaba cloud intelligent storage ,Jason How to view the relationship between storage and observability ？

Jason Take Alibaba cloud storage as an example . One side , Alibaba cloud will provide observable capabilities in system design and products , A series of functions of some products will be introduced later , To enhance the observability of Alibaba cloud products . Take storage products as an example , There will be several Lens, thus , Customers will know , Send a request to the block store or OSS after , Where and how long did this request take , What kind of a state . On the other hand , They will also help other products and customer services build their own observability , For example, provide some for Logging、Metrics、Trace Efficient storage system and analysis system , Or provide in the log service AI Analytical ability and root cause analysis ability .

The benefits of observability have been mentioned a lot before , But when it comes to data , The problem becomes very complicated . We are worried. , The more observable the system is , Whether the risk of data security compliance will be greater ？

“ I think this is a very good question . Our current thinking is in two aspects , First, when doing system design , We will consider what is sensitive information , What information can be transmitted through . Another one , When we are collecting these data , We will remove some sensitive fields , And it is equipped with security measures such as log audit to help us , These data to ensure these observability , It is reasonable and compliant .”Jason And tell us , This is an evolving process ,“ Some of the data governance work we have done , It's all about how to make the data more compliant , And the data flow is within the range that customers can expect , That is, the data cannot flow from one region to another . Data security management like this , It is the whole product design and service that should be considered .”（ The body of the finish ）

Click to read the original text of the dialogue video to watch

Alibaba cloud Wu Jiesheng ： Today's Application Architecture , Is at an unpredictable stage