AI redefines light perception, or a new direction for mobile phones

Every iteration of mobile communication technology is accompanied by a new round of reshuffle in the mobile phone market.

Such a view was so popular at the end of 2019 that all mobile phone manufacturers began to be cautious, for fear of missing the wave of 5G and becoming the obsolete product of the times.

However, the end result seems to have exceeded many expectations. According to relevant data released by the China Academy of Information and Communications Technology, the domestic mobile phone market shipments in the first 11 months of 2020 were 281 million units, a year-on-year decrease of 21.5%, of which domestic brands shipped 249 million units. A year-on-year decrease of 23.6%.

Although 5G mobile phones have accounted for 68.1% of the shipments in November, it cannot hide a basic fact: in 2020, when 5G mobile phones are fully popularized, the industry has not waited for a new round of phone replacement. . There may be external environmental factors, but the sluggish market demand is already a norm.

There are many reasons that can be found. For example, the operator’s 5G base station deployment is too slow, and has not yet crossed the critical point of user replacement needs; another example is the lack of killer applications that match 5G, and it is only a scene where there is nothing to measure a speed, I am afraid it is difficult to open consumers’ pockets… …

There is no lack of rationality for such explanations, but back to the smartphone industry itself, in addition to external stimuli such as 5G, if you want to climb the hills again in terms of sales, you also need an “inner loop” in user experience.

01 Waiting for a “chemical reaction”

If you want to uncover the reasons why the sales volume of the mobile phone market continues to be sluggish, the figures reflect only the appearance, and the deep-seated reasons hidden under the appearance are precisely the strange circle of innovation that the smartphone industry has fallen into.

Looking back at the evolution of smartphones in the past 10 years, behind the seemingly dazzling innovations, there are no more than two paths:

One is the upgrade of hardware parameters. The screen-to-body ratio of smartphones is getting higher and higher, processors are getting stronger and stronger, and the number of cameras is increasing… so that some mobile phone manufacturers equate the accumulation of hardware with innovation. The essence of ten-year evolution is Constantly doing additions on the hardware.

The second is the centralization of the application ecology. Compared with the relatively scattered application ecology in 2010, mainstream mobile phone manufacturers have built a set of centralized application ecology, and gradually extended from smartphones to VR, smart home and other scenarios, and even began to lay out content.

The magic scene is also here. For a long time, the two paths have been advanced almost in parallel. Most of the matching of services and hardware is focused on “physical upgrades”, and there is rarely a “chemical reaction” that gets through the bottom layer of hardware. Perhaps it also provides a direction for the next disruptive innovation of smartphones, that is, a “chemical reaction” that fully integrates hardware and services, and the most likely breaking point is the light perception system.

According to the data given by iResearch in the “2020 China Artificial Intelligence Mobile Phone White Paper”: the light perception system of a smartphone is mainly composed of chips, screens, algorithms and cameras. At present, the light perception system accounts for the value of the key components of smartphones. As high as 46.6%. At the same time, NPU has been integrated into the SoC of smartphones one after another, and multi-core GPU has gradually become the standard configuration of smartphones.

Smartphone manufacturers with a keen sense of smell seem to have insight into such a trend. The most direct example is that the space for the “photographing” link has continued to grow at the launch of the flagship new phone in the past two years.

Performance, craftsmanship, design, etc. originally occupied more than half of the time, and more and more mobile phone manufacturers have taken over them. Instead, the camera capabilities of mobile phones, from the introduction of camera parameters, to the optimization of beauty algorithms, to inviting famous photographers The teacher’s presentation took up almost one-third of the entire press conference. Features such as optical image stabilization, periscope lenses, and super night shots have gradually replaced performance and design as the core selling point of smartphones.

However, the current limitation of the light perception system is also an indisputable fact. The traditional tuning is only a simple parameter adjustment of the optical system, the perception system and the computing system. For example, the communication between the camera and the processor is carried out through fixed parameters. The parameter optimization algorithm based on the aperture size can improve the night shooting capability of the mobile phone. There is still a long way to go for the co-evolution of the optical system, the perception system and the computing system.

02 They are planning ahead

If the stimulation of 5G to the mobile phone industry is revolutionary, the impact of artificial intelligence is a gradual process, affecting the rhythm of industry evolution in a subtle way.

At present, AI’s empowerment of light perception systems is mainly concentrated in the two fields of biometrics and computational photography, such as combining computer vision, computer graphics, image processing and other technologies, with the optical system and sensing system of mobile phones, through image recognition, fusion, Enhancement, segmentation, etc., optimize the shooting and imaging capabilities of mobile phones, and break the limitations of mobile phones in optical processing and imaging by “combining software and hardware”.

To be precise, a co-evolution of light perception led by artificial intelligence technology is underway, and it has become the consensus of the entire smartphone industry chain.

In the upstream of the light perception industry chain, in addition to traditional lens production, Sunny Optical is also entering the fields of shooting modules and optical prisms. Among them, Sunny Optical has been able to achieve fixed focus, Autofocus, fast zoom, optical zoom and other technologies are used to improve the efficiency and quality of mobile phone imaging; another company with 3D sensing technology as the core, Aobi Zhongguang has independently developed a variety of 3D cameras, and has covered OPPO, Xiaomi, Lenovo, Meizu and other mobile phone manufacturers.

In the middle of the light perception industry chain, Megvii proposed the concept of redefining the light perception system with artificial intelligence, and introduced the concept of “using algorithms to learn the imaging characteristics of the camera, and then restoring the original detailed texture of the scene to improve the overall image quality” “Super Graphics Technology. Based on the capabilities of the Megvii AI productivity platform Brain++, the super image quality technology has been iterated to version 3.0 within two years. According to the information released by iResearch in the white paper, 70% of the top 10 domestic mobile phones on the DXOMark rankings of the camera evaluation agency have applied super-picture quality technology.

Downstream of the light perception industry chain, Huawei started to use RYYB array CMOS from the P30 series, replacing the green pixels of the traditional sensor filter with yellow super pixels, which increased the light input by about 40%, and then passed the ISP hardware and algorithm. The joint optimization processes the color information, which greatly improves the night shooting capability of the mobile phone. During this process, Huawei began to adjust the underlying architecture of the light perception system, realizing the co-evolution of light and sense to a certain extent, and tried to migrate the product focus from the hardware to the algorithm side.

Optical perception system is still a typical buyer’s market, and mobile phone manufacturers occupy an absolute dominant position. Usually, after the mobile phone manufacturer determines the detailed requirements for the light sensing system, the module factory will package the components according to the designated supplier or standard, and then deliver it to the mobile phone manufacturer uniformly.

When Huawei and other leading players have begun to explore the co-evolution of light perception, mid- and upstream companies such as Megvii Technology and Sunny Optical have already made arrangements in advance, and with a high probability, they can guess the next evolutionary direction of smartphones.

03 Who is left and right in the direction of the tide

In fact, no matter whether the smartphone market is in the trough or the climax, it is the users who ultimately decide the direction of the tide. Reflected in the ups and downs of sales, consumers are ultimately inseparable from voting with their feet.

Referring to the data given by QuestMobile, a third-party data agency, in the “2020 China Mobile Internet Autumn Report”, in the time period from September 2019 to September 2020, the proportion of users of Toutiao and Kuaishou systems showed a significant proportion of usage time. The growth trend of the Toutiao department increased from 12% last year to 15.4%, and the Kuaishou department increased from 4.4% last year to 7.2%, and the chips for the growth of user time are the two “short video + live broadcast”. ” product form.

iResearch has also disclosed the data of the UserTracker platform to the outside world: in the first half of 2020, short video has become the most commonly used mobile APP for consumers, with a single-machine single-day effective time accounting for as high as 31.9%, much higher than online reading. and news.

According to changes in user behavior, a reverse derivation can be made: when users leave more and more time for short video and live broadcast applications, it means that taking pictures has become the most concerned purchasing factor for consumers, and smartphone manufacturers need to continuously improve their products. shooting ability and experience.

The implication is that the upgrade of the light perception system of smart phones just hits the needs and preferences of current consumers, and it is also an inevitable choice for the mobile phone market to get out of the bottleneck period.

As for whether this logic works, some third-party cases are needed to verify:

In 2019, Zhongguancun Online launched a survey on the purchase intention of smartphone consumers. The results showed that the proportion of photos was as high as 29.2%, and the proportion of processors was only 17.3%. The screen, the attention of consumers has been less than 10%;

According to the data disclosed in the “2020 China Artificial Intelligence Mobile Phone White Paper”, as of 2019, the coverage rate of AI shooting algorithms in domestic Android mobile phones has reached 91.6%. About 75% of the high-end Android mobile phone series of domestic mainstream mobile phone manufacturers have applied or cooperated with Megvii AI algorithm;

On the basis of RYYB array and algorithm adjustment, Huawei P30 series is promoted with “night vision device”, and “night scene” is used as a differentiated selling point in the mobile phone market with serious homogeneity. According to Huawei’s official data at the 2019 China International Import Expo, the P30 series shipped more than 20 million units in eight months…

It seems that a conclusion can be drawn: Compared with the 5G dividend that has not yet won the recognition of users, the optical sensing system has already been verified by the market. Whether it can gain a dominant position in the competition of optical sensing systems will be directly related to mobile phone manufacturers. stage of product competitiveness.

To put it in a cliché, all smartphone technologies are worth redoing with artificial intelligence. If the light perception system that was originally optimized independently can be integrated with software and hardware with the help of algorithms, there will still be more on the track of computational photography. Lots of room to dig deeper.

04 Write at the end

The lack of innovation in smartphones is no longer a new topic.

The difference is that in the past, when thinking about this issue, most of them pinned their hopes on explosive new technologies, but when 5G has failed to rewrite the current situation of the mobile phone market, this road is almost unworkable.

It is precisely the gradual innovation of artificial intelligence in the light perception system that makes more and more mobile phone manufacturers realize that, rather than passively waiting for explosive technologies, perhaps they should re-disassemble the user value of smartphones like “paodingjie cattle”. Find certainty in uncertainty.