腾讯音视频AI

7. Introduction :

Scheme introduction:

Tencent Smart Eye·Editing and Auditing is an enterprise-level solution created by Tencent Cloud to solve the pain points of "low audio and Media Processing Service efficiency, high manual review costs, and difficulty in reusing media resources", positioning itself as an "AI efficiency engine for the audio and video industry." The plan is based on the results of Tencent Cloud's many years of audio and video technology and AI research, and builds a full-process capability of "identification-review-editing-quality inspection": the AI engine automatically generates audio and video multi-dimensional tags (faces, objects, scenes, etc.) to realize rapid retrieval of media resources; Avoid illegal content through multi-dimensional review of audio and picture characters and reduce the pressure of manual review; improve content production efficiency through intelligent strip removal and collection generation; At the same time, it supports privatization deployment of sensitive industries such as radio and television to ensure data security. It has served customers such as WeChat "Take a Look", Dianduo Live Broadcast, and Future TV, and has become the core supporting tool for the digital transformation of the audio and video industry.

Core functions and scheme features:

1. Core functional modules

1. AI-driven intelligent recognition capabilities

   • Full-dimensional content tagging : Integrate a million face database with 9 categories of massive video tags (People, scenery, food, buildings, animals and plants, scenes, etc.), support Face Recognition (Locate the frame where the person is located, customize the face database, store face vector data for secondary retrieval), speech recognition (Extract audio text information), Optical Character Recognition (identify subtitles and watermarked text in the picture), frame label recognition (label picture elements frame by frame), video label recognition (generate overall video theme tags), realizing the "structured dismantling" of audio and video content. For example, the radio and television media database can quickly locate specific characters or scenes in historical news materials through tags.
   • Intelligent retrieval and media resource reuse : To build a retrieval system based on tags generated by identification, editors can quickly find target materials through keywords (such as "a host" and "press conference scene"), solving the problem of "relying on manual labeling and low retrieval efficiency" in traditional media resources. A typical case is Yunfan Digital Link using this ability to build an image archive for Beijing TV to improve the utilization rate of archived media resources.

2. Multi-dimensional intelligent review of sound, picture and character

   • Full-scene compliance testing : Supports multi-dimensional review of live and on-demand videos, covering screens (Illegal characters, scenes, logos), audio (illegal voice, noise), text (illegal subtitles, bullet screens), and can also detect 13 video quality issues (painted screens, black edges, mosaics, noise, etc.) and provide quality scoring; Provide customized review standards for radio and television, e-commerce and other industries (such as radio and television customers 'character database standards, and detect violations of Pendoduo e-commerce live broadcasts), and support large customers to call for special services.
   • Manual review improves efficiency : Machine review filters compliance content first, and only pushes suspected violations to manual review, which greatly reduces labor costs. For example, Pinduo uses this ability to review e-commerce live broadcasts, effectively preventing violations and ensuring Brand reputation.

3. Efficient intelligent editing assistance

   • Automated content production : Support intelligent strip-breaking (automatically splitting long videos into Short Video, such as news strip-breaking), intelligent highlights (automatically extracting wonderful moments of game events and sports competitions), intelligent covers (automatically generating high-quality video cover pictures), and title and title recognition (automatically locating and clipping titles and titles) to reduce manual editing workload and improve content output efficiency.
   • Fast editing solution : Provide lightweight editing tools combined with AI recognition results to adapt to high-frequency content production scenarios such as news and events. For example, radio and television customers can quickly split long-term press conference videos into multiple themed Short Video.

4. Full process video quality inspection

   • Multi-dimensional quality detection : Conduct comprehensive detection of picture frames (painted screen, black edges, abnormal color) and sound quality (noise, abnormal volume) of live and on-demand videos, providing 13 detection types and overall quality scores to assist in screening high-quality content. For example, live broadcast platforms can eliminate low-quality anchor pictures through quality inspection.

2. Core characteristics of the program

1. AI engine integration advantages : Based on Tencent Cloud audio and video coding, deep learning technology and the latest laboratory results, it provides one-stop AI understanding and recognition services to avoid the efficiency loss of multiple tool splicing;
2. Easy-to-use access method : Provide external standard REST API interfaces that integrate public/private and live broadcast/on-demand, which can quickly connect with Tencent Video on Demand services, COS storage and third-party storage systems, and support independent scheduling and switching;
3. Flexible deployment model : Supports public cloud (on-demand call, reducing initial investment) and private deployment (local computing and storage of data, adapted to industries sensitive to data security such as radio, television, and government affairs). The two models can be selected according to customer needs.

Industry solutions and typical cases

1. Solutions for segmented industries

1. Radio and Television Privatization Solution

   • Core capabilities : Deploy smart eye capabilities on customers 'local servers to support the operation of news networks, production networks, and media assets systems, and provide full capabilities of intelligent review, identification, and editing;
   • Solution advantages : Local data storage ensures copyright and security, intranet transmission saves cloud costs, quickly connects with existing business systems, adapts to regulatory requirements of the radio and television industry, and typical customers are future TVs (customize audit standards according to national regulatory requirements to improve manual review efficiency).

2. Education solutions : For online educational videos, provide content review (to avoid illegal content), intelligent stripping (to split course chapters), and label generation (to facilitate students to retrieve knowledge points) to improve the efficiency of educational content management and dissemination.

3. Online video solution : Provide video platforms with smart tag generation (for content recommendation), quality quality inspection (to ensure playback experience), and copyright protection (digital watermark function). For example, WeChat "Take a Look" implements smart recommendation of Short Video through tag clustering and improves recommendation success rate.

4. Live broadcast solution : Provide real-time review (to prevent violations) and quality inspection (to ensure the smoothness of live broadcast images) for e-commerce and entertainment live broadcasts. Typical customers are Dianduo (review e-commerce live broadcast content and reduce labor costs).

2. Typical customer cases

Applicable groups and industries

• Radio, television and media organizations : TV stations, Internet TV (such as Future TV), and video platforms need to efficiently manage media assets, ensure content compliance, and improve content production efficiency;
• E-commerce and live broadcast platforms : Pinduoduo, live broadcast guilds, etc. need to review live broadcast content in real time to reduce the risk of violations and labor costs;
• Educational institutions : Online education platforms and universities need to manage course videos, split knowledge points, and ensure compliance with educational content;
• Enterprise content department : The marketing and training departments of large enterprises need to process brand promotion videos and training videos to achieve content reuse and efficient dissemination;
• Technical service providers : For example, media asset management system developers (such as Yunfan Digital Link) need to integrate AI identification capabilities to provide value-added services to customers.

unique advantages

1. Deep technical base : Relying on Tencent Cloud audio and video coding and decoding technology and AI research results (such as computer vision large model), the industry is leading in recognition accuracy and review coverage;
2. Closed-loop of full-process capabilities : Covering the entire link of "identity-review-editing-quality inspection", eliminating the need to integrate multi-vendor tools and reducing enterprise docking costs;
3. Strong industry customization : Provide privatization deployment and regulatory adaptation for radio and television, and provide special review of live broadcasts for e-commerce to meet the core needs of different industries;
4. Close ecological linkage : It can be seamlessly connected with Tencent Cloud Media Processing (MPS), Video on Demand (VOD), COS storage and other products to form a "storage-processing-distribution" full ecological service and improve overall operational efficiency.