# Adaptive Beauty Effects for Pinch-to-Zoom in Flutter Video Social Apps In video social applications, users often use the two-finger pinch gesture to zoom in and out on their video feed, whether for close-up facial shots or relaxed half-body framing. Behind this intuitive interaction lies a complex technical challenge: when the viewport is scaled, facial beautification effects such as face slimming and eye enlargement — which are tightly bound to facial features — must be adjusted synchronously. Otherwise, visual artifacts or malfunction of beauty filters will occur. Implementing this feature on the Flutter framework requires a well-designed collaborative solution. ## Challenges of Beauty Adaptation for Gesture Zoom Gesture zoom essentially changes the framing of camera footage. Technically, it is more than simple image scaling and triggers a chain of changes across three dimensions. First, the coordinate system is completely altered. The LetMagic Beauty SDK, especially algorithms relying on facial landmark detection, performs all calculations based on an absolute pixel coordinate system aligned with the original image. When users zoom in to focus on the central area of the original frame, every pixel displayed on the screen corresponds to a shifted position in the raw image. Without awareness of this change, the SDK will still apply effects according to the original global coordinates. For example, a reshaping effect designed for the chin may be mistakenly rendered on the cheeks or even the background, resulting in distorted visuals. Second, the effective range of face detection changes dynamically. In close-up mode, the face occupies most of the frame with fine details fully visible. In wide-shot mode, the face becomes smaller and more background content appears. Beauty algorithms must adapt to variations in target size and detection precision to ensure smooth transition of effect intensity. Fixed parameters will lead to over-intense and unnatural results for close-ups, or overly subtle effects that are barely noticeable in wide shots. Third, there is a trade-off between performance and resource consumption. Full-frame beautification on high-resolution images involves heavy computation. When zoomed in for close-ups, high-precision processing is only required for the facial area; applying identical computing power to the largely shrunk background is a waste of resources. The system needs to allocate computing resources dynamically to guarantee quality for key regions while optimizing overall performance. ## Building an Adaptive Beauty Pipeline for Scaled Viewports To address the above challenges, we have developed an end-to-end adaptive solution covering interaction capture through rendering execution. Its core is enabling the beauty pipeline to perceive and respond to real-time zoom status. The entire workflow starts with accurate gesture recognition. On the Flutter layer, we listen to two-finger gesture events on the video preview component to calculate the current zoom ratio and view offset in real time. These data are encapsulated into a **view transformation matrix**, which precisely defines the mapping from the original camera frame to the content displayed on screen. Information synchronization is the critical next step. The transformation matrix is instantly transmitted to the underlying native code responsible for beauty processing via efficient communication channels between Flutter and native plugins. Equipped with this view mapping information, the native beauty engine works with full awareness of the current display status. After receiving each raw camera frame, the SDK conducts preprocessing using the transformation matrix. It calculates the actually visible region under the current zoom level and dynamically focuses face detection and effect rendering on this area. Real-time coordinate correction is equally important. After computing beautification offsets — for instance, shifting a facial landmark inward by 10 pixels — the SDK reversely converts the absolute displacement based on the original image into relative displacement matching the scaled viewport using the transformation matrix. The corrected instructions are then delivered to the graphics pipeline for rendering. This ensures effects like face slimming always align with the jawline and eye enlargement stays centered around the eye area, regardless of zoom level. ## Seamless Integration with Audio & Video Services The adaptively processed video frames require smooth output for two destinations. One stream is rendered locally via Flutter texture components for real-time preview. The other, more critical stream is sent to the Tencent Cloud Audio & Video SDK for encoding and network transmission. We adopt a dedicated integration architecture: the entire beauty module including zoom adaptation logic is packaged as a **custom video pre-processing plugin** for the audio and video SDK. Raw frames captured by the camera first go through our plugin for zoom-aware beautification. Processed frames are then passed to the audio & video SDK for compression and transmission. Remote users will always receive stable, properly enhanced video streams with consistent visual quality. Performance optimization runs through the whole implementation. We dynamically adjust the internal processing resolution according to the zoom ratio. High-precision processing is enabled for close-up shots, while computing load for non-critical areas is moderately reduced in wide-shot mode, maintaining a steady frame rate across all scenarios. ## Delivering Smooth and Natural User Experience All technical optimizations ultimately aim to deliver a seamless user experience. When users zoom the viewport with gestures, they only feel flexible perspective switching, while facial beauty effects remain consistent, natural and fully synchronized. Such invisible technical support creates outstanding user experience.