现有 App 大部分业务场景都是以长列表呈现,为更好满足用户内容分享的诉求,Android 各大厂商都在系统层面提供十分便捷的长截屏能力。然而我们发现 Flutter 长列表页面在部分 Android 手机上无法截长屏,Flutter 官方和社区也没有提供框架层面的长截屏能力。闲鱼作为 Flutter 在国内业务落地的代表作,大部分页面都以 Flutter 承接。为了闲鱼用户也能享受厂商系统的长截屏能力,更好的满足商品、社区内容分享的诉求,闲鱼技术团队主动做了分析和适配。
针对线上舆情做了统计分析,发现小米用户舆情反馈量占比最多,其次少量是华为用户。为此我们针对 Miui 长截屏功能做了适配。
小米长截屏解读这里华为、OPPO、VIVO 基于无障碍服务实现,长截屏功能已经适配 Flutter 页面。这里少量用户反馈,是因为截屏反馈小把手 PopupWindow 有可能出现遮挡,导致系统无法驱动长列表滚动。通过重写 isImportantForAccessibility 便能解决。
操作和表现
小米手机可通过音量键 电源键、或顶部下拉功能菜单“截屏”,触发截屏。经过简单尝试,可以发现,原生长列表页面支持截长屏,原生页面无长列表不支持,闲鱼 Flutter 长列表页面(如详情页、搜索结果页)不支持。
点击“截长屏”后,能看到长列表页面会自动滚动,点击结束或者触底的时候,自动打开图片编辑页面,能看到生成的长截图。那小米系统是如何无侵入的实现以下关键点:
-
1. 当前页面是否支持滚动截屏(长截屏 按钮是否置灰)
-
2. 如何触发 App 长列表页面滚动
-
3. 如何判断是否已经滚动触底
-
4. 如何合成长截图
系统源码获取
小米厂商能判断前台 App 页面能否滚动,必然需要调用前台 App 视图的关键接口来获取信息。编写一个自定义 RecyclerView 列表页面,日志输出 RecycleView 方法调用:
已知长截屏需要调用的方法,再查看堆栈,可以看到调用方是系统类:miui.util.LongScreenshotUtils&ContentPort
使用低版本 miui(这里 miui8)手机,获取对应的代码:/system/framework/framework.jar 或 github 查找 miui 开放代码。
实现原理介绍
整体流程:查找滚动视图 → 驱动视图滚动 → 分段截图→截图内容合并
查找滚动视图
其中检查条件:
-
1. View visibility == View.VISIBLE
-
2. canScrollVertically(1) == true
-
3. View 在屏幕内的宽度 > 屏幕宽度/3
-
4. View 在屏幕内的高度 > 屏幕高度/2
触发视图滚动
-
1. 每次滚动前,使用 canScrollVertically(1) 判断是否向下滚动
-
2. 触发滚动逻辑
-
a. 特殊视图: dispatchFakeTouchEvent(2);
private boolean checkNeedFakeTouchForScroll {if ((this.mMainScrollView instanceof AbsListView) ||(this.mMainScrollView instanceof ScrollView) ||isRecyclerView(this.mMainScrollView.getClass) ||isNestedScrollView(this.mMainScrollView.getClass)) {return false;}return !(this.mMainScrollView instanceof AbsoluteLayout) ||(Build.VERSION.SDK_INT > 19 &&!"com.ucmobile".equalsIgnoreCase(this.mMainScrollView.getContext.getPackageName) &&!"com.eg.android.AlipayGphone".equalsIgnoreCase(this.mMainScrollView.getContext.getPackageName));}
-
b. AbsListView: scrollListBy(distance);
-
c. 其他:view.scrollBy(0, distance);
-
3. 滚动结束,对比 scrollY 和 mPrevScrolledY 是否相同,相同则认为触底,停止滚动流程
生成长截图
每次滚动后广播,触发 mMainScrollView 局部截图,最后生成多个 Bitmap,最后合成 File 文件。在适配 Flutter 页面,这里并没有差异,所以这里就不做源码解读(不同 Miui 版本实现也有所不同)。
闲鱼适配方案Flutter 长截屏不适配原因
通过分析源码可知,Flutter 容器(SurfaceView/TextureView) canScrollVertically 方法并未被重写,为此无法被找到作为 mMainScrollView。假如我们重写 Flutter 容器,我们需要真实实现 getScrollY 才能保证触发滚动后 scrollY 和 mPrevScrolledY 不相等。不幸的是,getScrollY 是 final 类型,无法被继承类重写,为此我们无法在 Flutter 容器上做处理。
@InspectablePropertypublic final int getScrollY {return mScrollY;}
系统事件代理
转变思路,我们并不需要让 Flutter 容器被 Miui 系统识为可滚动视图,而是让 Flutter 接收到 Miui 系统指令。为此,我们构建一个不可见、不影响交互的滚动视图 ControlView 被 Miui 系统识别,并接收系统指令。ControlView 最后把指令传递给 Flutter,最终建立了 Miui 系统(ContentPort)和闲鱼 Flutter(可滚动 RenderObject)之间的通信。
其中通信事件:
-
1. void scrollBy(View view, int x, int y)
-
2. boolean canScrollVertically(View view, int direction, boolean startScreenshot)
-
3. int getScrollY(View view)
关键实现源码如下
public static FrameLayout setupLongScreenshotSupport(FrameLayout parent,View targetChild,IMiuiLongScreenshotViewDelegate delegate) {Context context = targetChild.getContext;MiuiLongScreenshotView screenshotView = new MiuiLongScreenshotView(context);screenshotView.setDelegate(delegate);screenshotView.addView(targetChild, new FrameLayout.LayoutParams(ViewGroup.LayoutParams.MATCH_PARENT,ViewGroup.LayoutParams.MATCH_PARENT));MiuiLongScreenshotControlView controlView = new MiuiLongScreenshotControlView(context);controlView.bindRealScrollView(screenshotView);if (parent == ) {parent = new FrameLayout(context);}parent.addView(screenshotView, new FrameLayout.LayoutParams(ViewGroup.LayoutParams.WRAP_CONTENT, ViewGroup.LayoutParams.WRAP_CONTENT));parent.addView(controlView);return parent;}
public class MiuiLongScreenshotControlView extends ScrollViewimplements MiuiScreenshotBroadcast.IListener {private IMiuiLongScreenshotView mRealView;...public void bindRealScrollView(IMiuiLongScreenshotView v) {mRealView = v;removeAllViews;Context context = getContext;LinearLayout ll = new LinearLayout(context);addView(ll);View btn = new View(context);LinearLayout.LayoutParams lp = new LinearLayout.LayoutParams(ViewGroup.LayoutParams.MATCH_PARENT,UIUtil.dp2px(context, 20000));ll.addView(btn, lp);resetScrollY(true);}public void resetScrollY(boolean startScreenshot) {if (mRealView != ) {setScrollY(0);if (getWindowVisibility == VISIBLE) {ThreadUtil.runOnUI(-> mRealView.canScrollVertically(1, startScreenshot));}}}@Overridepublic void onReceiveScreenshot {// 每次收到截屏广播,将 ControlView 滚动距离置 0// 提前查找滚动 RenderObject 并缓存// 提前计算 canScrollVerticallyresetScrollY(true);}@Overrideprotected void onAttachedToWindow {super.onAttachedToWindow;mContext = getContext;// 截屏广播监听MiuiScreenshotBroadcast.register(mContext, this);}@Overrideprotected void onDetachedFromWindow {super.onDetachedFromWindow;MiuiScreenshotBroadcast.unregister(mContext, this);}@Overridepublic boolean canScrollVertically(int direction) {if (mRealView != ) {return mRealView.canScrollVertically(direction, false);}return super.canScrollVertically(direction);}@Overridepublic void scrollBy(int x, int y) {super.scrollBy(x, y);if (mRealView != ) {mRealView.scrollBy(x, y);}}// 代理获取 DrawingCache@Overridepublic void setDrawingCacheEnabled(boolean enabled) {super.setDrawingCacheEnabled(enabled);if (mRealView != ) {mRealView.setDrawingCacheEnabled(enabled);}}@Overridepublic boolean isDrawingCacheEnabled {if (mRealView != ) {return mRealView.isDrawingCacheEnabled;}return super.isDrawingCacheEnabled;}@Overridepublic Bitmap getDrawingCache(boolean autoScale) {Bitmap result = (mRealView != )? mRealView.getDrawingCache(autoScale): super.getDrawingCache(autoScale);return result;}@Overridepublic void destroyDrawingCache {super.destroyDrawingCache;if (mRealView != ) {mRealView.destroyDrawingCache;}}@Overridepublic void buildDrawingCache(boolean autoScale) {super.buildDrawingCache(autoScale);if (mRealView != ) {mRealView.buildDrawingCache(autoScale);}}// 不消费屏幕操作事件@Overridepublic boolean onInterceptTouchEvent(MotionEvent ev) {return false;}@Overridepublic boolean onTouchEvent(MotionEvent ev) {return false;}}
无侵入识别滚动区域
获取 RenderObject 根节点
使用 mixin 扩展 WidgetsFlutterBinding,进而获取 RenderView
关键实现源码如下:
mixin NativeLongScreenshotFlutterBinding on WidgetsFlutterBinding {@overridevoid initInstances {super.initInstances;// 初始化FlutterMiuiLongScreenshotPlugin.inst;}@overridevoid handleDrawFrame {super.handleDrawFrame;try {NativeLongScreenshot.singleInstance._renderView = renderView;} catch (error, stack) {}}}
计算前台滚动 RenderObject
其中第 2 步条件检查:
-
1. width >= RenderView.width/2
-
2. height >= RenderView.height/2
-
3. 类型是 RenderViewportBase
-
4. axis == Axis.vertical
实现源码如下:
RenderViewportBase? findTopVerticalScrollRenderObject(RenderView? root) {Size rootSize = size(root, Size.zero);// if (root != ) {// _debugGetRenderTree(root, 0);// }RenderViewportBase? result = _recursionFindTopVerticalScrollRenderObject(root, rootSize);if (_hitTest(root, result)) {return result;}return ;}
RenderViewportBase? _recursionFindTopVerticalScrollRenderObject(RenderObject? renderObject, Size rootSize) {if (renderObject == ) {return ;}///get RenderObject Sizeif (_tooSmall(rootSize, size(renderObject, rootSize))) {return ;}if (renderObject is RenderViewportBase) {if (renderObject.axis == Axis.vertical) {return renderObject;}}final ListQueue<RenderObject> children = ListQueue<RenderObject>;if (renderObject.runtimeType.toString == '_RenderTheatre') {renderObject.visitChildrenForSemantics((RenderObject? child) {if (child != ) {children.addLast(child);}});} else {renderObject.visitChildren((RenderObject? child) {if (child != ) {children.addLast(child);}});}for (var child in children) {RenderViewportBase? viewport =_recursionFindTopVerticalScrollRenderObject(child, rootSize);if (viewport != ) {return viewport;}}return ;}
找到首个满足条件的 RenderViewportBase 并不一定是我们需要的对象,如下图所示:闲鱼详情页通过上述方法能找到红色框的 RenderViewportBase,在左图情况下,能满足滚动截图要求;但在右图情况下,留言面板遮挡了长列表,此时红色框 RenderObject 并不是我们想要的。
此刻我们需要检测 Widget 可见性/可交互检测能力。查看 Flutter 官方 visibility_detector 组件并不满足我们的要求,其通过在子 Widget 上放置一个 Layer 来间接检测可见状态,但因为通过在屏幕内的宽高判断,无法检测 Widget 被遮挡的情况。
左图长列表没有被遮挡,可以被操作;右图被留言面板遮挡,事件无法传递到长列表,无法被操作;为此,我们模拟用户的点击能否被触达来检测 RenderViewportBase 是否被遮挡,能否用来做长截屏滚动。
特别注意的是,当 Widget 被 Listener 包装,事件消费会被 RenderPointerListener 拦截,如下图所示。
查看 Flutter Framework 源码,Scrollable Widget 包装了 Listener,Semantics,IgnorePointer;闲鱼 PowerScrollView 使用了 ShrinkWrappingViewPort。为此,递归找到的 RenderSliverList 和点击测试找到的 RenderPointerListener 的距离为 5,如上图所示。
点击测试校验代码如下
bool _hitTest(RenderView? root, RenderViewportBase? result) {if (root == || result == ) {return false;}Size rootSize = size(root, Size.zero);HitTestResult hitResult = HitTestResult;root.hitTest(hitResult, position: Offset(rootSize.width/2, rootSize.height/2));for (HitTestEntry entry in hitResult.path) {if (entry.target == result) {return true;}}/*** 处理如下 case* RenderPointerListener 2749d135RenderSemanticsAnnotations 1cd639bfRenderIgnorePointer 7e33fffRenderShrinkWrappingViewport 1167ca33*/RenderPointerListener? pointerListenerParent;AbstractNode? parent = result.parent;const int lookUpLimit = 5;int lookupCount = 0;while (parent != &&lookupCount < lookUpLimit &&parent.runtimeType.toString != '_RenderTheatre') {lookupCount ;if (parent is RenderPointerListener) {pointerListenerParent = parent;}parent = parent.parent;}if (pointerListenerParent != ) {for (HitTestEntry entry in hitResult.path) {if (entry.target == pointerListenerParent) {return true;}}}return false;}
异步 Channel 通信方案
Flutter channel 通信方案如上图所示,其中 EventChannel 和 MethodChannel 运行在 Java 主线程,同 Dart Platform Isolate,而 Dart 层事件处理逻辑在 UI Isolate,为此并不在同一线程。可以发现,Java → Dart → Java 发生了 2 次线程切换。使用小米 K50 测试性能,从 EventChannel 发送事件 到 MethodChannel 接收返回值,记录耗时。可见,首次 canScrollVertically (由截屏广播触发)需要递归查找滚动组件,耗时为 10-30ms,之后耗时均在 5ms 以内。
08-08 16:15:56.060 11079 11079 E longscreenshot: canScrollVertically use_time=2508-08 16:15:56.278 11079 11079 E longscreenshot: canScrollVertically use_time=208-08 16:16:05.342 11079 11079 E longscreenshot: canScrollVertically use_time=1008-08 16:16:05.562 11079 11079 E longscreenshot: canScrollVertically use_time=1
为保证在异步调用的情况下,MIUI ContentPort 下发命令均能获取到最新值,这里做以下特殊处理
1. 截屏广播提前计算 canScrollVerticallly 并缓存结果
2. MIUI ContentPort 调用 canScrollVerticallly 直接返回最新缓存值,异步触发计算
3. MIUI ContentPort 调用 scrollBy 后,及时更新 canScrollVerticallly 和 getScrollY 缓存值
同步 FFI 通信方案
异步调用方案,在高端机且 App 任务队列无阻塞情况下,能正确且准确运行,但在低端机和 App 任务较重时,可能存在返回 ContentPort 数据非最新的情况,为此我们考虑使用 FFI 同步通信的方案。
以上同步方案,一次同步调用性能分析,基本在 5ms 以内:
关键实现代码如下:
@Keeppublic class NativeLongScreenshotJni implements Serializable {static {System.loadLibrary("flutter_longscreenshot");}public static native void nativeCanScrollVertically(int direction,boolean startScreenshot,int callbackId);public static native void nativeGetScrollY(int screenWidth, int callbackId);public static native void nativeScrollBy(int screenWidth, int x, int y);public static boolean canScrollVertically(final int direction,final boolean startScreenshot) {FlutterLongScreenshotCallbacks.AwaitCallback callback =FlutterLongScreenshotCallbacks.newCallback;nativeCanScrollVertically(direction, startScreenshot, callback.id);int result = callback.waitCallback.getResult;return result == 1;}public static int getScrollY(final int screenWidth) {FlutterLongScreenshotCallbacks.AwaitCallback callback =FlutterLongScreenshotCallbacks.newCallback;nativeGetScrollY(screenWidth, callback.id);// waitCallback 同步等待 C 调用 FlutterLongScreenshotCallbacks.handleDartCallint result = callback.waitCallback.getResult;return result;}public static void scrollBy(int screenWidth, int x, int y) {nativeScrollBy(screenWidth, x, y);}}@Keeppublic class FlutterLongScreenshotCallbacks implements Serializable {public static AwaitCallback newCallback {AwaitCallback callback = new AwaitCallback;CALLBACKS.put(callback.id, callback);return callback;}// C DART_EXPORT void resultCallback(int callbackId, int result) 反射调用public static void handleDartCall(int id, int result) {AwaitCallback callback = CALLBACKS.get(id);if (callback != ) {CALLBACKS.remove(id);callback.release(result);}}private static final SparseArray<AwaitCallback> CALLBACKS = new SparseArray<>;@Keeppublic static class AwaitCallback {public static final int RESULT_ERR = -1;private final CountDownLatch mLatch = new CountDownLatch(1);private int mResult = RESULT_ERR;public int id {return hashCode;}public AwaitCallback waitCallback {try {mLatch.await(100, TimeUnit.MILLISECONDS);} catch (Throwable e) {e.printStackTrace;}return this;}public void release(int result) {mResult = result;mLatch.countDown;}public int getResult {return mResult;}}}
void setDartInt(Dart_CObject& dartObj, int value) {dartObj.type = Dart_CObject_kInt32;dartObj.value.as_int32 = value;}JNIEXPORT void JNICALLnativeCanScrollVertically(JNIEnv *env, jclass cls,jint direction, jboolean startScreenshot, jint callbackId) {Dart_CObject* dart_args[4];Dart_CObject dart_arg0;Dart_CObject dart_arg1;Dart_CObject dart_arg2;Dart_CObject dart_arg3;setDartString(dart_arg0, strdup("canScrollVertically"));setDartInt(dart_arg1, direction);setDartBool(dart_arg2, startScreenshot);setDartLong(dart_arg3, callbackId);dart_args[0] = &dart_arg0;dart_args[1] = &dart_arg1;dart_args[2] = &dart_arg2;dart_args[3] = &dart_arg3;Dart_CObject dart_object;dart_object.type = Dart_CObject_kArray;dart_object.value.as_array.length = 4;dart_object.value.as_array.values = dart_args;Dart_PostCObject_DL(send_port_, &dart_object);}// getScrollY 和 scrollBy 实现类似
DART_EXPORT void resultCallback(int callbackId, int result) {JNIEnv *env = _getEnv;if (env != ptr) {auto cls = _findClass(env, jCallbackClassName);jmethodID handleDartCallMethod = ptr;if (cls != ptr) {// 调用 java 代码 FlutterLongScreenshotCallbacks.handleDartCall(int id, int result)handleDartCallMethod = env->GetStaticMethodID(cls,"handleDartCall", "(II)V");}if (cls != ptr && handleDartCallMethod != ptr) {env->CallStaticVoidMethod(cls, handleDartCallMethod,callbackId, result);} else {print("resultCallback. find method handleDartCall is ptr");}}}
class NativeLongScreenshot extends Object {...late final NativeLongScreenshotLibrary _nativeLibrary;late final ReceivePort _receivePort;late final StreamSubscription _subscription;NativeLongScreenshot {..._nativeLibrary = initLibrary;_receivePort = ReceivePort;var nativeInited = _nativeLibrary.initializeApi(ffi.NativeApi.initializeApiDLData);assert(nativeInited == 0, 'DART_API_DL_MAJOR_VERSION != 2');_subscription = _receivePort.listen(_handleNativeMessage);_nativeLibrary.registerSendPort(_receivePort.sendPort.nativePort);}void _handleNativeMessage(dynamic inArgs) {List<dynamic> args = inArgs;String method = args[0];switch (method) {case 'canScrollVertically': {int direction = args[1];bool startScreenshot = args[2];int callbackId = args[3];final bool canScroll = canScrollVertically(direction, startScreenshot);int result = canScroll ? 1 : 0;_nativeLibrary.resultCallback(callbackId, result);} break;case 'getScrollY': {int nativeScreenWidth = args[1];int callbackId = args[2];int result = getScrollY(nativeScreenWidth);_nativeLibrary.resultCallback(callbackId, result);} break;case 'scrollBy': {int nativeScreenWidth = args[1];int nativeX = args[2];int nativeY = args[3];scrollBy(nativeY, nativeScreenWidth);} break;}}}
完成国内主要机型适配,现在线上几乎不再有用户反馈 Flutter 页面不支持长截屏。闲鱼 Android 用户已经能用系统长截屏能力,分享自己喜欢的商品、圈子内容,卖家能使用一张图片推广自己的全部商品,买家能帮助家里不会用 App 的老人找商品。面对系统功能适配,业务 App 侧也并不是完全束手无策。通过以下过程便有可能找到解决之道:
-
• 合理猜想(系统模块会调用业务视图接口)
-
• 工具辅助分析和验证(ASM 代码 hook,日志输出)
-
• 源码查找和截图(代码查找和反编译)
-
• 发散思考(ControlView 顶替 Flutter 容器,瞒天过海)
-
• 方案实现(业务无侵入,一次实现全部业务页面适配)
,