如何解决SwitchAccess 兼容的虚拟视图节点的深层树状结构
我正在尝试在 Android 中创建一个可访问的自定义 View
。
View
包含一个虚拟节点的树层次结构,在我的情况下可以是多级深度。
树不一定是二元的:每一层的每个节点都可以有用户想要的任意数量的子节点。
具体来说,我试图使其与 TalkBack 的 SwitchAccess 辅助功能服务兼容。
最终目标是使其兼容所有 TalkBack 的无障碍服务,但至少必须与 SwitchAccess 兼容。
在这篇文章中,我正在尝试单开关访问。
树的结构事先未知,其扫描顺序也未知。它们都是在运行时定义的。
我正在使用 ExploreByTouchHelper
类,它是 AccessibilityDelegate
的便利包装器。
以下类似于事件序列和虚拟树的图像:
一个示例扫描序列由节点内的数字表示。
从用户的角度来看,他们将叶子视为某种随机颜色的矩形。
这只是自定义 View
的内容。
描绘的扫描顺序如下:
- 首先,用户看到所有的节点都被一起扫描(有根被扫描)。他们按下开关并选择根,将扫描过程移到树的第二级。
- 然后扫描第一个内部节点,用户忽略它,所以扫描移动到第二个根的子节点,用户点击开关,将扫描过程移动到树的第三级,即点击节点的子节点。
- 依此类推,直到用户点击/选择叶节点(在名为
7. Click!
的步骤),在这种情况下会发生自定义操作。
命名法:
- 什么是 SwitchAccess?这是一项服务,允许用户通过将单个(或几个)硬件开关连接到他们的手机来键入键/字母。 你可以把它想象成一个残障人士的物理键盘,它有一个(或几个)开关,而不是每个字母一个开关。 用户在一个称为扫描的过程的帮助下输入一个字母,其中字母表中的每个字母(或任何类型的键)都被一个一个扫描,当用户点击他们的单个开关然后输入相应的字母。 这就像点击一个开关,点击频率被这项服务转换成字母。
- 什么是 TalkBack?一组可供 Android 用户使用的辅助功能服务(包括 SwitchAccess)。 这是一个普通的 Android 应用程序(Android Accessibility Suite),如果您的系统中尚未包含该应用程序,则可以从 here 下载。
我尝试过(但失败了):
- 试验每个
AccessibilityNodeInfo
的焦点状态、选中状态和无障碍焦点状态。 - 每次点击只报告我感兴趣的子树部分。这意味着每次点击都会报告不同的树(通过
getVisibleVirtualViews
)。 - 受
AccessibilityNodeInfo.CollectionInfo
的实现启发,对AccessibilityNodeInfo.CollectionItemInfo
和GridView
进行实验。 - 仅在用户点击叶子时发送
AccessibilityEvent
s(而不是内部节点),以便让扫描在点击内部节点时继续。 - 仅报告叶组而不是内部节点。
- 在每次用户点击时重新安装/更改整个
AccessibilityNodeProvider
和/或AccessibilityDelegate
。
按照我迄今为止的最大努力(这是我上面的一些努力的组合),我们也可以就此进行讨论:
import android.content.Context;
import android.graphics.Canvas;
import android.graphics.Color;
import android.graphics.Paint;
import android.graphics.Point;
import android.graphics.Rect;
import android.graphics.RectF;
import android.os.Bundle;
import android.view.KeyEvent;
import android.view.MotionEvent;
import android.view.View;
import android.view.ViewGroup;
import android.view.accessibility.AccessibilityEvent;
import androidx.annotation.NonNull;
import androidx.annotation.Nullable;
import androidx.appcompat.app.AppCompatActivity;
import androidx.core.view.ViewCompat;
import androidx.core.view.accessibility.AccessibilityNodeInfoCompat;
import androidx.customview.widget.ExploreByTouchHelper;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Locale;
import java.util.Random;
public class MainActivity extends AppCompatActivity {
public static int randomColor(final int alpha,final Random rand) {
final byte[] components = new byte[3];
rand.nextBytes(components);
return Color.argb(alpha,components[0] & 0xFF,components[1] & 0xFF,components[2] & 0xFF);
}
public static class Node {
public Node parent = null;
public final ArrayList<Node> children = new ArrayList<>();
public final Point index = new Point(),//Location of this Node in its parent. Root Node does not use this.
size = new Point(); //Number of children this Node has per dimension (x == columns,y == rows).
public String text = null; //The text associated with each Node.
public int id = -1,//It is initialized at Tree construction.
color = 0; //It is only used for leaves,but for simplicity included in every Node.
/**
* Used as a default way to create the children of this Node.
* @param title Some value used to construct the text of each children.
* @param sizeX Number of columns each children will be initialized to have.
* @param sizeY Number of rows each children will be initialized to have.
* @param rand Used for producing each child's color.
*/
public void addChildren(final String title,final int sizeX,final int sizeY,final Random rand) {
for (int row = 0; row < size.y; ++row) {
for (int col = 0; col < size.x; ++col) {
final Node child = new Node();
child.parent = this;
children.add(child);
child.index.set(col,row);
child.size.set(sizeX,sizeY);
child.text = String.format(Locale.ENGLISH /*Just use ENGLISH for only the demonstration purposes.*/,"%s|%s:%d,%d",text,title,row,col);
child.color = randomColor(255,rand);
}
}
}
/** @param bounds Serves as input (initialized with the root Node's bounds) and as output (giving the bounds relative to root for the calling Node). */
public void updateBounds(final RectF bounds) {
if (parent != null) {
parent.updateBounds(bounds);
//Adjust parent bounds to locate the current node:
final float cellWidth = bounds.width() / parent.size.x,cellHeight = bounds.height() / parent.size.y;
bounds.left += (cellWidth * index.x);
bounds.top += (cellHeight * index.y);
bounds.right -= (cellWidth * (parent.size.x - index.x - 1));
bounds.bottom -= (cellHeight * (parent.size.y - index.y - 1));
}
}
}
/**
* Gets a subtree (starting from the given Node) of nodes into the given lists.
* @param node the root of the subtree we are interested in.
* @param allNodes all nodes of the subtree will go in here.
* @param leavesOnly only the leaves of the subtree will go in here.
*/
public static void getNodes(final Node node,final ArrayList<Node> allNodes,final ArrayList<Node> leavesOnly) {
allNodes.add(node);
if (node.children.isEmpty())
leavesOnly.add(node);
else
for (final Node child: node.children)
getNodes(child,allNodes,leavesOnly);
}
/** Sacrificing memory for speed: this is essentially a huge cache. */
public static class Tree {
public final Node root;
public final List<Node> nodes,//All nodes of the tree.
leaves; //Only leaves of the tree (which will exist in both 'nodes' property and in 'leaves' property).
public Tree(final Node root) {
this.root = root;
final ArrayList<Node> nodesList = new ArrayList<>();
final ArrayList<Node> leavesList = new ArrayList<>();
getNodes(root,nodesList,leavesList);
nodes = Collections.unmodifiableList(nodesList);
leaves = Collections.unmodifiableList(leavesList);
final int sz = nodes.size();
for (int i = 0; i < sz; ++i)
nodes.get(i).id = i; //As you can see the id corresponds exactly to the index of the Node in the list (so as to have easier+faster retrieval of Node by its id).
}
}
/** @return a Tree for testing. */
public static Tree buildTestTree() {
final Random rand = new Random();
final Node root = new Node();
root.size.set(2,1); //2 columns,1 row.
root.text = "Root";
root.addChildren("Inner",2,1,rand); //2 columns,1 row.
for (final Node rootChild: root.children) {
rootChild.addChildren("Inner",1 row.
for (final Node rootInnerChild: rootChild.children)
rootInnerChild.addChildren("Leaf",rand); //1 column,1 row. Basically a leaf.
}
return new Tree(root);
}
/** @return a value conforming to the measureSpec,while being as close as possible to the preferredSizeInPixels. */
public static int getViewSize(final int preferredSizeInPixels,final int measureSpec) {
int result = preferredSizeInPixels;
final int specMode = View.MeasureSpec.getMode(measureSpec);
final int specsize = View.MeasureSpec.getSize(measureSpec);
switch (specMode) {
case View.MeasureSpec.UNSPECIFIED: result = preferredSizeInPixels; break;
case View.MeasureSpec.AT_MOST: result = Math.min(preferredSizeInPixels,specsize); break;
case View.MeasureSpec.EXACTLY: result = specsize; break;
}
return result;
}
/** The custom View which maintains the tree hierarchy of virtual views. */
public static class HierarchyView extends View {
private final MyAccessibilityDelegate delegate; //The ExploreByTouchHelper implementation.
public final Tree tree; //The tree of virtual views.
public Node selected; //The last 'clicked' node from all the nodes in the tree.
private final int preferredWidth,preferredHeight; //The preferred size of this View.
private final Paint tmpPaint; //Used for drawing.
private final RectF tmpBounds; //Used for drawing.
public HierarchyView(final Context context) {
super(context);
tmpPaint = new Paint();
tmpBounds = new RectF();
selected = null;
//Hardcoded magic numbers for the dimensions of this View,only in order to keep things simple in this demonstration:
preferredWidth = 600;
preferredHeight = 300;
tree = buildTestTree();
super.setContentDescription("Hierarchy");
ViewCompat.setImportantForAccessibility(this,ViewCompat.IMPORTANT_FOR_ACCESSIBILITY_YES);
ViewCompat.setAccessibilityDelegate(this,delegate = new MyAccessibilityDelegate(this));
}
@Override
protected void onMeasure(final int widthMeasureSpec,final int heightMeasureSpec) {
setMeasuredDimension(getViewSize(preferredWidth,widthMeasureSpec),getViewSize(preferredHeight,heightMeasureSpec));
}
/**
* Use this method instead of {@link Node#updateBounds(RectF)},which (this method) will properly initialize the root Node's bounds.
* @param bounds The output bounds for the given Node.
* @param node The input Node to get the bounds for.
*/
public void updateBounds(final RectF bounds,final Node node) {
bounds.left = bounds.top = 0;
bounds.right = getWidth();
bounds.bottom = getHeight();
node.updateBounds(bounds);
}
@Override
protected void onDraw(final Canvas canvas) {
for (final Node leaf: tree.leaves) {
tmpPaint.setColor(leaf.color);
tmpPaint.setAlpha(selected == leaf? 255: 64);
updateBounds(tmpBounds,leaf); //Not the most efficient (needs logN),but remember this is just a demo.
canvas.drawRect(tmpBounds,tmpPaint);
}
}
@Override
public boolean dispatchHoverEvent(final MotionEvent event) {
//This is required by ExploreByTouchHelper's docs:
return delegate.dispatchHoverEvent(event) || super.dispatchHoverEvent(event);
}
@Override
public boolean dispatchKeyEvent(final KeyEvent event) {
//This is required by ExploreByTouchHelper's docs:
return delegate.dispatchKeyEvent(event) || super.dispatchKeyEvent(event);
}
@Override
protected void onFocusChanged(final boolean gainFocus,final int direction,final @Nullable Rect prevIoUslyFocusedRect) {
super.onFocusChanged(gainFocus,direction,prevIoUslyFocusedRect);
//This is required by ExploreByTouchHelper's docs:
delegate.onFocusChanged(gainFocus,prevIoUslyFocusedRect);
}
/*
@Override
public boolean onTouchEvent(final MotionEvent event) {
final int virtualViewId = delegate.getVirtualViewAt(event.getX(),event.getY());
if (virtualViewId != ExploreByTouchHelper.INVALID_ID)
selected = tree.nodes.get(virtualViewId);
invalidate();
return super.onTouchEvent(event);
}
*/
}
public static class MyAccessibilityDelegate extends ExploreByTouchHelper {
private final HierarchyView host;
/**
* This is used as the <b>parent</b> of each node that should be interactive. If null,then the
* root should be interactive,otherwise if not null,then the its children should be interactive.
*/
private Node last;
public MyAccessibilityDelegate(final @NonNull HierarchyView host) {
super(host);
this.host = host;
last = null; //Start with root.
}
/** Helper method to retrieve a read-only Iterable of the nodes that should be interactive. */
private Iterable<Node> readVisibleNodes() {
return last == null? Collections.singletonList(host.tree.root) : Collections.unmodifiableList(last.children);
}
@Override
protected int getVirtualViewAt(final float x,final float y) {
final RectF bounds = new RectF();
for (final Node node: readVisibleNodes()) {
host.updateBounds(bounds,node);
if (bounds.contains(x,y))
return node.id;
}
return INVALID_ID;
}
@Override
protected void getVisibleVirtualViews(final List<Integer> virtualViewIds) {
for (final Node node: readVisibleNodes())
virtualViewIds.add(node.id);
}
@Override
protected void onPopulateNodeForVirtualView(final int virtualViewId,final @NonNull AccessibilityNodeInfoCompat info) {
final Node node = host.tree.nodes.get(virtualViewId);
//Just set all text to node#text for simplicity:
info.setText(node.text);
info.setHintText(node.text);
info.setContentDescription(node.text);
//Get the node's bounds:
final RectF bounds = new RectF();
host.updateBounds(bounds,node);
/*Although deprecated,setBoundsInParent is actually what ExploreByTouchHelper requires,and itself
then computes the bounds in screen. So lets just setBoundsInParent,instead of setBoundsInScreen...*/
if (node.parent == null) { //If node is the root:
info.setParent(host); //The View itself is the parent of it (or maybe not,I am not sure).
info.setBoundsInParent(new Rect(Math.round(bounds.left),Math.round(bounds.top),Math.round(bounds.right),Math.round(bounds.bottom)));
}
else {
/*To get the bounds of any node which is not the root,I simply subtract the parent's bounds
with the current node's bounds. I kNow... not the most efficient,but it's just a demo Now.*/
info.setParent(host,node.parent.id);
final RectF parentBounds = new RectF();
host.updateBounds(parentBounds,node.parent);
info.setBoundsInParent(new Rect(Math.round(bounds.left - parentBounds.left),Math.round(bounds.top - parentBounds.top),Math.round(bounds.right - parentBounds.left),Math.round(bounds.bottom - parentBounds.top)));
}
//As I have found out,those calls are absolutely necessary for the virtual views:
info.setEnabled(true);
info.setFocusable(true);
//These calls seem to not be absolutely necessary,but I am not sure:
info.setVisibletoUser(true);
info.setImportantForAccessibility(true);
// info.setContentInvalid(false);
// info.setAccessibilityFocused(last == node);
// info.setFocused(last == node);
// info.setChecked(last == node);
// info.setSelected(last == node);
if (node.parent == last) { //This is the way I am testing if the current node should be interactive.
info.setClickable(true);
info.setCheckable(true);
//info.setCanopenPopup(true);
//info.setContextClickable(true);
//info.addAction(AccessibilityNodeInfoCompat.ACTION_SELECT);
info.addAction(AccessibilityNodeInfoCompat.ACTION_CLICK);
}
if (!node.children.isEmpty()) {
info.setCollectionInfo(AccessibilityNodeInfoCompat.CollectionInfoCompat.obtain(node.size.y,node.size.x,true,AccessibilityNodeInfoCompat.CollectionInfoCompat.SELECTION_MODE_SINGLE));
for (final Node child: node.children)
info.addChild(host,child.id);
}
if (node.parent != null)
info.setCollectionItemInfo(AccessibilityNodeInfoCompat.CollectionItemInfoCompat.obtain(node.index.y,node.index.x,false,false));
}
@Override
protected boolean onPerformActionForVirtualView(final int virtualViewId,final int action,final @Nullable Bundle arguments) {
if (action == AccessibilityNodeInfoCompat.ACTION_CLICK) {
host.selected = host.tree.nodes.get(virtualViewId);
last = host.selected.children.isEmpty()? null: host.selected;
// if (host.selected.children.isEmpty()) {
invalidateVirtualView(virtualViewId); //,AccessibilityEventCompat.CONTENT_CHANGE_TYPE_SUBTREE);
sendEventForVirtualView(virtualViewId,AccessibilityEvent.TYPE_VIEW_CLICKED);
host.invalidate(); //To redraw the UI.
// }
// else
// invalidateRoot();
// invalidateVirtualView(virtualViewId);
// host.sendAccessibilityEvent(AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED);
// host.sendAccessibilityEvent(AccessibilityEvent.TYPE_VIEW_CLICKED);
// host.sendAccessibilityEvent(AccessibilityEventCompat.TYPE_VIEW_CONTEXT_CLICKED);
// sendEventForVirtualView(virtualViewId,AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED);
// invalidateVirtualView(virtualViewId,AccessibilityEventCompat.CONTENT_CHANGE_TYPE_SUBTREE);
return true;
}
return false;
}
}
@Override
protected void onCreate(final Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(new HierarchyView(this),new ViewGroup.LayoutParams(ViewGroup.LayoutParams.WRAP_CONTENT,ViewGroup.LayoutParams.WRAP_CONTENT));
}
}
这会导致扫描在点击时冻结(但至少它在树中更深一层),而实际需要的行为是让扫描继续进行,直到点击一个叶子,扫描应该结束。 几乎就像扫描软件键盘一样(即首先是一组键,然后点击,扫描每个键,依此类推)。
请注意,由给定代码构建的树与图像的树并不完全相同。 图片作为视觉示例,代码作为讨论的基础。
我基本上是想找出 SwitchAccess 中的哪些规则会使我的应用程序与之兼容。 理想情况下,我希望用户看到正在扫描的叶子组,而不是像提供的代码那样每次都看到单个节点,但我想这是一个不同的故事。
我认为我要问的是可能的,因为否则 setParent
之类的方法不会包含在 AccessibilityNodeInfo
类中。
我也在考虑用 drawing order 进行实验,但我根本不知道它是否相关。
我正在使用最低 SDK 版本 14(如果有的话)。
互联网上有一些关于单级深度虚拟树的例子,但我就是不知道如何使它们成为多级深度。
请注意,我们采取了几个步骤来缩短代码,因此它不会遵循有关面向对象编程的最佳实践、时间复杂度、内存使用等,因为它只是作为演示。
一些资源:
-
The corresponding Google I/O 2013 video(从
ExploreByTouchHelper
用例的介绍开始)。 - TalkBack's source code。我看了很多遍,但我仍然不知道如何解决我的问题。
- How to use SwitchAccess on your phone(用户视角)。
解决方法
我不相信你想要的是可能的。或者至少,我不相信您想要的具有支持它的“显式 API”。让我们来谈谈 Switch Access 将关注什么。基本上,它会关注任何
- 未明确标记为“无障碍不重要”
- 有某种类型的动作与之相关
- 点按
- 点击并按住
- 自定义操作
- 等
Switch Access 对事物进行分组的方式不会响应任何特定的 API,而是根据与标准用户体验相关的现有信息并基于 Switch Access 配置计算得出。您可能会根据行列扫描与组选择的不同thigns 来确定这一点。一般来说,重要的事情是:
- 在视图层次结构中排序
- 在屏幕上的位置
为“精确分组”操作这些将非常困难。每个版本的 Switch Access 都可以为所欲为。没有记录的 API 说“这是分组”。 Switch Access 只是尽最大努力理解标准的 Android API。
从用户的角度准确表达您所拥有的信息。
- 将事物组放入虚拟布局中。
- 确保可以与之交互的所有内容都标记为此类。
- 在您的虚拟层次结构中以合理的顺序排列您的视图
这就是你真正能做的。事实上,试图对其进行更多的操作可能会让用户感到困惑。
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。