Download 7.AdvTouching

Mobile Computing
Advanced Touching
Assg Part1 AssgPart2
1
Copyright 2014 by Janson Industries
Objectives
▀
Explain
 Motion
events
 Multi-touch
 Gestures
 Creating
gestures
2
Copyright 2014 by Janson Industries
Hardware
▀
The touch screen is constructed
to pick up pressure on the
screen and convert that to data
 Screen
coordinates
 Pressure
 Size
of touch
 Time
▀
All info stored in a MotionEvent
object
3
Copyright 2014 by Janson Industries
Motion Sequence Actions
▀
The action property holds value
that identifies action type
 action
= 0 (ACTION_DOWN) is
the initial touch
 action
= 2 (ACTION_MOVE) any
up, down, or sideways movement
on the screen
 action
= 1 (ACTION_UP) is when
finger is lifted off the screen
4
Copyright 2014 by Janson Industries
Motion Sequence

onTouch or onTouchEvent
method called when motion
action occurs
A
MotionEvent object passed to
these methods


Copyright 2014 by Janson Industries
For classes that implement
onTouchListener, onTouch called
For subclasses of View,
onTouchEvent called
5
Example



Will create a view subclass
called TouchTest
Its onTouchEvent will get info
from the MotionEvent object and
display in LogCat
Will also use the switch structure
6
Copyright 2014 by Janson Industries
Nested if/else
▮ Placing an if statement as one
of the statements to be
executed in an if/else clause
if (month == 1) {
monthTV.setText(“Jan”);}
Nested if
else {if (month == 2) {
monthTV.setText(“Feb”);}
else { if (month == 3) {
monthTV.setText(“Mar”);}
else { if (month == 4) {………
7
Copyright 2014 by Janson Industries
Switch
▮ Instead of complicated nested
ifs, can use a switch
switch (Month) {
case 1: monthTV.setText(“Jan”);
case 2: monthTV.setText(“Feb”);
case 3: monthTV.setText(“Mar”);
:
:
:
:
:
:
:
:
:
:
default: System.out.println(“Not
month!”);
}
break;
break;
break;
a valid
8
Copyright 2014 by Janson Industries
Switch
▮ Need the break statements
because once the condition is
true, all subsequent statements
are executed
▮ In the example, this means
▮ The label is set to Dec
▮ And the “Not a valid month”
message displayed
9
Copyright 2014 by Janson Industries
TouchTest

TouchTest (a subclass of View)
will:


Check for touches
When a touch occurs, display a
variety of info about the touch
10
Copyright 2014 by Janson Industries
TouchTest

Created new TouchProj project,
my.touch.com package and Main
activity, then create new class
TouchTest
package my.touch.com;
import android.content.Context;
import android.util.AttributeSet;
import android.view.MotionEvent;
import android.view.View;
public class TouchTest extends View {
public TouchTest(Context context, AttributeSet attrs) {
super(context);
}
11
Copyright 2014 by Janson Industries
TouchTest
public boolean onTouchEvent(MotionEvent event) {
int action = event.getAction();
switch (action) {
case 0:
System.out.println("Action: " + event.getAction());
System.out.println("Location: " + event.getX() + " x "
+ event.getY() + " y");
System.out.println("Pressure: " + event.getPressure());
System.out.println("Size: " + event.getSize());
System.out.println("Down time: " +
event.getDownTime() + " ms");
System.out.println("Event time: " +
event.getEventTime() + " ms");
System.out.println("Elapsed: " + (event.getEventTime()
- event.getDownTime()) + " ms");
break;
12
Copyright 2014 by Janson Industries
TouchTest
case 1:
System.out.println("Action: " + event.getAction());
System.out.println("Location: " + event.getX() + " x "
+ event.getY() + " y");
System.out.println("Pressure: " + event.getPressure());
System.out.println("Size: " + event.getSize());
System.out.println("Down time: " +
event.getDownTime() + " ms");
System.out.println("Event time: " +
event.getEventTime() + " ms");
System.out.println("Elapsed: " + (event.getEventTime()
- event.getDownTime()) + " ms");
break;
}
return (true);
}
13
Copyright
2014 by Janson Industries
}
main.xml

Change activity_main.xml to add
TouchTest to main screen
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout
xmlns:android="http://schemas.android.com/apk/res/android"
android:orientation="vertical"
android:layout_width="fill_parent"
android:layout_height="fill_parent">
<my.touch.com.TouchTest android:id="@+id/tt"
android:layout_width="wrap_content"
android:layout_height="wrap_content" />
</LinearLayout>
14
Copyright 2014 by Janson Industries
TouchTest

Run

Click & hold

Release
15
Copyright 2014 by Janson Industries
Touch Test
02-28 13:42:54.086: I/System.out(18244): Action: 0
02-28 13:42:54.086: I/System.out(18244): Location: 325.0 x 500.0 y
02-28 13:42:54.097: I/System.out(18244): Pressure: 1.0
02-28 13:42:54.097: I/System.out(18244): Size: 0.0
02-28 13:42:54.127: I/System.out(18244): Down time: 684320130 ms
02-28 13:42:54.127: I/System.out(18244): Event time: 684320130 ms
02-28 13:42:54.167: I/System.out(18244): Elapsed: 0 ms
02-28 13:42:56.106: I/System.out(18244): Action: 1
02-28 13:42:56.106: I/System.out(18244): Location: 325.0 x 500.0 y
02-28 13:42:56.117: I/System.out(18244): Pressure: 1.0
02-28 13:42:56.131: I/System.out(18244): Size: 0.0
02-28 13:42:56.131: I/System.out(18244): Down time: 684320130 ms
02-28 13:42:56.131: I/System.out(18244): Event time: 684322150 ms
02-28 13:42:56.131: I/System.out(18244): Elapsed: 2020 ms
16
Copyright 2014 by Janson Industries
Touch Test



In emulator, pressure and size
will always be 1 and 0
If you click and drag, a series of
move actions (2) will be
generated
We’ll add a check for an action 2
in the switch
17
Copyright 2014 by Janson Industries
TouchTest

Took out the pressure and size
displays
case 2:
System.out.println("Action: " + event.getAction());
System.out.println("Location: " + event.getX() + " x "
+ event.getY() + " y");
System.out.println("Down time: " +
event.getDownTime() + " ms");
System.out.println("Event time: " +
event.getEventTime() + " ms");
System.out.println("Elapsed: " + (event.getEventTime()
- event.getDownTime()) + " ms");

Click and drag quickly to right
18
Copyright 2014 by Janson Industries
Touch Test
02-28 13:48:09.017: I/System.out(18304): Action: 2
02-28 13:48:09.017: I/System.out(18304): Location: 294.843 x 423.79776 y
02-28 13:48:09.037: I/System.out(18304): Down time: 684633418 ms
02-28 13:48:09.037: I/System.out(18304): Event time: 684635056 ms
02-28 13:48:09.037: I/System.out(18304): Elapsed: 1638 ms
02-28 13:48:09.129: I/System.out(18304): Action: 2
02-28 13:48:09.129: I/System.out(18304): Location: 264.81342 x 436.74582 y
02-28 13:48:09.137: I/System.out(18304): Down time: 684633418 ms
02-28 13:48:09.137: I/System.out(18304): Event time: 684635169 ms
02-28 13:48:09.137: I/System.out(18304): Elapsed: 1751 ms
02-28 13:48:09.150: I/System.out(18304): Action: 2
02-28 13:48:09.157: I/System.out(18304): Location: 262.70416 x 437.40497 y
02-28 13:48:09.157: I/System.out(18304): Down time: 684633418 ms
02-28 13:48:09.157: I/System.out(18304): Event time: 684635179 ms
02-28 13:48:09.169: I/System.out(18304): Elapsed: 1761 ms
19
Copyright 2014 by Janson Industries
Implementing Touch




Create a View subclass called
Square
When screen touched a square
will appear around the touch
If movement, square will follow
the movement
When touch lifted square
disappears
20
Copyright 2014 by Janson Industries
Square
package my.touch.com;
import
import
import
import
import
import
import
android.content.Context;
android.graphics.Canvas;
android.graphics.Color;
android.graphics.Paint;
android.util.AttributeSet;
android.view.MotionEvent;
android.view.View;
public class Square extends View {
private float x = -30; // Set initial location off screen
private float y = -30;
private Paint myPaint;
public Square(Context context, AttributeSet attrs) {
super(context, attrs);
myPaint = new Paint();
myPaint.setColor(Color.BLUE);
} 2014 by Janson Industries
Copyright
21
public boolean onTouchEvent(MotionEvent event) {
int action = event.getAction();
switch (action) {
case MotionEvent.ACTION_DOWN: // In both cases these
case MotionEvent.ACTION_MOVE: // statements are run
x = event.getX();
y = event.getY();
break;
case MotionEvent.ACTION_UP:
x = -30;
// set the location outside the screen area
y = -30;
break;
}
return (true);
}
Square
public void draw(Canvas canvas) {
canvas.drawRect(x-15, y-15, x+15, y+15, myPaint);
invalidate();
}
Copyright
2014 by Janson Industries
}
22
activity_main.xml
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout
xmlns:android="http://schemas.android.com/apk/res/android"
android:orientation="vertical"
android:layout_width="fill_parent"
android:layout_height="fill_parent">
<my.touch.com.Square android:id="@+id/sq"
android:layout_width="wrap_content"
android:layout_height="wrap_content" />
</LinearLayout>
23
Copyright 2014 by Janson Industries
Click
Drag
Release
24
Copyright 2014 by Janson Industries
Assignment - Part 1


Create Square such that the
square is dragged around (as
shown earlier)
But when released, square
returns to the original clicked
location
25
Copyright 2014 by Janson Industries
Velocity


Can measure using a
VelocityTracker
VelocityTracker initially obtained
then
 Specify
pixels per time period in
milliseconds
 Per
 Feed
second means 1000
the events
 Retrieve
the x and y speeds
26
Copyright 2014 by Janson Industries
VelocityEx
package my.touch.com;
import
import
import
import
import
android.content.Context;
android.util.AttributeSet;
android.view.MotionEvent;
android.view.VelocityTracker;
android.view.View;
public class VelocityEx extends View {
private VelocityTracker vTracker = null;
public VelocityEx(Context context, AttributeSet attrs) {
super(context, attrs);
}
27
Copyright 2014 by Janson Industries
VelocityEx
public boolean onTouchEvent(MotionEvent event) {
int action = event.getAction();
switch (action) {
case 0:
if (vTracker == null) {
//Get the VelocityTracker object or clear out old data
vTracker = VelocityTracker.obtain();
} else {
vTracker.clear();
}
//Feed initial click down event to set starting point
vTracker.addMovement(event);
break;
Copyright 2014 by Janson Industries
28
VelocityEx
case 2:
vTracker.addMovement(event);
// Sets speed per second
vTracker.computeCurrentVelocity(1000);
// Displays the x and y velocity
System.out.println("X velocity is " +
vTracker.getXVelocity() + " pixels per second");
System.out.println("Y velocity is " +
vTracker.getYVelocity() + " pixels per second");
break;
}
return (true);
}}
29
Copyright 2014 by Janson Industries
VelocityEx

Change activity_main.xml to
display VelocityEx
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout
xmlns:android="http://schemas.android.com/apk/res/android"
android:orientation="vertical"
android:layout_width="fill_parent"
android:layout_height="fill_parent">
<my.touch.com.VelocityEx
android:id="@+id/ve"
android:layout_width="wrap_content"
android:layout_height="wrap_content" />
</LinearLayout>
30
Copyright 2014 by Janson Industries
When run
31
Copyright 2014 by Janson Industries
Touch

Can put listener on any view
subclass
onTouch
method will be invoked

Add a TextView to the layout

Main activity must
 Implement
onTouchListener
 Get
the text view and add the
listener to it
32
Copyright 2014 by Janson Industries
Touch

New TextView
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout
xmlns:android="http://schemas.android.com/apk/res/android"
android:orientation="vertical"
android:layout_width="fill_parent"
android:layout_height="fill_parent">
<TextView android:layout_height="wrap_content"
android:text="Example text in text view"
android:id="@+id/tV"
android:layout_width="wrap_content"
android:textSize="35dp"></TextView>
<my.touch.com.VelocityEx
android:id="@+id/ve"
android:layout_width="wrap_content"
android:layout_height="wrap_content" />
33
</LinearLayout>
Copyright 2014 by Janson Industries
:
:
:
import android.view.MotionEvent;
import android.view.View;
import android.view.View.OnTouchListener;
import android.widget.TextView;
Touch
public class Main extends ActionBarActivity implements
OnTouchListener {
TextView tv;
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
//TextView retrieved and listener added
tv = (TextView)this.findViewById(R.id.tV);
tv.setOnTouchListener(this);
}
public boolean onTouch(View v, MotionEvent event) {
System.out.println("**** Main's onTouch invoked");
return false;
}
34
:
:
:
Copyright 2014 by Janson Industries
onTouch & onTouchEvent
▀
Must return a Boolean value
▀
If it returns true, it means:
 The
action has been consumed
 No other view needs to be notified
▀
If it returns false, it means:
 The
action has not been consumed
 The method is not interested in any
other future events for this action
 Other views should be notified
35
Copyright 2014 by Janson Industries
onTouch returns false so if we
click, drag, and then release on the
TextView, only one message printed
If clicked again, one message will be
displayed because it’s a new action
36
Copyright 2014 by Janson Industries
onTouch & onTouchEvent
▀
▀
But specifying true also has
implications
We will put a checkbox on the
main screen
<CheckBox android:id="@+id/sampleCB" android:text="example"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:textSize="25dp"/>
▀
onTouch returns false and the
checkbox works like normal
37
Copyright 2014 by Janson Industries
Clicked and released and two messages are
displayed
38
Copyright 2014 by Janson Industries
CheckBox is checked
Copyright 2014 by Janson Industries
39
Change it to true, click and release, and
two messages are displayed
40
Copyright 2014 by Janson Industries
But the CheckBox is not checked.
True says don’t notify other View methods,
in this case, the CheckBox’s onTouchEvent
method which displays the green check
mark and changes the state
41
Copyright 2014 by Janson Industries
Can sense pressure and size of touch but
must run on a device (not emulator)
public boolean onTouch(View v, MotionEvent event) {
tv.setText("Pressure: " + String.valueOf(event.getPressure())
+"
Size: " + String.valueOf(event.getSize()));
System.out.println("**** Main's onTouch invoked");
return true;
}
42
Copyright 2014 by Janson Industries
43
Copyright 2014 by Janson Industries
Assignment – Part 2
▀
▀
In Square, make the square size
relative to the size and pressure
of initial touch
Make sure square is easy to see
regardless of size and pressure
►Please,
no microscopic squares
44
Copyright 2014 by Janson Industries
MultiTouch
▀
▀
Complicated and doesn’t always
work the same way on different
devices and Android
implementations
For example, getPointerCount
returns the number of fingers
touching the screen
 On
some devices it only reports
some of the fingers!
45
Copyright 2014 by Janson Industries
MultiTouch
▀
▀
Assuming getPointerCount
returned 3, the three touches are
referenced by a pointer index
In this case, the pointer index has
pointer ids of 0, 1, and 2 located
at index locations 0, 1 , and 2
 Calls
to get info about a touch must
include a pointer index location
► getX(pointerIndex);
46
Copyright 2014 by Janson Industries
MultiTouch
▀
In addition when the second
finger touch occurs the action
code is 261
A
▀
third finger 517
Android returns the pointer index
(1) and action code (5) for an
additional touch
A
hex 0105 equals 261
► Third
finger is 2 and 5, 0205 = 517.
47
Copyright 2014 by Janson Industries
MultiTouch
▀
To prove must create new project
with latest build level
 1.6
▀
doesn’t support getPointerCount
Will print out the action code and
the count of number of touches
 Will
touch the screen with 3 fingers
one after another
48
Copyright 2014 by Janson Industries
package com.example.multitouchproj;
import
import
import
import
Touch
android.content.Context;
android.util.AttributeSet;
android.view.MotionEvent;
android.view.View;
public class MultiTouchTest extends View {
int action, count;
public MultiTouchTest(Context context, AttributeSet attrs) {
super(context);
}
public boolean onTouchEvent(MotionEvent event) {
action = event.getAction();
count = event.getPointerCount();
System.out.println("Action: " + action);
System.out.println("Count: " + count);
return true;
}
}
Copyright 2014 by Janson Industries
49
Initial touch
and count
2nd touch
and count
3rd touch
and count
50
Copyright 2014 by Janson Industries
MultiTouch
▀
▀
If the first finger gets removed
the pointer ids are shifted left
in the index
 Index
0 holds pointer id 1
 Index
1 holds pointer id 2
If the first finger touches again
it is assigned pointer id 0 in
index 0
51
Copyright 2014 by Janson Industries
:
:
:
public class MultiTouchTest extends View {
int action, count;
float xLoc, yLoc;
:
:
:
public boolean onTouchEvent(MotionEvent event) {
action = event.getAction();
count = event.getPointerCount();
System.out.println("Action: " + action);
System.out.println("Count: " + count);
for (int counter = 0; counter < count; counter++){
xLoc = event.getX(counter);
yLoc = event.getY(counter);
System.out.println("Location of " + counter + "
touch x:" + xLoc + " y:" + yLoc);
}
return true;
}
}
Touch
52
Copyright 2014 by Janson Industries
MultiTouch
▀
▀
Will touch once then twice and
then remove first touch
Notice what index is displayed
and what the location is
53
Copyright 2014 by Janson Industries
1st
2nd
2nd
Removed first finger
Copyright 2014 by Janson Industries
54
MultiTouch
▀
Keeping track complicated!
 Thank
▀
the Lord for gestures
Gestures are recognizable
motions
 Been
around a long time
55
Copyright 2014 by Janson Industries
Gestures
▀
▀
Pinch not
supported until
2.2
Create a new
project
(GestureProj),
package
(my.gestures.c
om), and set
build target to
2.2
56
Copyright 2014 by Janson Industries
Gestures
▀
Specify
 Activity
name
 Layout
name
57
Copyright 2014 by Janson Industries
Gestures
▀
Recognizable recorded motions
▀
Can be single or multi-stroke
▀
▀
When defining, the order of the
strokes make a difference
Creating a T gesture like this
Different than this
58
Copyright 2014 by Janson Industries
Gestures
▀
▀
▀
▀
Can create new ones with
Gestures Builder
Start the emulator and display
all applications (center icon)
Double click
A blank screen with two buttons
on the bottom will be displayed
59
Copyright 2014 by Janson Industries
60
Copyright 2014 by Janson Industries
Gestures Builder
▀
If it’s not on your device,
download it from
http://code.google.com/p/quickdroid/downloads/detail?
name=com.android.gesture.builder.apk&can=2&q
▀
On phone need to change
download settings to allow
unknown source
 Settings,
Security, tap “Unknown
sources”
61
Copyright 2014 by Janson Industries
62
Copyright 2014 by Janson Industries
Gestures
▀
▀
Click the Add gesture button
and move the cursor in the
shape of the gesture you want
to create
In the data entry field called
name, type in a name for the
gesture and click Done
 Location
where it's stored will be
displayed
► mnt/sdcard/gestures
63
Copyright 2014 by Janson Industries
64
Copyright 2014 by Janson Industries
65
Copyright 2014 by Janson Industries
Gestures
▀
▀
If you don’t like the gesture,
click the discard button and try
again
You can record many gestures
with the same name
That’s
how you capture all the
different ways of drawing a
gesture
66
Copyright 2014 by Janson Industries
Once saved the gestures will be listed
Click/tap and hold a gesture to see a menu
of functions that can be performed against
one (delete, rename)
67
Copyright 2014 by Janson Industries
Deciphering Gestures
▀
▀
Need a GestureOverlayView on the
screen
Class must
 Implement
OnGesturePerformedListener
 Add
listener to the GestureOverlayView
 Have
an onGesturePerformed method
► Will
be invoked when a gesture is
performed
68
Copyright 2014 by Janson Industries
Deciphering Gestures
▀
▀
onGesturePerformed will be
passed a Gesture object
Class needs a GestureLibrary
 Accepts
a gesture object
 Compares
it to all the recorded
gestures
 Returns
an ArrayList of predictions
► Prediction
consists of the gesture
name and a score
69
Copyright 2014 by Janson Industries
Deciphering Gestures
▀
▀
The array list has the predictions
in order from most to least likely
So the gesture in position 0 of
the ArrayList is what the system
thinks is the correct gesture
70
Copyright 2014 by Janson Industries
Gestures
If multi-stroke
possible must
specify
<?xml version="1.0" encoding="utf-8"?>
<!-- This file is /res/layout/main.xml -->
<LinearLayout
xmlns:android="http://schemas.android.com/apk/res/android"
android:orientation="vertical"
android:layout_width="fill_parent"
android:layout_height="fill_parent">
<TextView android:layout_width="fill_parent"
android:layout_height="wrap_content"
android:text="Draw gestures and I'll guess what they are"
/>
<android.gesture.GestureOverlayView
android:id="@+id/gestureOverlay"
android:layout_width="fill_parent"
android:layout_height="fill_parent"
android:gestureStrokeType="multiple"
android:fadeOffset="1000" />
71
</LinearLayout>
Copyright 2014 by Janson Industries
Gestures
package my.gestures.com;
import java.util.ArrayList;
import android.app.Activity;
import android.gesture.Gesture;
import android.gesture.GestureLibraries;
import android.gesture.GestureLibrary;
import android.gesture.GestureOverlayView;
import android.gesture.Prediction;
import
android.gesture.GestureOverlayView.OnGesturePerformedListener;
import android.os.Bundle;
import android.util.Log;
import android.widget.Toast;
public class GestureProjActivity extends Activity implements
OnGesturePerformedListener {
72
Copyright 2014 by Janson Industries
Gestures
GestureLibrary gestureLib = null;
Prediction prediction;
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.main_gesture);
gestureLib = GestureLibraries.fromFile("/sdcard/gestures");
if (!gestureLib.load()) {
Toast.makeText(this, "Could not load /sdcard/gestures",
Toast.LENGTH_SHORT).show();
finish();
}
GestureOverlayView gestureView = (GestureOverlayView)
findViewById(R.id.gestureOverlay);
gestureView.addOnGesturePerformedListener(this);
}
Copyright 2014 by Janson Industries
73
Deciphering Gestures
▀
If you want to package the
gestures with your app
 Create
 Copy
the raw folder in res
the gestures file into raw
 Create
the GestureLibrary as
follows
gestureLib = GestureLibraries.fromRawResource(this,
R.raw.gestures);
► Instead
of creating it from sdcard
74
Copyright 2014 by Janson Industries
75
Copyright 2014 by Janson Industries
76
Copyright 2014 by Janson Industries
Gestures
public void onGesturePerformed(GestureOverlayView view,
Gesture gesture) {
ArrayList<Prediction> predictions =
gestureLib.recognize(gesture);
if (predictions.size() > 0) {
prediction = predictions.get(0);
if (prediction.score > 1.0) {
Toast.makeText(this, prediction.name,
Toast.LENGTH_SHORT).show();
}
}
}
Toast will display the
}
gesture name
77
Copyright 2014 by Janson Industries
Initial screen
78
Copyright 2014 by Janson Industries
79
Copyright 2014 by Janson Industries
80
Copyright 2014 by Janson Industries
Gestures

Too see all the predictions
scores, add this after the
statement that displays the toast
for (int i = 0; i < predictions.size(); i++) {
System.out.println("prediction " + predictions.get(i).name
+ "'s score = " + predictions.get(i).score);
}
81
Copyright 2014 by Janson Industries