Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Mobile Computing Advanced Touching Assg Part1 AssgPart2 1 Copyright 2014 by Janson Industries Objectives ▀ Explain Motion events Multi-touch Gestures Creating gestures 2 Copyright 2014 by Janson Industries Hardware ▀ The touch screen is constructed to pick up pressure on the screen and convert that to data Screen coordinates Pressure Size of touch Time ▀ All info stored in a MotionEvent object 3 Copyright 2014 by Janson Industries Motion Sequence Actions ▀ The action property holds value that identifies action type action = 0 (ACTION_DOWN) is the initial touch action = 2 (ACTION_MOVE) any up, down, or sideways movement on the screen action = 1 (ACTION_UP) is when finger is lifted off the screen 4 Copyright 2014 by Janson Industries Motion Sequence onTouch or onTouchEvent method called when motion action occurs A MotionEvent object passed to these methods Copyright 2014 by Janson Industries For classes that implement onTouchListener, onTouch called For subclasses of View, onTouchEvent called 5 Example Will create a view subclass called TouchTest Its onTouchEvent will get info from the MotionEvent object and display in LogCat Will also use the switch structure 6 Copyright 2014 by Janson Industries Nested if/else ▮ Placing an if statement as one of the statements to be executed in an if/else clause if (month == 1) { monthTV.setText(“Jan”);} Nested if else {if (month == 2) { monthTV.setText(“Feb”);} else { if (month == 3) { monthTV.setText(“Mar”);} else { if (month == 4) {……… 7 Copyright 2014 by Janson Industries Switch ▮ Instead of complicated nested ifs, can use a switch switch (Month) { case 1: monthTV.setText(“Jan”); case 2: monthTV.setText(“Feb”); case 3: monthTV.setText(“Mar”); : : : : : : : : : : default: System.out.println(“Not month!”); } break; break; break; a valid 8 Copyright 2014 by Janson Industries Switch ▮ Need the break statements because once the condition is true, all subsequent statements are executed ▮ In the example, this means ▮ The label is set to Dec ▮ And the “Not a valid month” message displayed 9 Copyright 2014 by Janson Industries TouchTest TouchTest (a subclass of View) will: Check for touches When a touch occurs, display a variety of info about the touch 10 Copyright 2014 by Janson Industries TouchTest Created new TouchProj project, my.touch.com package and Main activity, then create new class TouchTest package my.touch.com; import android.content.Context; import android.util.AttributeSet; import android.view.MotionEvent; import android.view.View; public class TouchTest extends View { public TouchTest(Context context, AttributeSet attrs) { super(context); } 11 Copyright 2014 by Janson Industries TouchTest public boolean onTouchEvent(MotionEvent event) { int action = event.getAction(); switch (action) { case 0: System.out.println("Action: " + event.getAction()); System.out.println("Location: " + event.getX() + " x " + event.getY() + " y"); System.out.println("Pressure: " + event.getPressure()); System.out.println("Size: " + event.getSize()); System.out.println("Down time: " + event.getDownTime() + " ms"); System.out.println("Event time: " + event.getEventTime() + " ms"); System.out.println("Elapsed: " + (event.getEventTime() - event.getDownTime()) + " ms"); break; 12 Copyright 2014 by Janson Industries TouchTest case 1: System.out.println("Action: " + event.getAction()); System.out.println("Location: " + event.getX() + " x " + event.getY() + " y"); System.out.println("Pressure: " + event.getPressure()); System.out.println("Size: " + event.getSize()); System.out.println("Down time: " + event.getDownTime() + " ms"); System.out.println("Event time: " + event.getEventTime() + " ms"); System.out.println("Elapsed: " + (event.getEventTime() - event.getDownTime()) + " ms"); break; } return (true); } 13 Copyright 2014 by Janson Industries } main.xml Change activity_main.xml to add TouchTest to main screen <?xml version="1.0" encoding="utf-8"?> <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android" android:orientation="vertical" android:layout_width="fill_parent" android:layout_height="fill_parent"> <my.touch.com.TouchTest android:id="@+id/tt" android:layout_width="wrap_content" android:layout_height="wrap_content" /> </LinearLayout> 14 Copyright 2014 by Janson Industries TouchTest Run Click & hold Release 15 Copyright 2014 by Janson Industries Touch Test 02-28 13:42:54.086: I/System.out(18244): Action: 0 02-28 13:42:54.086: I/System.out(18244): Location: 325.0 x 500.0 y 02-28 13:42:54.097: I/System.out(18244): Pressure: 1.0 02-28 13:42:54.097: I/System.out(18244): Size: 0.0 02-28 13:42:54.127: I/System.out(18244): Down time: 684320130 ms 02-28 13:42:54.127: I/System.out(18244): Event time: 684320130 ms 02-28 13:42:54.167: I/System.out(18244): Elapsed: 0 ms 02-28 13:42:56.106: I/System.out(18244): Action: 1 02-28 13:42:56.106: I/System.out(18244): Location: 325.0 x 500.0 y 02-28 13:42:56.117: I/System.out(18244): Pressure: 1.0 02-28 13:42:56.131: I/System.out(18244): Size: 0.0 02-28 13:42:56.131: I/System.out(18244): Down time: 684320130 ms 02-28 13:42:56.131: I/System.out(18244): Event time: 684322150 ms 02-28 13:42:56.131: I/System.out(18244): Elapsed: 2020 ms 16 Copyright 2014 by Janson Industries Touch Test In emulator, pressure and size will always be 1 and 0 If you click and drag, a series of move actions (2) will be generated We’ll add a check for an action 2 in the switch 17 Copyright 2014 by Janson Industries TouchTest Took out the pressure and size displays case 2: System.out.println("Action: " + event.getAction()); System.out.println("Location: " + event.getX() + " x " + event.getY() + " y"); System.out.println("Down time: " + event.getDownTime() + " ms"); System.out.println("Event time: " + event.getEventTime() + " ms"); System.out.println("Elapsed: " + (event.getEventTime() - event.getDownTime()) + " ms"); Click and drag quickly to right 18 Copyright 2014 by Janson Industries Touch Test 02-28 13:48:09.017: I/System.out(18304): Action: 2 02-28 13:48:09.017: I/System.out(18304): Location: 294.843 x 423.79776 y 02-28 13:48:09.037: I/System.out(18304): Down time: 684633418 ms 02-28 13:48:09.037: I/System.out(18304): Event time: 684635056 ms 02-28 13:48:09.037: I/System.out(18304): Elapsed: 1638 ms 02-28 13:48:09.129: I/System.out(18304): Action: 2 02-28 13:48:09.129: I/System.out(18304): Location: 264.81342 x 436.74582 y 02-28 13:48:09.137: I/System.out(18304): Down time: 684633418 ms 02-28 13:48:09.137: I/System.out(18304): Event time: 684635169 ms 02-28 13:48:09.137: I/System.out(18304): Elapsed: 1751 ms 02-28 13:48:09.150: I/System.out(18304): Action: 2 02-28 13:48:09.157: I/System.out(18304): Location: 262.70416 x 437.40497 y 02-28 13:48:09.157: I/System.out(18304): Down time: 684633418 ms 02-28 13:48:09.157: I/System.out(18304): Event time: 684635179 ms 02-28 13:48:09.169: I/System.out(18304): Elapsed: 1761 ms 19 Copyright 2014 by Janson Industries Implementing Touch Create a View subclass called Square When screen touched a square will appear around the touch If movement, square will follow the movement When touch lifted square disappears 20 Copyright 2014 by Janson Industries Square package my.touch.com; import import import import import import import android.content.Context; android.graphics.Canvas; android.graphics.Color; android.graphics.Paint; android.util.AttributeSet; android.view.MotionEvent; android.view.View; public class Square extends View { private float x = -30; // Set initial location off screen private float y = -30; private Paint myPaint; public Square(Context context, AttributeSet attrs) { super(context, attrs); myPaint = new Paint(); myPaint.setColor(Color.BLUE); } 2014 by Janson Industries Copyright 21 public boolean onTouchEvent(MotionEvent event) { int action = event.getAction(); switch (action) { case MotionEvent.ACTION_DOWN: // In both cases these case MotionEvent.ACTION_MOVE: // statements are run x = event.getX(); y = event.getY(); break; case MotionEvent.ACTION_UP: x = -30; // set the location outside the screen area y = -30; break; } return (true); } Square public void draw(Canvas canvas) { canvas.drawRect(x-15, y-15, x+15, y+15, myPaint); invalidate(); } Copyright 2014 by Janson Industries } 22 activity_main.xml <?xml version="1.0" encoding="utf-8"?> <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android" android:orientation="vertical" android:layout_width="fill_parent" android:layout_height="fill_parent"> <my.touch.com.Square android:id="@+id/sq" android:layout_width="wrap_content" android:layout_height="wrap_content" /> </LinearLayout> 23 Copyright 2014 by Janson Industries Click Drag Release 24 Copyright 2014 by Janson Industries Assignment - Part 1 Create Square such that the square is dragged around (as shown earlier) But when released, square returns to the original clicked location 25 Copyright 2014 by Janson Industries Velocity Can measure using a VelocityTracker VelocityTracker initially obtained then Specify pixels per time period in milliseconds Per Feed second means 1000 the events Retrieve the x and y speeds 26 Copyright 2014 by Janson Industries VelocityEx package my.touch.com; import import import import import android.content.Context; android.util.AttributeSet; android.view.MotionEvent; android.view.VelocityTracker; android.view.View; public class VelocityEx extends View { private VelocityTracker vTracker = null; public VelocityEx(Context context, AttributeSet attrs) { super(context, attrs); } 27 Copyright 2014 by Janson Industries VelocityEx public boolean onTouchEvent(MotionEvent event) { int action = event.getAction(); switch (action) { case 0: if (vTracker == null) { //Get the VelocityTracker object or clear out old data vTracker = VelocityTracker.obtain(); } else { vTracker.clear(); } //Feed initial click down event to set starting point vTracker.addMovement(event); break; Copyright 2014 by Janson Industries 28 VelocityEx case 2: vTracker.addMovement(event); // Sets speed per second vTracker.computeCurrentVelocity(1000); // Displays the x and y velocity System.out.println("X velocity is " + vTracker.getXVelocity() + " pixels per second"); System.out.println("Y velocity is " + vTracker.getYVelocity() + " pixels per second"); break; } return (true); }} 29 Copyright 2014 by Janson Industries VelocityEx Change activity_main.xml to display VelocityEx <?xml version="1.0" encoding="utf-8"?> <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android" android:orientation="vertical" android:layout_width="fill_parent" android:layout_height="fill_parent"> <my.touch.com.VelocityEx android:id="@+id/ve" android:layout_width="wrap_content" android:layout_height="wrap_content" /> </LinearLayout> 30 Copyright 2014 by Janson Industries When run 31 Copyright 2014 by Janson Industries Touch Can put listener on any view subclass onTouch method will be invoked Add a TextView to the layout Main activity must Implement onTouchListener Get the text view and add the listener to it 32 Copyright 2014 by Janson Industries Touch New TextView <?xml version="1.0" encoding="utf-8"?> <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android" android:orientation="vertical" android:layout_width="fill_parent" android:layout_height="fill_parent"> <TextView android:layout_height="wrap_content" android:text="Example text in text view" android:id="@+id/tV" android:layout_width="wrap_content" android:textSize="35dp"></TextView> <my.touch.com.VelocityEx android:id="@+id/ve" android:layout_width="wrap_content" android:layout_height="wrap_content" /> 33 </LinearLayout> Copyright 2014 by Janson Industries : : : import android.view.MotionEvent; import android.view.View; import android.view.View.OnTouchListener; import android.widget.TextView; Touch public class Main extends ActionBarActivity implements OnTouchListener { TextView tv; public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); //TextView retrieved and listener added tv = (TextView)this.findViewById(R.id.tV); tv.setOnTouchListener(this); } public boolean onTouch(View v, MotionEvent event) { System.out.println("**** Main's onTouch invoked"); return false; } 34 : : : Copyright 2014 by Janson Industries onTouch & onTouchEvent ▀ Must return a Boolean value ▀ If it returns true, it means: The action has been consumed No other view needs to be notified ▀ If it returns false, it means: The action has not been consumed The method is not interested in any other future events for this action Other views should be notified 35 Copyright 2014 by Janson Industries onTouch returns false so if we click, drag, and then release on the TextView, only one message printed If clicked again, one message will be displayed because it’s a new action 36 Copyright 2014 by Janson Industries onTouch & onTouchEvent ▀ ▀ But specifying true also has implications We will put a checkbox on the main screen <CheckBox android:id="@+id/sampleCB" android:text="example" android:layout_width="wrap_content" android:layout_height="wrap_content" android:textSize="25dp"/> ▀ onTouch returns false and the checkbox works like normal 37 Copyright 2014 by Janson Industries Clicked and released and two messages are displayed 38 Copyright 2014 by Janson Industries CheckBox is checked Copyright 2014 by Janson Industries 39 Change it to true, click and release, and two messages are displayed 40 Copyright 2014 by Janson Industries But the CheckBox is not checked. True says don’t notify other View methods, in this case, the CheckBox’s onTouchEvent method which displays the green check mark and changes the state 41 Copyright 2014 by Janson Industries Can sense pressure and size of touch but must run on a device (not emulator) public boolean onTouch(View v, MotionEvent event) { tv.setText("Pressure: " + String.valueOf(event.getPressure()) +" Size: " + String.valueOf(event.getSize())); System.out.println("**** Main's onTouch invoked"); return true; } 42 Copyright 2014 by Janson Industries 43 Copyright 2014 by Janson Industries Assignment – Part 2 ▀ ▀ In Square, make the square size relative to the size and pressure of initial touch Make sure square is easy to see regardless of size and pressure ►Please, no microscopic squares 44 Copyright 2014 by Janson Industries MultiTouch ▀ ▀ Complicated and doesn’t always work the same way on different devices and Android implementations For example, getPointerCount returns the number of fingers touching the screen On some devices it only reports some of the fingers! 45 Copyright 2014 by Janson Industries MultiTouch ▀ ▀ Assuming getPointerCount returned 3, the three touches are referenced by a pointer index In this case, the pointer index has pointer ids of 0, 1, and 2 located at index locations 0, 1 , and 2 Calls to get info about a touch must include a pointer index location ► getX(pointerIndex); 46 Copyright 2014 by Janson Industries MultiTouch ▀ In addition when the second finger touch occurs the action code is 261 A ▀ third finger 517 Android returns the pointer index (1) and action code (5) for an additional touch A hex 0105 equals 261 ► Third finger is 2 and 5, 0205 = 517. 47 Copyright 2014 by Janson Industries MultiTouch ▀ To prove must create new project with latest build level 1.6 ▀ doesn’t support getPointerCount Will print out the action code and the count of number of touches Will touch the screen with 3 fingers one after another 48 Copyright 2014 by Janson Industries package com.example.multitouchproj; import import import import Touch android.content.Context; android.util.AttributeSet; android.view.MotionEvent; android.view.View; public class MultiTouchTest extends View { int action, count; public MultiTouchTest(Context context, AttributeSet attrs) { super(context); } public boolean onTouchEvent(MotionEvent event) { action = event.getAction(); count = event.getPointerCount(); System.out.println("Action: " + action); System.out.println("Count: " + count); return true; } } Copyright 2014 by Janson Industries 49 Initial touch and count 2nd touch and count 3rd touch and count 50 Copyright 2014 by Janson Industries MultiTouch ▀ ▀ If the first finger gets removed the pointer ids are shifted left in the index Index 0 holds pointer id 1 Index 1 holds pointer id 2 If the first finger touches again it is assigned pointer id 0 in index 0 51 Copyright 2014 by Janson Industries : : : public class MultiTouchTest extends View { int action, count; float xLoc, yLoc; : : : public boolean onTouchEvent(MotionEvent event) { action = event.getAction(); count = event.getPointerCount(); System.out.println("Action: " + action); System.out.println("Count: " + count); for (int counter = 0; counter < count; counter++){ xLoc = event.getX(counter); yLoc = event.getY(counter); System.out.println("Location of " + counter + " touch x:" + xLoc + " y:" + yLoc); } return true; } } Touch 52 Copyright 2014 by Janson Industries MultiTouch ▀ ▀ Will touch once then twice and then remove first touch Notice what index is displayed and what the location is 53 Copyright 2014 by Janson Industries 1st 2nd 2nd Removed first finger Copyright 2014 by Janson Industries 54 MultiTouch ▀ Keeping track complicated! Thank ▀ the Lord for gestures Gestures are recognizable motions Been around a long time 55 Copyright 2014 by Janson Industries Gestures ▀ ▀ Pinch not supported until 2.2 Create a new project (GestureProj), package (my.gestures.c om), and set build target to 2.2 56 Copyright 2014 by Janson Industries Gestures ▀ Specify Activity name Layout name 57 Copyright 2014 by Janson Industries Gestures ▀ Recognizable recorded motions ▀ Can be single or multi-stroke ▀ ▀ When defining, the order of the strokes make a difference Creating a T gesture like this Different than this 58 Copyright 2014 by Janson Industries Gestures ▀ ▀ ▀ ▀ Can create new ones with Gestures Builder Start the emulator and display all applications (center icon) Double click A blank screen with two buttons on the bottom will be displayed 59 Copyright 2014 by Janson Industries 60 Copyright 2014 by Janson Industries Gestures Builder ▀ If it’s not on your device, download it from http://code.google.com/p/quickdroid/downloads/detail? name=com.android.gesture.builder.apk&can=2&q ▀ On phone need to change download settings to allow unknown source Settings, Security, tap “Unknown sources” 61 Copyright 2014 by Janson Industries 62 Copyright 2014 by Janson Industries Gestures ▀ ▀ Click the Add gesture button and move the cursor in the shape of the gesture you want to create In the data entry field called name, type in a name for the gesture and click Done Location where it's stored will be displayed ► mnt/sdcard/gestures 63 Copyright 2014 by Janson Industries 64 Copyright 2014 by Janson Industries 65 Copyright 2014 by Janson Industries Gestures ▀ ▀ If you don’t like the gesture, click the discard button and try again You can record many gestures with the same name That’s how you capture all the different ways of drawing a gesture 66 Copyright 2014 by Janson Industries Once saved the gestures will be listed Click/tap and hold a gesture to see a menu of functions that can be performed against one (delete, rename) 67 Copyright 2014 by Janson Industries Deciphering Gestures ▀ ▀ Need a GestureOverlayView on the screen Class must Implement OnGesturePerformedListener Add listener to the GestureOverlayView Have an onGesturePerformed method ► Will be invoked when a gesture is performed 68 Copyright 2014 by Janson Industries Deciphering Gestures ▀ ▀ onGesturePerformed will be passed a Gesture object Class needs a GestureLibrary Accepts a gesture object Compares it to all the recorded gestures Returns an ArrayList of predictions ► Prediction consists of the gesture name and a score 69 Copyright 2014 by Janson Industries Deciphering Gestures ▀ ▀ The array list has the predictions in order from most to least likely So the gesture in position 0 of the ArrayList is what the system thinks is the correct gesture 70 Copyright 2014 by Janson Industries Gestures If multi-stroke possible must specify <?xml version="1.0" encoding="utf-8"?> <!-- This file is /res/layout/main.xml --> <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android" android:orientation="vertical" android:layout_width="fill_parent" android:layout_height="fill_parent"> <TextView android:layout_width="fill_parent" android:layout_height="wrap_content" android:text="Draw gestures and I'll guess what they are" /> <android.gesture.GestureOverlayView android:id="@+id/gestureOverlay" android:layout_width="fill_parent" android:layout_height="fill_parent" android:gestureStrokeType="multiple" android:fadeOffset="1000" /> 71 </LinearLayout> Copyright 2014 by Janson Industries Gestures package my.gestures.com; import java.util.ArrayList; import android.app.Activity; import android.gesture.Gesture; import android.gesture.GestureLibraries; import android.gesture.GestureLibrary; import android.gesture.GestureOverlayView; import android.gesture.Prediction; import android.gesture.GestureOverlayView.OnGesturePerformedListener; import android.os.Bundle; import android.util.Log; import android.widget.Toast; public class GestureProjActivity extends Activity implements OnGesturePerformedListener { 72 Copyright 2014 by Janson Industries Gestures GestureLibrary gestureLib = null; Prediction prediction; public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.main_gesture); gestureLib = GestureLibraries.fromFile("/sdcard/gestures"); if (!gestureLib.load()) { Toast.makeText(this, "Could not load /sdcard/gestures", Toast.LENGTH_SHORT).show(); finish(); } GestureOverlayView gestureView = (GestureOverlayView) findViewById(R.id.gestureOverlay); gestureView.addOnGesturePerformedListener(this); } Copyright 2014 by Janson Industries 73 Deciphering Gestures ▀ If you want to package the gestures with your app Create Copy the raw folder in res the gestures file into raw Create the GestureLibrary as follows gestureLib = GestureLibraries.fromRawResource(this, R.raw.gestures); ► Instead of creating it from sdcard 74 Copyright 2014 by Janson Industries 75 Copyright 2014 by Janson Industries 76 Copyright 2014 by Janson Industries Gestures public void onGesturePerformed(GestureOverlayView view, Gesture gesture) { ArrayList<Prediction> predictions = gestureLib.recognize(gesture); if (predictions.size() > 0) { prediction = predictions.get(0); if (prediction.score > 1.0) { Toast.makeText(this, prediction.name, Toast.LENGTH_SHORT).show(); } } } Toast will display the } gesture name 77 Copyright 2014 by Janson Industries Initial screen 78 Copyright 2014 by Janson Industries 79 Copyright 2014 by Janson Industries 80 Copyright 2014 by Janson Industries Gestures Too see all the predictions scores, add this after the statement that displays the toast for (int i = 0; i < predictions.size(); i++) { System.out.println("prediction " + predictions.get(i).name + "'s score = " + predictions.get(i).score); } 81 Copyright 2014 by Janson Industries