Reading #18: Spatial Recognition and Grouping of Text and Graphics

COMMENTS ON OTHERS:

            Jonathan 

SUMMARY

              The paper proposes a spatial framework for simultaneous grouping and recognition of shapes and symbols in free-form ink diagrams. The recognition is done by linking each stroke into a proximity graph and then using a discriminative classifier to classify connected subgraphs as either known symbol or invalid combination of strokes. In the preprocessing the graph is created with the nodes corresponds to strokes and edges to the strokes in close proximity. Then a dynamic programming approach was used to iterate on nodes and discriminated recognition was applied on the set. The classifier is named as AdaBoost and features are the Viola-Jones image filters to evaluate each stroke group. Authors also note that it is possible to input strokes which may not make up a shape at all called garbage shapes. 

DISCUSSION

            The paper is extremely short and not enough information provided with the recognizer, and in this case the most important part of this work. Much of the discussions in first few chapters are about the previous work, the preprocessing and the search tree on the neighborhood graph making process. The authors mentioned a use of dynamic programming in the content somewhere but not sure A* is the best method to get the optimum results. In my opinion the sketches tend to represent more of a close loop connection of strokes from start to end and this makes it easy to find an optimum path using a forward backward algorithm or using Viterbi.

Reading #17: Distinguishing Text from Graphics in On-line Handwritten Ink

COMMENTS ON OTHERS:

             Wenzhe Li  

SUMMARY

              The authors states that the proposed study utilizes both characteristics of sketch strokes and also information provided by the gaps between the strokes, and temporal characteristics of the stroke sequence. The proposed approach is based on the use of discriminative machine learning techniques to infer the class of the stroke based on the observed ink.

            The proposed model starts with considering the isolated strokes and extracting features then training probabilistic classifier based on feed forward neural network. After that, the model is augmented with temporal information to capture the correlation between class labels. Finally it is proposed that to consider information extracted from the gaps between successive strokes. The model said to be based on 9 features extracted at the independent stroke model. 

DISCUSSION

            The proposed work uses a HMM model to enhance the performance by taking the context for each stroke into account. It is stated that the focus is on the use of temporal context since it leads to a 1D inference problem which can solve efficiently using dynamic programming techniques. The model uses Viterbi algorithm to find the states with the training data to get the most probably selection of the strokes. It is also obvious that the HMM require initial probabilistic information for the HMM model, and it is not clear from the context that how the initial probably set is chosen to train the system. Whether authors used a dummy set of probabilities or heuristic probably was used not really clear.

Reading #16: An Efficient Graph-Based Symbol Recognizer

COMMENTS ON OTHERS:

            JJ

SUMMARY

              The paper proposes a symbol recognizer for pen based user interfaces which the symbols are attributed to relational graphs and best matches of the unknown symbol based on that. The graph matching is done using four techniques, stochastic matching, error driven matching, greedy matching and sort matching.  In an attributed relational graph (ARG) each node represents the geometric primitives, and the edges represent the geometric relationship between them. A definition for a symbol is created by constructing an average ARG from a set of training samples. The proposed recognition applies the ARG of unknown symbol to compare it with ARG of each definition symbol to find a best match based on above defined graph matching techniques. 

DISCUSSION

             The authors claim that stochastic, error-driving matching, and greedy are search based classifiers and Sort is based on orientation fixed method. This means that the other 3 methods are orientation invariant? But the figure 3 actually presents a problem with searching where the orientation brings a dissimilar pair into the matching at the comparison.

Reading #15: An Image-Based, Trainable Symbol Recognizer for Hand-drawn Sketches

COMMENTS ON OTHERS:

           JJ  

SUMMARY

              The paper proposes a trainable, hand-drawn symbol recognizer based on multi layer recognition scheme with the symbols internal representation on a binary template. Ensembles of four different classifiers are used o rank symbols based on similarity to an unknown symbol and the scores are aggregated to produce a combined score. The best score is assigned to the unknown symbol. All four classifiers are template matching techniques to compute the similarity between symbols. The authors used a polar coordinate based technique to compensate the rotation sensitiveness of the template matching technique. The authors state that the proposed system is particularly useful for sketchy inputs like heavy over stroking and erasing due to its binary template approach. 

DISCUSSION

             The authors state that the binary template approach is useful in sketchy overly stroked and erased sketches, but the down sampling and framing it to a 48x48 is questionable whether the best approach. May be authors should experiment more with other techniques like binarizing the ink features and then getting a skeleton of the bits which will actually preserve the sketch input than reducing it.

Reading #14. Using Entropy to Distinguish Shape Versus Text in Hand-Drawn Diagrams

COMMENTS ON OTHERS:

            Jianjie (JJ) Zhang. 

SUMMARY

            The paper proposes a method to distinguish between shape and text strokes based on entropy rate. The authors state that the entropy rate is significantly higher for text strokes compared to shape strokes. They propose a single feature-zero-order entropy rate with a correct classification rate of 92.06% with a trained threshold. The zero-order entropy is determined independently from the previous symbol of the sketch point. The symbol refers to an associated symbol value for each point’s angle between two points of a text or a shape. 

DISCUSSION

             The authors argued that using an arbitrary set of features and trial and error approach is time consuming, which I totally agreed. I had the same in mind when reading the Ink feature based recognition on statistical analysis. Time is something to worry in a sketch system with both text and shapes, not only to distinguish them but also to recognize them from the library sketches.  The proposed system deems to be a simpler way to achieve the text and shape classification, but its still not the best solution when it comes to complex shapes, which quite similar in entropy even to a simple character set.

Reading #13. Ink Features for Diagram Recognition

COMMENTS ON OTHERS:

            Francisco Vides 

SUMMARY

             The paper proposes to use formal statistical analysis methods to identify key ink features to improve recognition. The features measure aspects of an ink stroke’s curvature, size, time, intersections and use similar aspects to detect relationships between strokes. The proposed approach begins with investigating a range of possible ink features, how to collect these feature data and analysis, and initial result of an evaluation of a text/shape divider based on these key ink feature set. The proposed feature set includes 46 features and grouped into 7 categories of size, time, intersection, curvature, pressure, operating system recognition values and inter-stroke gaps. 

DISCUSSION

            Am I missing it or the 46 feature set is not actually listed on the paper or what? Also I’m wondering this particular technique is just to divide the whole sketch system into 2 groups, text or shape, and not to identify further each individual component against a library component? The authors state that they concentrated on identifying the distinguishing features of text versus shape strokes using a formal method for optimal ink feature selection; this means only identifying those stated 2 groups? I kow this is something important when we have a system like COA where possibly a text and shapes mixed with each other, but how feasible approach this is when recognizing something within a time limit say like sixty seconds? We have a recognition engine to separate text and shape and then again apply a recognition engine for each shape and text separately?

Reading #12. Constellation Models for Sketch Recognition

Comments on Others:

Marty

Summary

            This paper describes a constellation or pictorial structure model to recognize strokes in sketches by capturing the structure of a particular class of objects and based on the local features of it. Learning of the system is done by probabilistic model from example sketches with known stroke labeling. Then the recognition algorithm determines a maximum likelihood labeling for an unlabelled sketch by searching through the space of possible label assignments using multi pass branch and bound algorithm. In the model, object representation is done by a constellation model, which based on the features of pairs of parts. Due to the complexity of n number of features, each of these features is broken into mandatory and optional.

            According to the authors, the sketch recognition process has two phases, first to search possible mandatory labels and then optional labels. The searching is based on ML search procedure and search over possible label assignments are done by branch-bound search tree.  Authors proposed a multiple threshold and hard constraints to avoid situations with higher mandatory labels or higher number of strokes.

Discussion

             The idea of using a constellation model for the features of a sketch is an interesting one. I’m just wondering how useful this for our second project to identify course of action diagrams????

            In my opinion restricting a user to draw a sketch based on the assumption that mandatory and optional features existence is sort of a constraining users freehand sketching. Yes, this makes the life much easier for the authors by making the searching is less complex, but I guess they are forgetting the golden rule of sketching recognition……….”freehand sketching”……….. 

Find the paper here.   

Reading #11. LADDER, a sketching language for user interface developers

Comments on Others:

 Hong-Hoe (Ayden) Kim 

Summary

             LADDER is a language to describe how sketched diagrams in a domain are drawn, displayed and edited. According to authors, LADDER structural descriptions can be automatically transform into domain specific shape recognizers, editing recognizers and shape exhibitors to use for sketch recognition domain. Furthermore, authors state that the LADDER is the first sketch description language as well as first implemented prototype system to prove that such a framework is possible to automatically generate a sketch interface for a domain from only domain description. Entire LADDER system consists of 3 sub systems, Domain Description, Translation and Sketch Recognition System. Authors also state that a language is a combination of predefined shapes, constraints, editing behaviors, display methods and domain description syntax. Still the domain descriptions are easy to specify and have enough details for accurate sketch recognition.

            LADDER is defined upon a set of shapes due to description limitations, things like abstract shapes, overly complicated irregular shapes, shapes not based on LADDER primitives, and domains with few curves or less details of curves. The LADDER system is based on vectors to define variable number of components. 

Discussion

            I’m not sure why there is a restriction to the shapes based on other than primitives defined on the LADDER? May be I’m not getting the concept right, are there any other primitives than a point, line and arc (in my opinion)? We should be able to define any other shape by simply using only 3 of these primitives, and I vaguely remember using only these 3 to do graphics programming during my undergraduate years. Correct me if I’m wrong…….      

Find the paper here.   

Reading #10. Graphical Input Through Machine Recognition of Sketches

Comments on Others:

Danielle

Summary

             This paper proposed an interactive system for graphical input in which the user openly participate in training the machine and massaging the data at all levels of interpretation. This includes 3 experiments, HUNCH: which tries to answer the question of existence of a syntax for sketching, architectural knowledge in recognizing a sketch and user involvement in the input process.

            HUNCH is a set of FORTRAN programs with different level of interpretation to process freehand sketches drawn with a data tablet or a light pen. The HUNCH used a program called a STRAIT as a corner finder, which actually based on the function of speed of the pen. Another program called CURVIT was used to finder the curves instead of corners. Even with the sketch recognition capabilities of the CURVIT and STRAIT, authors found that there is a difference when different people using the system which based on the human sketching behavior. Due to the latching problems the STRAIT was re-written without it called STRAIN. Overtracing was another interpretation problem that authors faced during the HUNCH development. The newest approach was using a much more interactive system to get user involvement to make decisions on machine selection which consisted of a database to and programs to manipulate data. The program is based on inference making procedures which is different from HUNCH and STRAIN and LATCH.

Discussion

              The paper is about a GUI to input and recognizer freehand sketches, but overall picture of the system is too hazy to understand. Author starts with a legacy system they used before the new system and the difficulties with the old system and how they overcome those in the new system. Without some performance measures and an overall picture of the system, it is hard to imagine how this new system works towards sketch recognition. I’m lost somewhere in the middle of the paper……………..!!! 

Find the paper here.

Reading #9. PaleoSketch: Accurate Primitive Sketch Recognition and Beautification

Comments on Others:

Francisco

Summary
           The paper proposes a new low-level recognition and beautification system to recognize 8 primitive shapes as well as combinations of these primitives with recognition rate at 98.56%. The Paleo process initiates with a pre-recognition calculation and after that sending these to lower-level shape recognizes for further processing. Each of these low-level recognizer correspond to a particular primitive shape then returns a Boolean flag to specify whether the recognizer passed or failed as well as a beautified shape object that best fits the input stroke. After all shapes are executed, are hierarchy function sorts each interpretation of the order of a best fit. The pre-recogntion eliminates duplicate points and then create series of graphs including directional graph, speed graph, curvature graph and corners based on a simple corner finder algorithm. In addition this phase computes 2 new features called NDDE and DCR to defer polylines from curves.  

Discussion 
             After using the Paleo in our first class project, I’m much satisfied with its processing capabilities w.r.t sketch recognition. It’s a powerful corner finder tool to get the basic primitive shapes of a sketch that you can apply towards devising other sketch recognition algorithms, (which makes life much easier). With my personal experience on hand writing recognition, I feel Paleo is a successful way of doing the preprocessing of an image for other low level tasks. I used to apply image processing techniques like binarization and skelatanizaton and then fuzzy rules to get primitive shapes and, yes, things are daunting that way.
           I’m just wondering whether Paleo is capable of giving few more of feature properties of interests??? Things like positive/negative slanted line, horizontal/vertical line, U-like, inverse U-like, V-like and inverse V like??? I’m not sure paleo is already doing this or not, but if so, things are pretty good for a new character recognition pre-processing task using PaleoSketch………..

Find the paper here. 

Sokushinbutsu (Self-Mummification) - Buddhism, 1/1000 ways to die………….

I thought this self immolation video is too emotional, (it was done to protest and bring attention to the world the prosecution of Buddhists), how WRONG AM I?????

Sokushinbutsu were Buddhist monks or priests who caused their own deaths in a way that resulted in their mummification. This practice reportedly took place almost exclusively in northern Japan around the Yamagata Prefecture. It is believed that many hundreds of monks tried, but only between 16 and 24 such mummifications have been discovered to date. The practice is not advocated or practiced today by any Buddhist sect. The practice was thought to be extinct in modern Japan, but a recent example was discovered in Tokyo in July 2010.

For 1,000 days (a little less than three years) the priests would eat a special diet consisting only of nuts and seeds, while taking part in a regimen of rigorous physical activity that stripped them of their body fat. They then ate only bark and roots for another thousand days and began drinking a poisonous tea made from the sap of the Urushi tree, normally used to lacquer bowls.

This caused vomiting and a rapid loss of bodily fluids, and most importantly, it made the body too poisonous to be eaten by maggots. Finally, a self-mummifying monk would lock himself in a stone tomb barely larger than his body, where he would not move from the lotus position. His only connection to the outside world was an air tube and a bell. Each day he rang a bell to let those outside know that he was still alive.

When the bell stopped ringing, the tube was removed and the tomb sealed. After the tomb was sealed, the other monks in the temple would wait another 1,000 days, and open the tomb to see if the mummification was successful. 

If the monk had been successfully mummified, they were immediately seen as a Buddha and put in the temple for viewing. Usually, though, there was just a decomposed body. Although they weren't viewed as a true Buddha if they weren't mummified, they were still admired and revered for their dedication and spirit.

Reading #8. A Lightweight Multistroke Recognizer for User Interface Prototypes

Comments on Others:

Youyou Wang 

Summary
           The paper describes $N recognizer, a lightweight, concise multi-stroke recognizer, which is a significant extension to the $1 uni-stroke recognizer.  The $N said to be capable of user defined complex gesture identification, customization, and operate at speeds supporting fluid interaction (not sure what it means though???). $N is an extension of $1 to overcome limitations of that version, such as uni-stroke nature, failure to recognize 1D gestures, and rotation invariance. More specifically, the $N recognizer is implemented to use with rapid prototyping as way of quickly incorporating it to user interfaces with small loc size.
           The intuitive idea behind the $N development is to support rapid prototyping applications by eliminating the need of permutations of multistroke gesture and enable to enter only a single version of the gesture and use it for recognition. This is done by creating unistroke permutations of the multistroke at the define time and then using those for comparison at the run time. The $N uses automatic differentiating using a threshold to identify 1D and 2D gestures so that 1D’s can be preserved its aspect ratio.
           The paper also describes $N limitations, such as provisions for scale and position, and not using gesture features. 

Discussion
            First the use of name $N kind a amusing, may be they took the idea of $1 literally and replace 1 by N to represent the idea of multi-stroke possibility in their algorithm recognition compared to $1 recognizers uni-stroke nature. But I guess $1 recognizer authors used that name to represent both how simple it is, less expensive (shorter implementation time and small size) and uni-stroked. But when it comes to $N, the idea comes to readers mind is this thing N times expensive to implement (that much complex) than $1 recognizer (don’t laugh, that’s how I feel).
           One of the best goal of the $N as I consider is the ability to employ recognition with minimal input support (just a single multistroke entry) compared to other available recognizers including $1.  The use of just the Euclidian distance for comparison between the candidate stroke to permutations unistroke is questionable, and in my opinion not the correct technique to do so. May be a RMSE value between the candidate and the unistroke permutation may be a good idea to verify the accuracy and may be to increase the performance.
           The $N requires separate step to do the recognition of 1D from 2D, and in my opinion this can be avoided by using size invariant algorithmic design, may be a comparison based on the basic segment structure (example: a square consists of similar lengthen 2 horizontal and 2 vertical lines, and a triangle consists of 2 slanted and 1 horizontal lines).

Find the paper here. 

Reading #7. Sketch Based Interfaces: Early Processing for Sketch Understanding

Comments on Others:

Chris Aikens

Summary
           The paper discusses a novel approach of developing an interface which feels as natural as a paper but smart enough to understand the user’s intension and identifying sketches to a geometric shape. More specifically the work is intended to provide natural sketching and have the sketches understood. Also the authors state their intension of this work by giving user a system with unrestricted capabilities, such as drawing a rectangle clock-wise or counter clock-wise or drawing it with multiple strokes and still which is able to identify by the computer, like how people perceive things in the real world, the geometric shape rather how it was drawn.  Also note that the authors specified their domain as mechanical engineering design which dives added difficulty and complexity into sketching in this domain. Therefore the paper states that preprocessing steps like finding corners, fitting both lines and curves particularly important.
           The system design consists of approximation, beautification and basic recognition in the preprocessing stage. According to the paper, approximation fits pixels to lines and curves, Beautification modifies the output from approximation to make it visually more appealing, and finally Basic Recognition do the interpretation of the strokes, like sequence of 4 lines as a rectangle or a square. Subsequent recognition into complex structures handles by some other system in the design. The approximation consist of vertex detection, which is based on Average based filtering to find extrema corresponding to vertices while avoiding those due to noise. 

Discussion
           This paper particular grabs my interests due to its nature of regarding sketches as graphic objects and doing preprocessing before recognition to break them into basic objects (lines, curves etc). I consider this technique a sort of best way of tackling sketch recognition problems (just my personal opinion).
            The paper discusses vertex detection by the average based filtering which is on curvature graph and speed graph thresholds. I’m just wondering is there any other sophisticated ways to find the vertexes than this particular technique? In my experience, examining pixel distribution (neighboring pixels) is a hard problem; I’m interested in knowing something simple (Dr. Hammond mentioned tools to do this, am I missing something here?). Also I’m skeptical about using just the mean of the curvature data or speed data, a single higher order noise signal may ruin the whole detection process.   Also the Beautification is based on adjusting the slopes, but may be thickening the line and then skeltoning it would be a better choice, because slope adjustments may weaken the intended stroke appearance.
              In my personal opinion, the paper is lacking the required recognition design as a description which makes it more abstract to readers when it comes to the so called “higher level recognizer”.     

Find the paper here. 

Reading #6: Protractor: A Fast and Accurate Gesture Recognizer

Comments on Others:

chris aikens

Summary
            The paper describes a novel gesture recognizer using a nearest neighbor approach (I assume KNN), to recognize unknown gesture (testing sample or runtime sample) based on their features (or similarity) to the known gestures (training samples). The author states that simple, template-based recognizers are much more usable (advantageous) than its peers (sophisticated gesture recognition algorithms) in situations where personalized gesture interaction required.
           The Protractor preprocessing of gestures said to transform the 2D trajectory of the gesture to a uniform vector representation and claimed to be applicable to both invariant and orientation sensitive gesture sets with different aspect ratios. The paper further describes Protractor design which is similar to several of steps involved in the $1 recognizer including preprocessing with N=16 samples as opposed to N=64, and noise removal using the resampling. But the author sates that the Protractor does not rescale as of $1 recognizer therefore it makes it to even recognize narrow gestures such as horizontal or vertical lines.   

Discussion
           The paper discusses a simple but effective way of developing a gesture recognizer using an available feature classification algorithm, nearest neighbor. Even though claim of Protractor’s superiority compared to other gesture recognition is stated, in my opinion more performance evaluation needed to keep it in the firm ground (comparing it just to $1 recognizer is simply not enough to claim its superiority over other algorithms).   But I kind a biased and like the authors idea of comparing it to $1 recognizer, as I mentioned this as an advantage in my discussion on $1 recognizer paper. This may be a good point to consider when creating gesture recognizers, comparing it to both Protractor and $1 recognizer (both claims to be simple to implement). Also I agree that nearest neighbor algorithm is something pretty simple to implement using eucledia distance between the testing and training samples. But, based on my experience, the KNN tends to give mix set of results and it mostly depends on how you choose the feature sets for the training samples and quality of the feature set.
           The paper describes the optimal angular distance calculation which is used to compare with the nearest neighbor algorithm and use it to obtain the maximum similarity (inverse minimum cosine distance).
           The paper discusses nearest neighbor approach with K=1, but lack what will be the situation when the K value is changing (say 3, 5, 7 etc). As an example when Protractor was given 3 templates and finding assigning the highest similarity among 3 templates to the unknown one is not discusses as in the case of 5 or 7 or etc samples. With my personal experience, k=3 makes a better solution than k=1 where the comparison is happened between 3 known samples against 1 unknown sample (may be highly depends on the content area). Also missing is the description on how the performance of the Protractor when the template sample increases.

Find the paper here. 

Reading #5: Gestures without Libraries, Toolkits or Training: A $1 Recognizer for User Interface Prototypes

Comments on Others:

Wenzhe Li

Summary
           The paper states that “$1 recognizer” (I guess $1 represents how easy, cheap, and usable with less loc to program) was developed to enable novice programmers to incorporate gestures into their UI prototypes. The paper includes a detailed pseudo code, so that programmers can easily test the gesture recognizer through their interface design. The authors correctly states that the making of gesture recognition a topic of interests to experts in the field of AI, not experts in HCI with primarily works on interactive level (not completely agree!). Therefore, authors assume that this perhaps limited the opportunity to slip in gesture recognition into UI design. The paper also presents why Ad-hoc recognizers prevents applications due to limitations in defining own gestures at run time,  and when the gesture set is very large.
           The $1 recognizer is essentially limited to uni-stroke due to the authors’ interest in recognizing paths delineated by users interactively. The authors state comparatively, $1 recognizer is better because sophisticated methods like HMMs, ANNs, statistical classifiers require extensive training before practical use, and also they are difficult to program and debug. The authors describe, even the popular Rubine’s linear classifier requires advance computation before its use.
           The paper also states 8 criteria’s defined upon the recognizer to make it simple (These criteria’s also considered as goals of this methodology). The paper describes its algorithm implementation is 4 step process. As limitations, authors states that $1 algorithm is rotation, scale and position invariant (circle and oval both similar). Also gestures are not possible differentiate based on time. The paper states the testing procedure with comparison to Rubin classifier and Dynamic Time Warping (DTW).   

Discussion
              This paper explains a simple gesture recognizer which can be easily program and plugged in to a UI design and support gestures through the UI. The authors believe that perhaps the area of interests of HCI experts makes it limited for the UI designs to be incorporated with gesture recognition. In my opinion, I’m not completely agreed to that, may be limitations on the existing technology hinder the opportunities to do more research on HMM, ANN etc. and to apply that knowledge to UI usability and development research.
           I’m not sure why authors state that HMMs require extensive sample training and complex coding and debugging. With my personal experience on developing HMM based eye movement detection algorithm, I feel authors bit mislead just by the content before applying it to real world applications (HMM is reasonably fast with online applications though it require some sort of a training before hand). Also, I’m just wondering the use of N=64 for the resampling. May be using a digitization (select the path and make adjacent pixels 1 or zero depends on the next pixel and direction)
           Overall, the paper presents easy to read/understand concept on simple gesture recognizer, and I believe is a plus for gesture/sketch recognition community to implement it and compare it with their own algorithms (Because authors suggests that their work is comparatively competitive in terms of results in previously used complex algorithms, Rubine and DTW).

Find the paper here.

Reading #4: Sketchpad: A Man-Made Graphical Communication System

Comments on Others:

Wenzhe Li

Summary
         This paper describes earliest (more importantly, first), man-machine-interaction (MMI) technology called “Sketchpad” by Ivan E. Sutherland (According to Dr. Hammond, Sutherland was recognized as the father of the sketch recognition). The sketch pad was capable of drawing sketches according the movement of a light pen (for position trace) and using a push button to say a specific function (turn on and off switches) to execute. This way the sketchpad system said to be eliminated typed statements (so far we are still using typing, but its’ OK the paper dates back to 1964 with the original idea) in favor of line drawings.
            The authors demonstrated its intended use of light pen with a novel idea of using pseudo pen location to detect where the intended location that the pen needs to be aiming at. The paper explains its displays are using straight line segments, circle arcs and single points as the basic (or fundamental) design symbols for its sketchpad displays. The authors also discusses the use of display abstraction to support in drawing properties, like similar length, similar size etc using constrains. 

Discussion

             The sketchpad is an important finding for the sketch recognition domain (I mean the idea of sketch/gesture recognition),   and I believe it makes totally a new era of human computer interaction. How important this area of research going to be for our day today life or to the near future, readers can look at your own pocket (touch phones, touch pads, touch media players etc). For the moment we are using some sort of gesture input for these devices, and we can expect whole new way of interactions (May be you can look at “Microsoft's Vision of the Future” video just to get a clue).
           It is an interesting idea of having a pseudo pen location for segments where a light pen is aiming at. But as of my opinion, when it comes to heavily multi-stroked gestures/sketches, the system tends to fail due to cross-mapping of pseudo pen location (it is possible to map the pen location to a wrong place due to multi-strokes already sketched on). I’m not clear how the authors solved this problem in their related work (any thoughts???).
           Most of the future works of the paper are in successful outcomes (in my personal opinion), and the concept of sketching using an input device is outdated (for the purpose of creating an electrical circuit or complex drawing etc.), there are already number of software products to do all these drawing stuff (like smartdraw).   

Find the paper here. 

Reading #3: “Those Look Similar!” Issues in Automating Gesture Design Advice

Comments on others:

Wenzhe Li

Summary

            This paper describes an interface design tool that uses unsolicited advice to help designers of pen-based user interfaces to create pen gestures which are dissimilar. The tool said to be designed such a way that it warns designers when their gestures will be perceived to be similar and advise how to make their gestures less similar. The authors state their gesture design tool “quill” which will advises designers on how to improve their gestures. To detect when people will perceive gestures as similar, quill said to be equipped with experimentally-derived model of human-perceived gesture similarity. This paper states its similarity experiment with outcomes of 99.8% for not similar gestures and 22.4% for similar ones with a gesture pair to perceive as similar or not. The paper also discusses interface and implementation challenges on quill.

Discussion

             The paper discusses an interesting topic of helping interface designers to create less similar gestures with the pen based user interfaces. The paper states cut and paste as gestures with similar abstract operations which often confuse designers when assigning gestures. I’m not sure how true the statement is, may be authors wants to compare cut and copy, not cut and paste!!!
           With that much lower accuracy rate for identifying two similar gestures (22.4%), it is hard to understood that quill is so much accurate or powerful enough to predict when two gestures are similar (in my opinion the similarity experiment result accuracy is not even in the mid rage which is unacceptable). Also I feel the readers may want more detailed explanation on similarity experiment (as the backbone of the quill, by the way that’s what provide intelligence to quail) to get a fair idea of how the quill works.
           The paper discusses difficulties in providing advice, specially at what time, as soon or later. But the authors state that they found that the unsolicited advice is the best way though there are difficulties with the timing in situations like brainstorming. May be the solution is too simple to think (or stupid), but why don’t we create a condition for brainstorming (may be user can press a brainstorm button to stop advice, or stop advice button!! Genius or what).

Find the paper here.

Reading #2: Specifying Gestures by Example (Rubine)

Comments on others:

Yue Li

Summary:

This paper describes possibility of creating automatic gesture recognizers from example gestures by removing the need for hand coding. The GRANDMA (Gesture Recognizers Automated in a Novel Direct Manipulation Architecture) toolkit developed for rapidly adding gestures to direct manipulation interfaces and trainable single stroke gesture recognizer used by GRANDMA are also explained. The paper also describes powerful combination of gesturing and direct manipulation in the two-phase interaction technique. The author suggests the requirement of 15 training samples (or examples) per gesture class which is selected empirically. The author explains a simple preprocessing applied towards the inputs by removing time stamped coordinates which are 3 pixels apart from the previous input point. Furthermore, the author describes the feature set was empirically determined to work well on a number of different gesture sets. The gesture feature classification is handled by a linear classifier over the features. In addition Eager recognition to recognize gestures as soon as they are unambiguous and Multi Finger recognition are also discussed.

 

Discussion

  According to the author, the GRANDMA took kit is essentially a single stroke based gesture system and states that this allows shorter timeouts to be used and therefore avoids segmentation problem. Still the author states the possibility of both timeouts and mouse button release, and this makes readers wonder that if button release action is a possibility then why its so hard to incorporate the multi-stroke gesture recognition to the GRANDMA toolkit. In my opinion, it is voluble for users to show that even the multi-stroke gesture recognition is possible through the GRANDMA toolkit though it is faster (or less problematic and simple) to use only single-stroked gesture recognition.
          The author suggests than 15 training samples are enough per gesture class and this essentially makes readers wonder that the use of terminology in the GRANDMA, "Automation". Author refers it as the possibility of automatic gesture recognition vs. need for hand coding, and in my opinion if the recognizer still requires some sort of manual input (gestures) for the system to be functional (just my opinion).
         The elimination of jiggle is another point to consider, the author states a simple preprocessing to remove input apart from 3 pixels from the previous input. In my opinion this may hinder the users intension as well as the actual input gesture. I believe a strong preprocessing; something like a binarization and then skeltonation would be much more appealing. The use of specific features set need to be independently verified, may be a sensitivity analysis and ranking order based on the analysis would be a better choice. Its quite nice if there’s some results from different gesture classifications, something like KNN compared to linear classifier.

Find the paper here.

Reading #1: Gesture Recognition

Comments on others:
Francisco Vides  
Jianjie (JJ) Zhang

Summary:
             This paper starts with providing fundamental difference between the sketch and a gesture and the usefulness of gesture recognition to recognize a sketch. In contrast, the author states to avoid use of gesture based techniques in sketching due to possible constraining users from brainstorming and design. Furthermore, the author states that gesture recognitions work well in concert as editing or action commands in a sketch recognition system.
           Furthermore, author discusses mainly the work of Dean Rubine’s gesture recognition method used for sketches in 1991, Long’s method which extended Rubine’s features to 22 and Wobbrock’s $1 recognizer. Author distinguish the Dean Rubine’s work as first and most recognized gesture recognition method for sketches with 13 stroke features and a linear classifier to feature classification. Author describes Christopher Long’s work on gesture recognition called the system Quill in 1996, which is similar to Rubine’s but slightly modified to have less reliance on time. Furthermore, the author states Jacob Wobbrock’s work on implementing a simple template matcher in 2007 that recognizes gestures slightly better than the feature based method. It is called the $1 recognizer because is it easy to implement.

Discussion:
            The author presented a number of methods for performing gesture recognition in a single roof survey. Much of the authors’ concentration is on Rubine’s work, giving a clear meaning and importance of each of the feature set than the Rubine’s paper (I feel Rubin’s paper bit complicated to understand than the authors explanation, I guess that’s what authors’ intension was in writing this as a survey paper).  
              In my opinion I would like to see the features mapped to a common ground with a same value scaled to its characteristics. Say, something like a fuzziness 0-1. If it is a line, how slanted, how straight etc. And also some sort of a sensitive analysis to see which features makes the recognition accuracy better, may be a ranking order of the identified features may be a good idea.

Find the paper here.

About me...............

E-mail address : UKSJayarathna(AT)gmail(DOT)com

Graduate standing : First year PhD (MS from Texas State University-San Marcos)

Why are you taking this class? My undergraduate research background is “Offline Character Recognition”, so I feel comfortable taking a course similar to my research interests. Also I hope the course content will give me some interesting clues to some of my pet research ideas. 

What experience do you bring to this class?
This is my 6^th year in the HCI research field, and I’m specifically interested on “Biologically inspired mathematical models”. I want to apply some of AI techniques to sketch recognition, say Fuzzy Logic. Any ideas people?  

What do you expect to be doing in 10 years?
Doing research on anything I like (hopefully something related to my PhD) (ii) working with graduate students (iii) teaching classes (I love teaching) (iv) applying for grants (v) flying around to work with other researchers and to give talks on my research

What do you think will be the next biggest technological advancement in computer science?
I’m expecting eye tracking technology (not the biggest though) to kicks in within next couple of month’s/year’s time. At least the notebook guys are having trouble with finding ways to keep their toys expensive (fingerprinting is already there, and some sort of voice/face recognition systems too),  so we have something realistic like in “Minority Report”.

What was your favorite course when you were an undergraduate?
“Remote Sensing & Geographic Information Systems (GIS)”, I ended up getting A+ for it.

What is your favorite movie and why?
“The Man from Earth”. I like movies which makes you to go back see what they are saying is true and end up feeling amazed how they organize and blend real world facts into the movie. Something like Da Vinci Code, and National Treasure.  May be I'm a bit biased to the "The Man from Earth", its producer Eric D. Wilkinson personally commented and thanked on my blog post about his movie.

If you could travel back in time, who would you like to meet and why?
This is kind a common answer I guess. But I desperately need to thank my Dad for everything he did for me.    

Give some interesting fact about yourself
Read my Bio here.