BYU Computer Science Graduate Program Announcements

James Carroll's PhD Dissertation Proposal

2008-10-01T14:53:24-06:00

ABSTRACT:

Today's computers are capable of performing many tasks that previously only people could perform.This is partly due to recent techniques that have allowed computer systems to "learn" from experience. Machine Learning (ML) is the field of computer science that attempts to understand what it means to learn, and how such learning can be performed.

One major missing ingredient in the study of Machine Learning is a cohesive theory and framework in which a broad range of questions can be analyzed and studied. An effective "theory" of Machine Learning should be able to answer questions in areas such as: Learnability, Bias, Overfit, No-Free-Lunch, Meta Learning, Transfer Learning, Active Learning, and Sample Complexity. Other important issues such as the connection between Machine Learning and other fields such as Function Optimization, Compression, and Information Theory should also be explored.

Several paradigms or theories that attempt to formalize the learning process have been proposed including: PAC learnability, VC dimensionality, and the Extended Bayesian Framework (EBF). To date, none of the proposed frameworks have been able to answer all of these questions in a satisfactory manner. For example, in some cases it is possible to use VC dimension or PAC learnability to show that some problems are learnable and to place theoretical upper bounds on their sample complexity (the number of examples required to learn a given problem). However, in practice, such bounds are seldom if ever used, because they vastly overestimate the actual number of examples required. Furthermore, PAC learnability and VC dimensionality do not provide a framework in which the other questions can be easily answered. No single theory has yet been proposed which deal with all of the above areas.

We propose a Unified Bayesian Decision Theoretic Model (UBDTM). The UBDTM models both the classification problem, the regression problem, and the function optimization problem as a single graphical model that expresses the dependencies and relationships between observed feature vectors, observed labels, unobserved (test and validation) feature vectors, unobserved labels, the function that maps them, and the location of the extremum (or multiple extremum) of the function. There are many advantages to thinking about Machine Learning and optimization in this way. Instead of creating algorithms, the machine learning practitioner's job is now to create representations and priors, while the algorithm is fixed as simple inference. The model makes the algorithm's bias explicit in the function representation and priors. The model makes the need for a bias clear. Given the model's explicit bias, we can determine the function class over which the algorithm is expected to perform well. Meta learning can be modeled in terms of a hierarchical Bayesian model. Active Learning can be seen as the result of decision theory involving utilities and the expected value of sample information (or EVSI). Finally, a similar calculation can be used to analyze average case sample complexity.

It will be beyond the scope of a single dissertation to adequately address all of the above issues in terms of the UBDTM, however, we believe that this framework will provide a sound theoretical framework in which all of the above issues can be eventually addressed. This dissertation will focus on addressing a few specific questions in the context of the UBDTM. Specifically, we will show that the UBDTM provides another way of thinking about the bias and No-Free-Lunch problems, and that given some utility assumptions a priori distinctions between learning algorithms are possible. We will show that UBDTM provides another useful way of looking at overfit and uncertainty and the importance of explicit utility functions in Machine Learning for decision making. We will then demonstrate the ability of UBDTM to explain existing Machine Learning techniques and to guide improvements to those techniques with an example using the CMAC ANN topology. We will then focus on the insights that the model provides for active learning. We will show that the model explains the behavior of existing active learning techniques and guides the creation of new active learning techniques. The performance of these techniques will be demonstrated on several synthetic problems and on real world problems including a tagging problem which is part of a larger joint project to build an annotated Syriac corpus with the BYU Center for the Preservation of Ancient Religious Texts (CPART). Thus, tools derived from our model will be used to build a publicly available corpus of tagged Syriac tests. Finally we will show that these concepts of active learning can also be applied to function optimization since they share the same model. Thus, one advantage of the UBDTM is that it makes the connection between
supervised learning and function optimization clear. Although we believe that the remaining questions of meta learning, transfer learning, learnability, and sample complexity can also be addressed in terms of the UBDTM we will leave these problems for future work.

Intellectual Merit: The UBDTM has the potential to advance our general understanding of the supervised learning and function optimization problems, and various proposed solutions to them, by better understanding the underlying processes involved.

Broader Impact: Supervised Machine Learning problems are important in a wide variety of problems of relevance in a wide variety of fields, for example: linguistics (tagging natural language text); robotics (machine vision); homeland security (face recognition, machine translation); and medicine (disease diagnosis). Function optimization is one of the most common computational problems encountered in all of computer science. It is hoped that a better understanding of the statistical theory involved in supervised learning and optimization will lead to better algorithms and algorithm analysis. Testing will involve cross department research involving individuals from Computer Science, Linguistics, and CPART.

Colloquium with Dr. Lise Getoor September 18, 2008 at 11:00 am in 1170 TMCB

2008-09-16T09:29:04-06:00

Abstract:

Within the machine learning and data mining communities, there has been a growing interest in learning structured models from input data that is itself structured. Graph identification refers to methods that transform observational data described as a noisy, input graph into an inferred "clean" output graph. Examples include inferring organizational hierarchies from social network data, identifying gene regulatory networks from protein-protein interactions, and understanding visual scenes based on inferred relationships among image parts. The key processes in graph identification are: entity resolution, link prediction, and collective classification. I will overview algorithms for these tasks, discuss the need for integrating the results to solve the overall problem collectively.

Bio:
Lise Getoor is an associate professor in the Computer Science Department at the University of Maryland, College Park. She received her PhD from Stanford University in 2001. Her current work includes research on link mining, statistical relational learning and representing uncertainty in structured and semi-structured data. She has published numerous articles in machine learning, data mining, database, and artificial intelligence forums. She was awarded an NSF Career Award, is an action editor for the Machine Learning Journal, is a JAIR associate editor, has been a member of AAAI Executive council, and has served on a variety of program committees including AAAI, ICML, IJCAI, KDD, SIGMOD, UAI, VLDB, and WWW.

William (Bill) Lund's Qualifying Presentation

2008-09-05T08:55:19-06:00

ABSTRACT:

This paper shows the degree to which the optical character recognition (OCR) output from poor quality documents can be improved through applying the results of multiple OCR engines to construct an aligned word lattice consisting of word hypotheses. Results from a collection of poor quality mid-twentieth century typewritten documents demonstrate an average reduction in word error rate (WER) of close to 40% through the use of three OCR engines. Additionally, an innovative admissible heuristic for the A* algorithm is developed, which results in a significant reduction in state space exploration to identify all optimal alignments of the OCR text output, a necessary step toward the construction of the word hypotheses lattice. On average 0.0079% of the state space is explored to identify all optimal alignments of the OCR output of documents in the collection.

Lanny Lin's Qualifying Presentation

2008-09-03T17:05:51-06:00

ABSTRACT:

n the priority search phase of Wilderness Search and Rescue (WiSAR), a probability distribution map is created. Areas with higher probabilities are searched first in order to find the missing person in the shortest expected time. When using a UAV to support search, the onboard video camera should cover as much of the important areas as possible within a set time. We explore several algorithms in solving this problem and compare their performances against typical WiSAR scenarios. This problem is NP-hard, and our algorithms yield high quality solutions that approximate the optimal solution, making efficient use of the limited UAV flying time.

Colloquium with Dr. Olfa Nasraoui Thursday, September 11, 2008 at 11:00 am in 1170 TMCB

2008-09-02T14:30:15-06:00

Abstract:

Join me on a journey through some of the research projects at the Knowledge Discovery & Web Mining Lab at University of Louisville, as we discover the common theme that led us into fascinating problems touching such diverse concepts as the massive amounts of footprints left behind by surfers on the Web, the violent eruptions of hot plasma that draw beautiful coronal loop structures above the surface of the sun, and the numerous illegal contraband exchanges on unstructured peer to peer networks. Massive amounts of data sets that are being generated by more and more applications represent a wealth of information, and this raw information can be turned into golden knowledge using Data Mining techniques. These data mining techniques are frequently being pushed to the limit by the increasing challenges of huge data sizes, high dimensionality, and evolving behavior. These challenges are at the center of several research endeavors that we have been working on in the past years, and have included adapting ideas inspired by robust statistics, the natural immune system and natural evolution to several machine learning techniques. Applications range from personalizing Websites and Information Retrieval on e-learning platforms to mining coronal loop occurrences on the solar corona, mining evolving topics on the fly from online newsfeeds, and helping catch child pornography creators and consumers on P2P networks.

Dan Delorey's PhD Dissertation Proposal

2008-08-13T14:14:27-06:00

ABSTRACT:

Programming languages are the mechanisms that software developers use to communicate their mental models of software both to computers and to other software developers. Peter Naur, one of the early pioneers of programming language theory, has claimed that the term 'programming language' is misleading and that programming languages are not, in fact, languages. We disagree. We propose to show that programming language is language by applying tools developed by corpus linguists for the study of natural language to programming language. In this proposal, we 1) evaluate the relationship between programming language and natural language, 2) report on our preliminary work in studying the Java programming language, and 3) propose the methods we will use to determine how programming language is used in software development and how this use compares to the use of natural language.

Hyrum Carroll's PhD Dissertation Defense

2008-08-12T15:53:46-06:00

ABSTRACT:

Researchers use multiple sequence alignment algorithms to detect conserved regions in genetic sequences and to identify drug docking sites for drug development. In this dissertation, a novel algorithm is presented for using physicochemical properties to increase the accuracy of multiple sequence alignments. Secondary structures are also incorporated in the evaluation function. Additionally, the location of the secondary structures is assimilated into the function. Multiple properties are combined with weights, determined from prediction accuracies of protein secondary structures using artificial neural networks.

A new metric, the PPD Score is developed, that captures the average change in physicochemical properties. Using the physicochemical properties and the secondary structures for multiple sequence alignment results in alignments that are more accurate, biologically relevant and useful for drug development and other medical uses.

In addition to a novel multiple sequence alignment algorithm, we also propose a new protein-coding DNA reference alignment database. This database is a collection of multiple sequence alignment data sets derived from tertiary structural alignments. The primary purpose of the database is to benchmark new and existing multiple sequence alignment algorithms with DNA data. The first known comparative study of protein-coding DNA alignment accuracies is also included in this work.

Kyle Dickerson's MS Thesis Proposal

2008-08-04T14:47:27-06:00

ABSTRACT:

The ever-increasing density of computer storage devices has allowed the average user to store enormous quantities of multimedia content, and a large amount of this content is usually music. Current search techniques for musical content rely on meta-data tags which describe artist, album, year, genre, etc. Query-by-content systems, however, allow users to search based upon the actual acoustical content of the songs. Recent systems have mainly depended upon textual representations of the queries and targets in order to apply common string-matching algorithms. However, these methods lose much of the information content of the song and limit the ways in which a user may search. We propose a query-by-content system which uses a Self-Organizing Map as its basis.

Qiuyi Duan's PhD Dissertation Defense

2008-07-30T15:12:40-06:00

ABSTRACT:

With the proliferation of mobile applications and the abundance of wireless devices, it is increasingly common for devices to support multiple radios. When two devices are communicating they should choose the best available radio based on user preference and application requirements. This type of “radio switching” should happen automatically, so that the system optimizes performance dynamically.

To achieve this objective, we design an Autonomous and Intelligent Radio Switching (AIRS) system to leverage the radio heterogeneity common in today’s wireless devices. The AIRS system consists of three key components. First, we design a radio preference evaluation module to dynamically select the best radio according to users’ preference, application’s QoS requirements, and the device battery usage. Second, we propose a link quality measurement and prediction module to predict the radio quality under a variety of mobility and interference conditions. Third, we present a radio switching decision making module to switch to the preferred available radio intelligently, based on the preference and link quality evaluations.

The AIRS system maintains connectivity, as well as improves link quality, via dynamic and intelligent radio switching, regardless of interference or collisions from the interfaces of other devices. The radio preference evaluation module is able to generate and adjust a preference list dynamically. Multiple users’ requirements are satisfied in a mutually beneficial manner and the selected radio is Pareto optimal. The link prediction module is able to achieve an accuracy above 90% under a variety of mobility and interference conditions. The module can dynamically increase the link measurement interval and significantly reduce its power consumption, without sacrificing accuracy. The decision algorithm uses several parameters to avoid switching radios too frequently, and is able to provide dynamic, but stable radio switching, while balancing the competing objectives of high throughput and low power consumption. Overall, the AIRS system is able to achieve high goodput (application level throughput) and long battery life as applied to handoff management in a frequently changing mobile environment.

Ilya Raykhel's MS Thesis Proposal

2008-07-29T14:46:12-06:00

ABSTRACT:

While Machine Learning is one of the most popular research areas in Computer Science right now, there are still only a few applications meant to be used by the general public. We propose to develop an exemplary application that can be directly applied to eBay trading. We would like to build a system that predicts how much an item would sell for on eBay based on that item's parameters; we intend to run our experiments on the eBay laptop category. Prior trades will be used as training data. The system will implement a feature-weighted k-Nearest Neighbor algorithm, using genetic algorithms to determine feature weights.