Research and Applications of Mining Techniques on Multimedia Data

¡@

As the advance of computer technology, data can be presented in various forms. By the evolution of storage media, much more data have been represented as images, audio and videos. Nowadays, archiving and analyzing multimedia data to acquire knowledge has become one of the important research issues. More attention to native culture in recent years has brought the concerns of the general public to the archiving and analysis of the aboriginal art assets. The academia has attempted archiving the aboriginal data of the individual tribes, such as the Digital Library and Museum project of National Taiwan University and the integrated research project of Academic Sinica. However, these projects do not consider the characteristics of multimedia data. The goals of this project are to develop new techniques for multimedia data mining, apply them to the analysis of the aboriginal arts, and then popularize the aboriginal arts.

The aboriginal arts of Taiwan, such as the clothing, architecture, music and dancing, have their own peculiar characters. In this project, we will consider the aboriginal arts in multimedia forms, such as pictures of clothes, images of buildings, recordings of songs, and videos of dances, to develop effective and efficient techniques for data mining. Moreover, we will apply the techniques to the analysis, archiving, and popularization of the aboriginal arts. In the first and the second years, we will consider three types of media, including image, music and video, and design methods for data mining based on their characteristics, respectively. In the third year, we will study the issues of data mining on multiple media and integrate the mining results to build a guided browsing and querying system. In this project, we classify the research items into four topics by media types and respectively describe them as follows:

1. Image Data Mining

This topic only considers image data, such as pictures of clothes or images of buildings. Our chief concern focuses on deriving semantic object features from low-level features of image data. After that, we will find out the characters of different aboriginal tribes in clothing or architecture and then utilize them for classifying images into tribes automatically.

2. Music Data Mining

This topic focuses on the analysis of music data, including the extraction of low-level features and the music classification based on these features. Except for classifying music by tribes, we will also consider to classify music by different purposes, such as sacred music and folk songs. In addition, we will find out the characters of different music by contents to infer the musical style of each aboriginal tribe.

3. Video Data Mining

For video data, we focus on the videos of aboriginal dancing. Because a dance is composed of a series of body motions, we can apply the methods for sequential pattern mining to not only compare the dances among different tribes but also derive the essential motions in the dances.

4. Multimedia Data Mining

This topic integrates all the three types of media instead of limiting to only one of them. For this purpose, we will develop a guided browsing and querying system that provides the knowledge discovered from multiple media.