Projects List

Ongoing Projects

AlloSphere & AlloLib:

The AlloSphere is a three-story full-surround, multimodal, immersive facility in the Media Arts and Technology at the University of California, Santa Barbara to represent large and complex data, including immersive visualization, sonification, and interactivity.

AlloLib is a cross-platform suite of C++ components for building interactive multimedia tools and applications.

- Developing and improving the overall system

  (graphics/acoustics/interface/network in C++)
 

 

Multi-user interactive system in virtual 3D spaces using the gesture of smartphones:

- Developed general-purpose gesture-based smartphone 3D interface (C++).
- Developing a multimodal 3D interface with pattern recognition using signal processing and machine learning.


• Music source conducting based on gestural control using machine learning:

Reproduction of acoustic signals based on the interpretation of conducting gestures using controllers with accelerometers and gyroscopes.

- Developed and performed machine learning and signal processing based interactive-musical system
(MATLAB, Python, C++).
- Designed and made a baton controller prototype.
- Improved the work using the smartphone with the gestural sensors.

Machine learning

Performed experiments on the application of various machine learning techniques

with Tensorflow, Python, Kaldi, and MATLAB

Applications: reverberation time estimation, acoustic models, jointly trained neural network, bandwidth expansion, and gesture interpretation.Contributed distributive research with Samsung electronics.

Completed Projects

• Summer internship in Nokia Bell Labs. Experiments in Art and Technology (EAT) 

        - Acoustic signal processing and machine learning task

Reverberation time estimation using machine learning:

Obtaining room acoustic information from sound sources received by microphones.
- Conducted an intensive study with a single & multi-channel based algorithm
using
 a deep neural network. (Published in IEEE IC-NIDC and Acta Acust united Ac)

- Developed an estimation algorithm for dereverberation and acoustic model
- Contributed distributive research with LG electronics.

• Crosstalk cancellation:

Reproducing audio to utilize binaural audio signals with two loudspeakers by
offsetting the crosstalk components
- Developed crosstalk cancellation algorithms based on HRTF with low sensitivity.
- Participated in a national project exclusively; Information Technology Research Centre: Center for Ultra Realistic Audio Technologies (ITRC: CURAT)

• Multi-channel audio spatialization:

Synthesis of audio signals with directivity using multiple loudspeakers.

- Developed an adaptive multi-channel audio spatialization algorithm (MATLAB)