The AlloSphere is a three-story full-surround, multimodal, immersive facility in the Media Arts and Technology at the University of California, Santa Barbara to represent large and complex data, including immersive visualization, sonification, and interactivity.
AlloLib is a cross-platform suite of C++ components for building interactive multimedia tools and applications.
- Developing and improving the overall system
(graphics/acoustics/interface/network in C++)
- Audiovisual interactive installation based on SARS-CoV-2 virus and human protein experiments & simulation
- Developed general-purpose gesture-based smartphone 3D interface (C++).
- Developing a multimodal 3D interface with pattern recognition using signal processing and machine learning.
Reproduction of acoustic signals based on the interpretation of conducting gestures using controllers with accelerometers and gyroscopes.
- Developed and performed machine learning and signal processing based interactive-musical system
(MATLAB, Python, C++).
- Designed and made a baton controller prototype.
- Improved the work using the smartphone with the gestural sensors.
• Machine learning
- Performed experiments on the application of various machine learning techniques
with Tensorflow, Python, Kaldi, and MATLAB
- Applications: reverberation time estimation, acoustic models, jointly trained neural network, bandwidth expansion, and gesture interpretation.- Contributed distributive research with Samsung electronics.
• Summer internship in Nokia Bell Labs. Experiments in Art and Technology (EAT)
- Acoustic signal processing and machine learning task
Obtaining room acoustic information from sound sources received by microphones.
- Conducted an intensive study with a single & multi-channel based algorithm
using a deep neural network. (Published in IEEE IC-NIDC and Acta Acust united Ac)
- Developed an estimation algorithm for dereverberation and acoustic model
- Contributed distributive research with LG electronics.
Reproducing audio to utilize binaural audio signals with two loudspeakers by
offsetting the crosstalk components
- Developed crosstalk cancellation algorithms based on HRTF with low sensitivity.
- Participated in a national project exclusively; Information Technology Research Centre: Center for Ultra Realistic Audio Technologies (ITRC: CURAT)
Synthesis of audio signals with directivity using multiple loudspeakers.
- Developed an adaptive multi-channel audio spatialization algorithm (MATLAB)