I am a Ph.D. student at AMILAB in Electrial Engineering at POSTECH, advised by Tae-Hyun Oh. I received my bachelor’s degrees in Physics and Electrical Engineering (Double major) from Chung-Ang University. I work on research projects lies at the intersection of computer vision and machine learning, with a focus on modalities that contain temporal information, such as video and audio.
My research interests are on multi-modal learning, especially within audio-visual understanding and generation, but not limited to.
🔥 News
- 2024.07: One paper is accepted to ECCV 2024.
- 2024.06: Two papers are accepted to INTERSPEECH 2024.
- 2024.04: One paper is accepted to RA-L 2024. This will be presented in IROS 2024 (oral presentation).
📝 Pre-print (*:equal contribution/co-first author)
A Benchmark for Audio-Visual Large Language Models
Kim Sung-Bin*, Oh Hyun-Bin*, JungMok Lee, Arda Senocak, Joon Son Chung, Tae-Hyun Oh
Project (to appear)
- We introduce a comprehensive benchmark specifically designed to evaluate the perception and comprehension capabilities of audio-visual LLMs.
Revisiting Learning-based Video Motion Magnification for Real-time Processing
Hyunwoo Ha*, Oh Hyun-Bin*, Kim Jun-Seong, Kwon Byung-Ki, Kim Sung-Bin, Linh-Tam Tran, Ji-Yun Kim, Sung-Ho Bae, Tae-Hyun Oh
Project (to appear)
- We magnify invisible small motions, but in real-time.
- Short version at CVPRW 2023 ‘Vision-based InduStrial InspectiON’ Workshop.
📝 Publications
Learning-based Axial Video Motion Magnification
Kwon Byung-Ki, Oh Hyun-Bin, Kim Jun-Seong, Hyunwoo Ha, Tae-Hyun Oh
- We magnify invisible small motions, but in user-specified directions.
Enhancing Speech-Driven 3D Facial Animation with Audio-Visual Guidance from Lip Reading Expert
Han EunGi*, Oh Hyun-Bin*, Kim Sung-Bin, Corentin Nivelet Etcheberry, Suekyeong Nam, JangHoon Ju, Tae-Hyun Oh
- We generate 3D facial animation speech-synchronized lip movements with audio-visual lip reading expert.
MultiTalk: Enhancing 3D Talking Head Generation Across Languages with Multilingual Video Dataset
Kim Sung-Bin*, Lee Chae-Yeon*, Gihun Son*, Oh Hyun-Bin, JangHoon Ju, Suekyeong Nam, Tae-Hyun Oh
- We generate accurate 3D talking heads from multilingual speech.
Uni-DVPS: Unified Model for Depth-Aware Video Panoptic Segmentation
Kim Ji-Yeon, Oh Hyun-Bin, Kwon Byung-Ki, Dahun Kim, Yongjin Kwon, Tae-Hyun Oh
- We present Uni-DVPS, a unified model for Depth-aware Video Panoptic Segmentation (DVPS).
🎖 Honors and Awards
- 2024: Best Paper Award, KRoC
- 2023: Best Paper Award, IPIU
- 2021: Summa Cum Laude, Chung-Ang University
- 2021: Department Honors Scholarship, Chung-Ang University
- 2018: Science Scholarship, Suwon Municipal Scholarship Foundation
- 2017: Department Honors Scholarship, Chung-Ang University
📖 Educations
- 2022.03 - Present, Joint M.S. & Ph.D., Pohang University of Science and Technology (POSTECH), South Korea.
- 2017.03 - 2021.08, Undergraduate, Chung-Ang University.
Teaching Experiences
- 2023, NAVER Boostcamp AI Tech Computer Vision Track (5th, 6th)
- 2022, Introduction to Machine Learning (EECE454), POSTECH