Driven machine learning enthusiast specializing in natural language processing and computer vision, committed to solving real-world problems through data-driven innovation.
Experience
Return ZeroJul. 2024 - Present
Research Engineer
- Pioneered research and development of SOTA ASR models by leveraging extensive user data. Designed and implemented a robust, scalable production pipeline.
NCSOFTGraphics AI Lab.Feb. 2021 - Jun. 2024
Machine Learning Researcher
- Spearheaded the implementation of preprocessing and alignment pipelines for high-fidelity motion and facial capture data, significantly enhancing data quality.
- Pioneered the development of a PyTorch-based framework for deep learning model training and experiment management, substantially boosting team productivity and streamlining research processes.
- Led the research on a deep learning model for generating speech-based 3D facial animation, contributing to advancements in digital human technology.
NaverClovaJul. 2020 - Aug. 2020
Machine Learning Engineer, Intern
- Contributed to the development of a sophisticated depth estimation network for 3D map reconstruction.
WebtoonResearchSep. 2018 - Feb. 2019
Machine Learning Researcher, Intern
- Developed an advanced annotation tool for 2D bounding box support, integrating features such as live data model inference, dataset management with custom query types, and various tagging options.
- Designed the Visual QA task for Workshop, developing a multi-modal network and dataset for its validation.
SW Maestro7thJul. 2016 - Aug. 2018
- Garnered acclaim as a top team, showcased at the "100+ Conference" by the Ministry of Science and ICT, highlighting exceptional project development and execution.
Projects
Text-to-Texture Project
- Developed a scalable infrastructure, including model serving capabilities.
- Spearheaded the development of a Proof-of-Concept (PoC) web application using React.js and three.js, along with a FastAPI server.
- Conducted extensive research and experimentation on models for generating natural textures on 3D objects.
Digital Human Project
- Led the development of a real-time and responsive facial expression generation model and system for digital humans.
- Pioneered the generation of dynamic facial expressions and persona integration in conversational scenarios, enhancing digital human interactivity.
- Orchestrated the creation of highly detailed facial animations, capturing nuanced expressions tailored to unique character traits.
- Architected a high-performance API server, optimizing it for diverse inputs in facial animation generation, ensuring scalability and robustness.
3D Facial Animation Generation Model Research
- Implemented advanced preprocessing and alignment pipelines for superior motion and facial capture data quality.
- Devised and deployed a PyTorch-based framework for deep learning model training, elevating team productivity and streamlining research management.
- Conducted innovative research on deep learning models for speech-driven 3D facial animation generation.
Webtoon Annotation Tool
- Engineered advanced annotation tools for efficient collection of domain-specific raw data.
- Developed a sophisticated analysis tool for cross-validating annotations, enabling comprehensive tracking of productivity, task difficulty assessment, and model inference comparison.
MariaDB Scalable Lock Manager
- Enhanced the InnoDB storage engine's performance by replacing a lock-free design and reusable object pool for the global lock mechanism in the transaction record manager.
Memento: All Information About That Person
- Created a robust, scalable crawling system to aggregate a wide array of articles and information.
- Designed a sophisticated system for vector embedding and clustering, linking information to entities and events, while allowing for continuous vector expansion.
DeepCheck: Deep Learning-Based Auto Face Recognition Attendance Check
- Enhanced a face recognition model specifically for low-resolution, crowded environments.
- Developed a cross-platform application and API server using React Native, enabling efficient, lightweight model inference.
Education
Hanyang University
M.S. in Computer ScienceJongwoo Lim
Hanyang University
B.S. in Computer Science
Yonsei University - Mirae Campus
B.S. in Computer EngineeringWithdrawal
Awards
- Google Machine Learning Challenge Korea 5th place2017
- Top start-up certification SW Maestro 7th2017
- ACM-ICPC Asia Daejeon Regional Contest 16th place2016
Publications
Object detection in fisheye lens environment
Object counting using object detection and re-identification in video sequence
Skills
Languages
- Python
- C++
- JavaScript
- Kotlin
Frameworks
- PyTorch
- TensorFlow
- OpenCV
- NumPy
- FastAPI
- Spring
- React
Tools
- Git
- Docker
- Linux