📝 Publications

Codebook Transfer with Vision-to-Language Translation for Vector Quantization
Baoquan Zhang, Guotao Liang*, Tianran Chen, Yunming Ye, Zhiyuan Wen, Xiaochen Qi, and Yao He

Project Code

• We propose a new perspective, \emph{i.e.}, codebook transfer from language models to VQ, to alleviate the codebook collapse issue. Its advantage is that the abundant transferable relationships from language codebooks can be fully exploited for enhancing VQ codebook learning.

Improved Masked Image Generation with Knowledge-Augmented Token Representations
Guotao Liang, Baoquan Zhang*, Zhiyuan Wen, Zihao Han,Yunming Ye.

Project Code

• We propose a novel knowledge-augmented masked image generation framework, named KA-MIG, which leverages prior semantic knowledge graphs to enrich internal token representations and improve generation performance.

Towards Improved Text-Aligned Codebook Learning: Multi-Hierarchical Codebook-Text Alignment with Long Text
Guotao Liang, Baoquan Zhang*, Zhiyuan Wen, Junteng Zhao,Yunming Ye, Xiaochen Qi, Yao He. (Highlights)

Project

• We propose a novel text-augmented codebook learning framework, TA-VQ, which leverages VLMs to generate longer text for each image, improving text-aligned codebook learning.

LG-VQ: Language-Guided Codebook Learning
Guotao Liang, Baoquan Zhang*, Yaowei Wang, Yunming Ye, Xutao Li , HuaiBin Wang, Luo Chuyao, Kola Ye, Linfeng Luo.

Project

• We propose a novel multi-modal codebook learning method, named LG-VQ, which can enable the codebook to effectively retain fine-grained reconstruction information while aligning with the text.

Codebook Transfer with Part-of-Speech for Vector-Quantized Image Modeling
Baoquan Zhang, Wang huaibin, Luo Chuyao, Xutao Li, Guotao Liang, Yunming Ye, Kola Ye, Linfeng Luo.

Project

• We propose a new perspective, i.e., codebook transfer from language models to VQIM, to alleviate the codebook collapse issue.