Dual-modality learning and transformer-based approach for high-quality vector font generation

Vector fonts, serving as the fundamental format of fonts, play a significant role in modern media. Described by a set of mathematical equations, vector fonts enable style modifications by adjusting drawing parameters, making them favored by font artists and designers. Due to the non-structural nature of vector font data, the task of vector font generation resembles sequence generation. Existing methods, limited in handling long sequences, are only capable of synthesizing simple character vector fonts. In this paper, we propose a dual-modal learning strategy to convert raster glyph images to vector glyph images in an end-to-end manner. Specifically, by employing vector quantization, we comprehensively utilize the dual-modal information of vector fonts. We quantize image and sequence modal features using a shared codebook, mapping them to the same discrete space and aligning them. Through aligned features, we reconstruct raster glyph images and vector glyph images. During the transformation process of vector glyph data, we redesign the Transformer module for vector glyph data, leveraging multiple stacked sliding window attention mechanisms to model local and global information. By integrating reversible residuals with attention and feedforward layers within the Transformer module, we enhance the model's capability and stability in handling long sequence data without sacrificing accuracy. Finally, we perform cross-modal model distillation to obtain the model's backbone network. We further refine the backbone network using a differentiable rasterizer to minimize error accumulation during sequence generation. Qualitative and quantitative results demonstrate that our method achieves high-quality synthesis results in complex Chinese character glyphs. The synthesized vector fonts can be easily converted into TrueType fonts for practical use, holding valuable applications for anyone interested in personalized vector font styles.

Actions (login required)

View Item