Our solution is to extend conditional VAE to a crossed version named Cross-VAE, which is able to use partially labeled data to disentangle expression from identity. We emphasis the following novel characteristics of our Cross-VAE: (1) It is based on an independent assumption that the two latent representations' distributions are orthogonal.

Therefore we design the Cross-VAE model to disentangle identity and expression from each other. In this section, we firstly introduce the Conditional VAE (CVAE) model, which is closely related...

We propose a novel VAE framework (called cross-domain VAE, CDVAE) for VC. Specifically, the proposed framework utilizes both STRAIGHT spectra and MCCs by explicitly regularizing multiple objectives in order to constrain the behavior of the learned encoder and decoder.

In this paper, we propose a novel cross-modal varia-tional alignment method in order to process and relate in-formation across different modalities. The proposed ap-proach consists of two variational autoencoder (VAE) net-works which generate and …

2020年5月1日 · Variational autoencoders(VAE) can overcome the shortcomings of traditional variational methods such as low efficiency and poor generality, and provide an efficient and extensible framework...

2020年12月21日 · We propose a cross-domain latent modulation mechanism within a variational autoencoders (VAE) framework to enable improved transfer learning. Our key idea is to procure deep representations from one data domain and use it as perturbation to the reparameterization of the latent variable in another domain.

“Large Motion Video Autoencoding with Cross-modal Video VAE” Authors: Yazhou Xing*, Yang Fei* , Yingqing He*†, Jingye Chen, Jiaxin Xie, Xiaowei Chi, Qifeng Chen† arXiv preprint

2024年12月23日 · In this paper, we present a novel and powerful video autoencoder capable of high-fidelity video encoding. First, we observe that entangling spatial and temporal compression by merely extending the image VAE to a 3D VAE can …

2023年9月8日 · The core component of the CUC-VAE S2 framework is the cross-utterance CVAE, which extracts acoustic, speaker, and textual features from surrounding sentences to generate context-sensitive prosodic features, more …

In this way, we create a cross-modal VAE (Cross-VAE). During training, the proposed Cross-VAE is trained to minimize the reconstruction loss of the two modalities, the distribution loss of the two VAEs, and a novel third loss called the space sharing loss.