How can I train the parameter in cross-attention? #3

Activity

1. jorgelerre commented on Jun 15, 2026

   

   jorgelerre

   on Jun 15, 2026

   More actions

   Hello, I found this same issue as dankimh. According to the paper, prototype learning is performed using cross-attention.
   However, the implementation of CrossAttention internally relies on MaskAttention, which does not include the projection matrices (W_Q), (W_K), and (W_V).
   Therefore, the prototype update module (p2c) contains no learnable parameters.
   As a consequence, the operation cluster_emb = self.p2c(cluster_emb_, x_emb_, x_emb_, mask=mask.transpose(0,1)) effectively behaves as a masked weighted average of channel embeddings.

   Thereby, I would like to make the following questions:

   • Was the intention to use MaskAttentionLayer instead (which actually has learnable weights), or is the current behavior the one used in the experiments reported in the paper?
   • In case of using MaskAttentionLayer, how did you train the resulting network so the gradients would be successfully propagated to its parameters, taking into account that the prototypes are replaced at each step?