Description
In _process_one_batch() (exp02_ccm.py), channels are always shuffled:
shuffled_indices = torch.randperm(channel)
batch_x = batch_x[:, :, shuffled_indices]
batch_y = batch_y[:, :, shuffled_indices]
This is done regardless of whether CCM is enabled.
Later, the similarity matrix is computed as:
simMatrix = self.get_similarity_matrix(batch_x)
using the original channel ordering.
The clustering loss is then computed as:
loss_s = self.similarity_loss_batch(self.model.cluster_prob, simMatrix)
However, self.model.cluster_prob corresponds to shuffled channels whereas simMatrix corresponds to the original ordering.
As a result, the similarity loss is effectively comparing two representations that refer to different channel index permutations, making the supervision signal inconsistent.
This introduces a silent misalignment in the clustering objective, potentially degrading training stability and interpretability.
Expected behavior
Either:
- apply the same permutation when computing
simMatrix, or
- compute the similarity matrix using the shuffled channels.
Otherwise, the clustering loss seems to be computed using mismatched information.
Description
In
_process_one_batch()(exp02_ccm.py), channels are always shuffled:This is done regardless of whether CCM is enabled.
Later, the similarity matrix is computed as:
using the original channel ordering.
The clustering loss is then computed as:
However,
self.model.cluster_probcorresponds to shuffled channels whereassimMatrixcorresponds to the original ordering.As a result, the similarity loss is effectively comparing two representations that refer to different channel index permutations, making the supervision signal inconsistent.
This introduces a silent misalignment in the clustering objective, potentially degrading training stability and interpretability.
Expected behavior
Either:
simMatrix, orOtherwise, the clustering loss seems to be computed using mismatched information.