Label-Specific Features for Multi-Label Classification

如果对所有示例采用相同的特征集合来归纳分类模型，则会忽略每个标记的独特特征。如基于形状的特征能较好的区分物体，如猫和狗等，而基于颜色的特征则能更好的区分背景，如天空和大海等。

基于类属特征学习（label-sepcific feature learning）的目标在于，找到类别标记的相关并可区别特征[1]。

LIFT[2]: 对每个标记聚类，分析其中正类和负类示例。最后度量原始示例到聚类中心的距离，获取标记限定特征。
优化：降维，聚类，局部邻居信息，全局空间拓扑信息，相关标记信息

许多算法通过探索标记间的语义关系，来改进标记限定特征的学习。标记相关性常以先验知识的形式来对学习过程做限制[3-7]。这些算法计算成对标记之间的相似性，并将基于标记相关性的相似特性输入模型训练中，施加约束使强相关标记之间共享更多特征[8-10]或相似预测[11]。但是这些算法常采用预计算的相似矩阵，并不利于下游任务的学习。

Collaborative Learning of Label Semantics and Deep Label-Specific Features for Multi-Label Classification

Collaborative learning:

• Label embedding: 选择标记最相关特征
• Discrimination errors: 通过反向传播，识别标记限定特征的判别属性

Label semantic encoding

采用GIN获得标记共现的语义空间的标记嵌入。

• 共现矩阵：

  $$A_{ij} = \frac{1}{2} [P(l_j|l_i)+P(l_i|l_j)]$$

• GIN优化标记嵌入矩阵：

  $$H^{t+1} = f^{t+1}[(1+\epsilon^{t+1})H^{t}+AH^{t}]$$

• pairwise decoder 区别非邻相关标记：

  $$L_{le} = \frac{1}{q^2} \sum_{i=1}^q \sum_{j=1}^q [cos(e_i,e_j)-\hat{A_{ij}}]^2$$

Semantic-Guided Feature-Disentangling

标记语义为类属特征的学习提供指导以及标记关联信息。

• 类属特征映射： $$\phi_k(x) = \phi(x,e_k)$$ 这个过程挺简单的，就是一个全连接层+attention，采用了leakyReLU。

参考文献:
[1] Hang J Y, Zhang M L. Collaborative Learning of Label Semantics and Deep Label-Specific Features for Multi-Label Classification[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.
[2] Zhang M L, Wu L. Lift: Multi-label learning with label-specific features[J]. IEEE transactions on pattern analysis and machine intelligence, 2014, 37(1): 107-120.
[3] J.-H. Ma and T. Chow, “Topic-based instance and feature selection in multilabel classification,” IEEE Trans. Neural Netw. Learn. Syst., early access, Oct. 27, 2020, doi: 10.1109/TNNLS.2020.3027745.
[4] X.-Y. Che, D.-G. Chen, and J.-S. Mi, “A novel approach for learning label correlation with application to feature selection of multilabel data,” Inf. Sci., vol. 512, pp. 795–812, 2020.
[5] H.-R. Han, M.-X. Huang, Y. Zhang, X.-G. Yang, and W.-L. Feng, “Multi-label learning with label specific features using correlation information,” IEEE Access, vol. 7, pp. 11 474–11 484, 2019.
[6] Z.-S. Chen and M.-L. Zhang, “Multi-label learning with regularization enriched label-specific features,” in Proc. 11th Asian Conf. Mach. Learn., Nagoya, Japan, 2019, pp. 411–424.
[7] J. Huang, G.-R. Li, Q.-M. Huang, and X.-D. Wu, “Learning label-specific features and class-dependent labels for multi-label classification,” IEEE Trans. Knowl. Data Eng., vol. 28, no. 12, pp. 3309–3323, Dec. 2016.
[8] J. Huang, G.-R. Li, Q.-M. Huang, and X.-D. Wu, “Learning label specific features for multi-label classification,” in Proc. 15th IEEE Int. Conf. Data Mining, Atlantic City, NJ, USA, 2015, pp. 181–190.
[9] J. Huang, G.-R. Li, Q.-M. Huang, and X.-D. Wu, “Joint feature selection and classification for multilabel learning,” IEEE Trans. Cybern., vol. 48, no. 3, pp. 876–889, 2018.
[10] J. Xu, H. Tian, Z. Wang, Y. Wang, W. Kang, and F. Chen, “Joint input and output space learning for multi-label image classification,” IEEE Trans. Multimedia, vol. 23, pp. 1696–1707, 2021.
[11] X.-Y. Jia, S.-S. Zhu, and W.-W. Li, “Joint label-specific features and correlation information for multi-label learning,” J. Comput. Sci. Technol., vol. 35, no. 2, pp. 247–258, 2020.