A Novel Hybrid Approach with Siamese Networks and Domain-Adversarial Training
Keywords:
Domain-Adversarial Neural Networks (DANN), Neural Networks, Tradeoff, Contrastive pair, Gradient Reversal Layer (GRL), Differential PrivacyAbstract
Balancing data utility and privacy is a persistent challenge in privacy-conscious machine learning, especially when handling personal or biometric data. We propose PES-DAT (Privacy-Enhanced Siamese Network with Domain-Adversarial Training), a novel framework that generates discriminative, privacy-preserving embeddings for clustering tasks. PES-DAT uniquely combines contrastive representation learning with a domain-adversarial classifier, enforced through a Gradient Reversal Layer, to produce task-relevant embeddings that are robust against sensitive attribute leakage. Furthermore, PES-DAT dynamically tunes the privacy-utility tradeoff during training via the Laplace mechanism, adapting privacy guarantees as learning progresses. Evaluations on benchmark clustering datasets, including human activity and campus behavior data, show that PES-DAT achieves up to 10% higher silhouette scores and reduces privacy leakage by 30% compared to state-of-the-art baselines such as AIB and PPGAN. Our results demonstrate that PES-DAT is a robust and flexible solution for privacy-preserving representation learning in both unsupervised and semi-supervised settings
References
L. Ge, H. Li, X. Wang, and Z. Wang, “A review of secure federated learning: Privacy leakage threats, protection technologies, challenges and future directions,” Neurocomputing, vol. 561, p. 126897, 2023.
L. Corò, “Privacy leakage analysis of text representations for natural language processing,” unpublished.
P. Omid and F. Soren, “The digital double: Data privacy, security, and consent in AI implants,” Digit. J. Eng. Sci. Technol., vol. 2, no. 1, p. 105, 2025.
Y. Shen, Z. Ji, J. Lin, and K. Koedginer, “Enhancing the de-identification of personally identifiable information in educational data,” arXiv preprint arXiv:250109765, 2025.
Ö. Öksüz, “Preserving identity leakage, data integrity and data privacy using blockchain in education system,” Int. J. Inf. Secur. Sci., vol. 11, no. 1, pp. 1–14, 2022.
N. M. Mirbahar, K. Kumar, A. A. Laghari, and M. A. Khuhro, “Crowdsourced data leaking user’s privacy while using anonymization technique,” Mehran Univ. Res. J. Eng. Technol., vol. 44, no. 2, pp. 93–116, 2025.
C. Gong, X. Zhang, Y. Lin, H. Lu, P.-C. Su, and J. Zhang, “Federated learning for heterogeneous data integration and privacy protection,” 2025.
S. Su, Y. Luo, T. Li, Q. Chen, and J. Liang, “Anti-leakage method of sensitive information of network documents based on differential privacy model,” Secur. Privacy, vol. 8, no. 1, p. e491, 2025.
P. Zhang, H. Sun, Z. Zhang, X. Cheng, Y. Zhu, and J. Zhang, “Privacy-preserving recommendations with mixture model-based matrix factorization under local differential privacy,” IEEE Trans. Ind. Informat., 2025.
C. Gilbert and M. Gilbert, “Privacy-preserving data mining and analytics in big data environments,” SSRN Electron. J., 2024.
D. Franzen, C. Müller-Birn, and O. Wegwarth, “Communicating the privacy-utility trade-off: Supporting informed data donation with privacy decision interfaces for differential privacy,” Proc. ACM Hum.-Comput. Interact., vol. 8, no. CSCW1, pp. 1–56, 2024.
Y.-S. Kil, Y.-J. Lee, S.-E. Jeon, Y.-S. Oh, and I.-G. Lee, “Optimization of privacy-utility trade-off for efficient feature selection of secure Internet of Things,” IEEE Access, 2024.
M. Eleks, I. Jakob, J. Rebstadt, H. Kortum-Landwehr, and O. Thomas, “Privacy, utility, effort, transparency and fairness: Identifying and swaying trade-offs in privacy preserving machine learning through hybrid methods,” in INFORMATIK 2024. Bonn, Germany: Gesellschaft für Informatik e.V., 2024, pp. 43–57.
W. Abbasi, P. Mori, and A. Saracino, “Trading-off privacy, utility, and explainability in deep learning-based image data analysis,” IEEE Trans. Dependable Secure Comput., 2024.
Q. Geng and P. Viswanath, “The optimal noise-adding mechanism in differential privacy,” IEEE Trans. Inf. Theory, vol. 62, no. 2, pp. 925–951, 2015.
M. Mohammadi et al., “Differential privacy for deep learning in medicine,” arXiv preprint arXiv:2506.00660, 2025.
H. Yang, L. Fu, Q. Lu, Y. Fan, T. Zhang, and R. Wang, “Research on the design of a short video recommendation system based on multimodal information and differential privacy,” arXiv preprint arXiv:2504.08751, 2025.
X. Zhang et al., “BGTplanner: Maximizing training accuracy for differentially private federated recommenders via strategic privacy budget allocation,” arXiv preprint arXiv:2412.02934, 2024.
I. Perez et al., “Locally differentially private embedding models in distributed fraud prevention systems,” in Proc. 2023 IEEE Int. Conf. Data Mining Workshops (ICDMW), 2023, pp. 475–484.
W. Jiang, Z. Ma, S. Li, H. Xiao, and J. Yang, “Privacy budget management and noise reusing in multichain environment,” Int. J. Intell. Syst., vol. 37, no. 12, pp. 10462–10475, 2022.
G. Garbuzov, “Issues in detecting confidential information leaks in unstructured data,” Int. J. Open Inf. Technol., vol. 13, no. 4, pp. 26–32, 2025.
C. Tao et al., “Siamese image modeling for self-supervised vision representation learning,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2023, pp. 2132–2141.
Z. Chen, L. Miao, X. Yang, and J. Ouyang, “Improving stability and generalization of magnetic anomaly detection using deep convolutional Siamese neural networks,” IEEE Sensors J., 2024.
A. Golfe, A. Colomer, J. Padres, and V. Naranjo, “Enhancing image retrieval performance with generative models in Siamese networks,” IEEE J. Biomed. Health Informat., 2025.
C. L. Seok, Y. D. Song, B. S. An, and E. C. Lee, “Photoplethysmogram biometric authentication using a 1D Siamese network,” Sensors, vol. 23, no. 10, p. 4634, 2023.
X. Zeng, X. Wang, and Y. Xie, “Multiple pseudo-Siamese network with supervised contrast learning for medical multi-modal retrieval,” ACM Trans. Multimedia Comput. Commun. Appl., vol. 20, no. 5, pp. 1–23, 2024.
X. Zhou, Z. Zhang, L. Wang, and P. Wang, “A model based on Siamese neural network for online transaction fraud detection,” in Proc. Int. Joint Conf. Neural Netw. (IJCNN), 2019, pp. 1–7.
A. Fedele, R. Guidotti, and D. Pedreschi, “Explaining Siamese networks in few-shot learning,” Mach. Learn., vol. 113, no. 10, pp. 7723–7760, 2024.
Y. Ganin and V. Lempitsky, “Unsupervised domain adaptation by backpropagation,” in Proc. Int. Conf. Mach. Learn. (ICML), 2015, pp. 1180–1189.
A. Jaiswal, A. R. Babu, M. Z. Zadeh, D. Banerjee, and F. Makedon, “A survey on contrastive self-supervised learning,” Technologies, vol. 9, no. 1, p. 2, 2020.
K. Zhou, Z. Liu, Y. Qiao, T. Xiang, and C. C. Loy, “Domain generalization: A survey,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 4, pp. 4396–4415, 2022.
L. Lu et al., “Adversarial training for multimodal large language models against jailbreak attacks,” arXiv preprint arXiv:2503.04833, 2025.
C. Liu, M. Salzmann, T. Lin, R. Tomioka, and S. Süsstrunk, “On the loss landscape of adversarial training: Identifying challenges and how to overcome them,” Adv. Neural Inf. Process. Syst., vol. 33, pp. 21476–21487, 2020.
T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” in Proc. Int. Conf. Mach. Learn. (ICML), 2020, pp. 1597–1607.
M. Abadi et al., “Deep learning with differential privacy,” in Proc. ACM SIGSAC Conf. Comput. Commun. Secur. (CCS), 2016, pp. 308–318.
J.-B. Grill et al., “Bootstrap your own latent: A new approach to self-supervised learning,” Adv. Neural Inf. Process. Syst., vol. 33, pp. 21271–21284, 2020.
W. Li, A. Yan, T. Zhu, T. Huang, X. Luo, and S. Wang, “Towards differentially private contrastive learning,” in Int. Conf. Mach. Learn. Cyber Secur. Cham, Switzerland: Springer, 2022, pp. 510–520.[38] W. Li et al., “DPCL: Contrastive representation learning with differential privacy,” Int. J. Intell. Syst., vol. 37, no. 11, pp. 9701–9725, 2022.
Y. Kim, D. Cho, and S. Hong, “Towards privacy-preserving domain adaptation,” IEEE Signal Process. Lett., vol. 27, pp. 1675–1679, 2020.
R. Shokri, M. Stronati, C. Song, and V. Shmatikov, “Membership inference attacks against machine learning models,” in Proc. IEEE Symp. Secur. Privacy (SP), 2017, pp. 3–18.
M. Fredrikson, S. Jha, and T. Ristenpart, “Model inversion attacks that exploit confidence information and basic countermeasures,” in Proc. 22nd ACM SIGSAC Conf. Comput. Commun. Secur. (CCS), 2015, pp. 1322–1333.
S. Narayanan, “Cluster analysis in high dimensions: Robustness, privacy, and beyond,” M.S. thesis, Massachusetts Institute of Technology, Cambridge, MA, USA, 2024.
S. Xie, Y. Wu, J. Li, M. Ding, and K. B. Letaief, “Privacy for fairness: Information obfuscation for fair representation learning with local differential privacy,” arXiv preprint arXiv:2402.10473, 2024.
Ambreen, M. A. Khuhro et al., “A survey on fashion recommendation systems,” vol. 8, no. 1, 2024.
N. Mirbahar, M. A. Khuhro, S. Hussain, S. Memon, and S. A. Awan, “Person identification through harvesting kinetic energy,” University of Sindh Journal of Information and Communication Technology (USJICT), vol. 5, no. 2, pp. 95–100, 2021.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 University of Sindh Journal of Information and Communication Technology
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
University of Sindh Journal of Information and Communication Technology (USJICT) follows an Open Access Policy under Attribution-NonCommercial CC-BY-NC license. Researchers can copy and redistribute the material in any medium or format, for any purpose. Authors can self-archive publisher's version of the accepted article in digital repositories and archives.
Upon acceptance, the author must transfer the copyright of this manuscript to the Journal for publication on paper, on data storage media and online with distribution rights to USJICT, University of sindh, Jamshoro, Pakistan. Kindly download the copyright for below and attach as a supplimentry file during article submission
