CHANNEL ASYMMETRY IN MULTI-COMPONENT PHISHING DETECTION: A CROSS-VALIDATED COMPARISON OF URL, HTML AND EMAIL-TEXT CLASSIFIERS

Authors

  • Nazokat Mirtakhirova Department of Computer Science, New Uzbekistan University Tashkent, Uzbekistan
  • Danish Shehzad Department of Computer Science, New Uzbekistan University Tashkent, Uzbekistan

Keywords:

phishing detection; DistilBERT; Random Forest; cross-validation; channel fusion; model reliability

Abstract

Multi-component phishing detectors are normally reported as a single headline accuracy, which conceals how differently the individual channels behave. This paper re-examines a three-component lightweight framework — a DistilBERT classifier over URLs, a Random Forest over thirteen structural HTML features, and a second DistilBERT over email body text — and asks not how well the framework performs but how unequally its channels perform. The three channels receive 99.87 %, 93.81 % and 96.79 % accuracy under the same five-fold cross-validation procedure. The spread in point estimates, however, is the less important part of the story: the standard deviation of accuracy across folds ranges by a factor of 15.7 from the URL to the email channel, and the standard deviation of recall by a factor of 37.5. Both content channels also exhibit high precision above recall, meaning the errors are in missed phishing rather than false alarms, and that the imbalance is largest (1.27 pp) in the email channel that faces the human victim. We argue that the per-channel error rate alone is not enough for designing a fusion layer and the dispersion across folds and the direction of precision–recall gap should be reported together.

References

1.A. C. Bahnsen, E. C. Bohorquez, S. Villegas, J. Vargas, and F. A. González, “Classifying phishing URLs using recurrent neural networks,” in Proc. APWG Symp. Electronic Crime Research (eCrime), 2017, pp. 1–18.

2.H. Le, Q. Pham, D. Sahoo, and S. C. H. Hoi, “URLNet: Learning a URL representation with deep learning for malicious URL detection,” arXiv:1802.03162, 2018.

3.K. Haynes, H. Shirazi, and I. Ray, “Lightweight URL-based phishing detection using natural language processing transformers for mobile devices,” Procedia Computer Science, vol. 191, pp. 127–134, 2021.

4.A. Aljofey et al., “An effective detection approach for phishing websites using URL and HTML features,” Scientific Reports, vol. 12, 8842, 2022.

5.E. Opare et al., “HTML-based phishing detection using deep learning,” 2024.

6.Y. Fang, C. Zhang, C. Huang, L. Liu, and Y. Yang, “Phishing email detection using improved RCNN model with multilevel vectors and attention mechanism,” IEEE Access, vol. 7, pp. 56329–56340, 2019.

7.S. Jamal et al., “Improved phishing email detection using transformer models,” 2024.

8.V. Sanh, L. Debut, J. Chaumond, and T. Wolf, “DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter,” arXiv:1910.01108, 2019.

9.J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proc. NAACL-HLT, 2019, pp. 4171–4186.

10.L. Breiman, “Random forests,” Machine Learning, vol. 45, no. 1, pp. 5–32, 2001.

11.M. T. Ribeiro, S. Singh, and C. Guestrin, “‘Why should I trust you?’ Explaining the predictions of any classifier,” in Proc. ACM SIGKDD, 2016, pp. 1135–1144.

Downloads

Published

2026-10-08

Conference Proceedings Volume

Section

Articles

Similar Articles

1 2 3 4 5 6 7 8 9 10 > >> 

You may also start an advanced similarity search for this article.