Q-LVS: A Q-Learning-based Algorithm for Video Streaming in Peer-to-Peer Networks Considering a Token-Based Incentive Mechanism

Imanimehr, Z.

doi:10.22044/jadm.2022.11293.2287

Document Type : Viewpoint/Perspective/Opinion

Author

Z. Imanimehr

Computer Engineering, University of Qom, Qom, Iran.

https://doi.org/10.22044/jadm.2022.11293.2287

Abstract

Peer-to-peer video streaming has reached great attention during recent years. Video streaming in peer-to-peer networks is a good way to stream video on the Internet due to the high scalability, high video quality, and low bandwidth requirements. In this paper the issue of live video streaming in peer-to-peer networks which contain selfish peers is addressed. To encourage peers to cooperate in video distribution, tokens are used as an internal currency. Tokens are gained by peers when they accept requests from other peers to upload video chunks to them, and tokens are spent when sending requests to other peers to download video chunks from them. To handle the heterogeneity in the bandwidth of peers, the assumption has been made that the video is coded as multi-layered. For each layer the same token has been used, but priced differently per layer. Based on the available token pools, peers can request various qualities. A new token-based incentive mechanism has been proposed, which adapts the admission control policy of peers according to the dynamics of the request submission, request arrival, time to send requests, and bandwidth availability processes. Peer-to-peer requests could arrive at any time, so the continuous Markov Decision Process has been used.

Keywords

20.1001.1.23225211.2022.10.3.10.2

References

[1] R. Schollmeier, "A definition of peer-to-peer networking for the classification of peer-to-peer architectures and applications", In Proceedings First International Conference on Peer-to-Peer Computing, IEEE, pp. 101-102, 2001.

[2] M. Karakaya, I. Korpeoglu, and Ö. Ulusoy, "Free riding in peer-to-peer networks", IEEE Internet computing, Vol. 13, No. 2, pp. 92-98, 2009.

[3] S. Saroiu, P. K. Gummadi, and S. D. Gribble. "Measurement study of peer-to-peer file sharing systems", In Multimedia Computing and Networking 2002, Vol. 4673, pp. 156-170. International Society for Optics and Photonics, 2001.

[4] G. Koloniari and A. Sifaleras. "Game-theoretic approaches in cloud and P2P networks: issues and challenges", Operational Research in the Digital Era–ICT Challenges, pp. 11-22, 2019.

[5] Lin, W. Sabrina, H. Vicky Zhao, and KJ Ray Liu. W. S. Lin, H. Vicky Zhao, and K. J. Ray Liu, "A game theoretic framework for incentive-based peer-to-peer live-streaming social networks", In 2008 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 2141-2144. IEEE, 2008.

[6] W. S. Lin, H. Vicky Zhao, and K. J. Ray Liu, "Incentive cooperation strategies for peer-to-peer live multimedia streaming social networks", IEEE transactions on multimedia 11, No. 3, pp. 396-412, 2009.

[7] Y. Zhang, and M. van der Schaar. "Designing incentives for P2P multimedia sharing", In 2011 IEEE Global Telecommunications Conference-GLOBECOM 2011, pp. 1-6, IEEE, 2011.

[8] V. Pai and A. E. Mohr, "Improving robustness of peer-to-peer streaming with incentives", 1st NetEcon, 2006.

[9] X. Xiao, Q. Zhang, Y. Shi and Y. Gao, "How much to share: a repeated game model for peer-to-peer streaming under service differentiation incentives", IEEE Transactions on Parallel and Distributed Systems 23, No. 2, pp. 288-295, 2011.

[10] A. Habib and J. Chuang, "Service differentiated peer selection: an incentive mechanism for peer-to-peer media streaming", IEEE Transactions on Multimedia 8, No. 3, pp. 610-621, 2006.

[11] Sh. Babazadeh, M., H. R. Navidi, H. Haj Seyed Javadi, and M. HosseinZadeh, "A New Incentive Mechanism to Detect and Restrict Sybil Nodes in P2P File-Sharing Networks with a Heterogeneous Bandwidth", Journal of AI and Data Mining 8, No. 4, pp. 557-571, 2020.

[12] E. Maani, Ch. Zhaofu, and A. K. K atsaggelos. "A game theoretic approach to video streaming over peer-to-peer networks", Signal processing: image communication 27, No. 5, 545-554, 2012.

[13] R. Aslani, V. Hakami, and M. Dehghan, "A token-based incentive mechanism for video streaming applications in peer-to-peer networks", Multimedia Tools and Applications 77, No. 12, pp. 14625-14653, 2018.

[14] Z. ImaniMehr, and M. DehghanTakhtFooladi, "Token-based incentive mechanism for peer-to-peer video streaming networks", The Journal of Supercomputing 75, No. 10, pp. 6612-6631, 2019.

[15] B. Alotibi, N. Alarifi, M. Abdulghani, and L. Altoaimy, "Overcoming free-riding behavior in peer-to-peer networks using points system approach", Procedia Computer Science 151, pp. 1060-1065, 2019.

[16] H. Kurdi, A. Althnian, M. Abdulghani, and S. Alkharji, "An Adjusted Free-Market-Inspired Approach to Mitigate Free-Riding Behavior in Peer-to-Peer Fog Computing", Electronics 9, No. 12, p. 2027, 2020.

[17] J. R. Douceur, "The sybil attack", In International workshop on peer-to-peer systems, pp. 251-260. Springer, Berlin, Heidelberg, 2002.

[18] P. Marbach, “Simulation-based methods for Markov decision processes”, Citeseer, 1998.

[19] S. J. Bradtke, and M. O. Duff, "Reinforcement Learning Methods for Continuous-Time Markov Decision", Advances in Neural Information Processing Systems 7 7, p. 393, 1995.

[20] D. P. Bertsekas and J. N. Tsitsiklis, “Neuro-dynamic programming”, Athena Scientific, 1996.

[21] H. Tong and T. X. Brown, "Reinforcement learning for call admission control and routing under quality of service constraints in multimedia networks." Machine Learning 49, No. 2, pp. 111-139, 2002.

[22] C. J. C. H. Watkins, "Learning from delayed rewards", 1989.

[23] T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra, "Overview of the H. 264/AVC video coding standard", IEEE Transactions on circuits and systems for video technology 13, No. 7, pp.560-576, 2003.

[24] H. Schwarz, D. Marpe, and T. Wiegand, "Overview of the scalable video coding extension of the H. 264/AVC standard", IEEE Transactions on circuits and systems for video technology 17, No. 9, pp. 1103-1120, 2007.

[25] http://ffmpeg.org/.

Q-LVS: A Q-Learning-based Algorithm for Video Streaming in Peer-to-Peer Networks Considering a Token-Based Incentive Mechanism

References

References

Volume 10, Issue 3July 2022Pages 411-422

Volume 10, Issue 3
July 2022
Pages 411-422