Reinforcement Learning based Node Sleep or Wake-up Time Scheduling Algorithm for Wireless Sensor Network
Department of Computer Science and Engineering, Uttaranchal University, Dehradun, Uttarakhand, India.
Department of Computer Science and Engineering, Graphic Era Deemed to be University, Dehradun, Uttarkahnd, India.
Department of Computer Science and Engineering, Government Engineering College, Bharatpur, Rajasthan, India.
Department of Computer Science and Engineering, Chandigarh University, Chandigarh, Punjab, India.
Received on December 30, 2019
Accepted on March 25, 2020
A wireless sensor network is a collection of small sensor nodes that have limited energy and are usually not rechargeable. Because of this, the lifetime of wireless sensor networks has always been a challenging area. One of the basic problems of the network has been the ability of the nodes to effectively schedule the sleep and wake-up time to overcome this problem. The motivation behind node sleep or wake-up time scheduling is to take care of nodes in sleep mode for as long as possible (without losing data packet transfer efficiency) and thus extend their useful life. This research going to propose scheduling of nodes sleeps and wake-up time through reinforcement learning. This research is not based on the nodes' duty cycle strategy (which creates a compromise between data packet delivery and nodes energy saving delay) like other existing researches. It is based on the research of reinforcement learning which gives independence to each node to choose its own activity from the transmission of packets, tuning or sleep node in each time band which works in a decentralized way. The simulation results show the qualified performance of the proposed algorithm under different conditions.
Keywords- Sleep or wake-up scheduling, Wireless sensor network, Sensor node energy.
Verma, P., Dumka, A., Vyas, D., & Bhardwaj, A. (2020). Reinforcement Learning based Node Sleep or Wake-up Time Scheduling Algorithm for Wireless Sensor Network. International Journal of Mathematical, Engineering and Management Sciences, 5(4), 707-731. https://doi.org/10.33889/IJMEMS.2020.5.4.057.
Conflict of Interest
The authors confirm that there is no conflict of interest to declare for this publication.
This research didn’t receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. The authors sincerely appreciate the editor and reviewers for their time and valuable comments.
Abdul-Salaam, G., Abdullah, A.H., & Anisi, M.H. (2017). Energy-efficient data reporting for navigation in position-free hybrid wireless sensor networks. IEEE Sensors Journal, 17(7), 2289-2297.
Acampora, G., Cook, D.J., Rashidi, P., & Vasilakos, A.V. (2013). A survey on ambient intelligence in healthcare. Proceedings of the IEEE, 101(12), 2470-2494.
Cao, Q., Abdelzaher, T., He, T., & Stankovic, J. (2005, April). Towards optimal sleep scheduling in sensor networks for rare-event detection. In International Processing in Sensor Network (IPSN) 2005. Fourth International Symposium on Information Processing in Sensor Networks, 2005. (pp. 20-27). IEEE. Boise, ID, USA.
Chen, C.P., Mukhopadhyay, S.C., Chuang, C.L., Lin, T.S., Liao, M.S., Wang, Y.C., & Jiang, J.A. (2014). A hybrid memetic framework for coverage optimization in wireless sensor networks. IEEE Transactions on Cybernetics, 45(10), 2309-2322.
Foerster, J., Nardelli, N., Farquhar, G., Afouras, T., Torr, P.H., Kohli, P., & Whiteson, S. (2017, August). Stabilising experience replay for deep multi-agent reinforcement learning. In Proceedings of the 34th International Conference on Machine Learning-Volume 70 (pp. 1146-1155). Sydney, Australia.
Fu, B., Xiao, Y., Liang, X., & Chen, C.P. (2014). Bio-inspired group modeling and analysis for intruder detection in mobile sensor/robotic networks. IEEE Transactions on Cybernetics, 45(1), 103-115.
Glavic, M., Fonteneau, R., & Ernst, D. (2017). Reinforcement learning for electric power system decision and control: Past considerations and perspectives. IFAC-PapersOnLine, 50(1), 6918-6927.
Gong, Y.J., Chen, W.N., Zhan, Z.H., Zhang, J., Li, Y., Zhang, Q., & Li, J.J. (2015). Distributed evolutionary algorithms and their models: a survey of the state-of-the-art. Applied Soft Computing, 34, 286-300.
Guo, P., Jiang, T., Zhang, Q., & Zhang, K. (2011). Sleep scheduling for critical event monitoring in wireless sensor networks. IEEE Transactions on Parallel and Distributed Systems, 23(2), 345-352.
Jang, B., Lim, J.B., & Sichitiu, M.L. (2013). An asynchronous scheduled MAC protocol for wireless sensor networks. Computer Networks, 57(1), 85-98.
Keshavarzian, A., Lee, H., & Venkatraman, L. (2006, May). Wakeup scheduling in wireless sensor networks. In Proceedings of the 7th ACM International Symposium on Mobile ad hoc networking and computing (pp. 322-333). ACM. New York, USA.
Kim, J., Lin, X., Shroff, N.B., & Sinha, P. (2009). Minimizing delay and maximizing lifetime for wireless sensor networks with anycast. IEEE/ACM Transactions on Networking, 18(2), 515-528.
Lai, S., Ravindran, B., & Cho, H. (2010). Heterogenous quorum-based wake-up scheduling in wireless sensor networks. IEEE Transactions on Computers, 59(11), 1562-1575.
Leibo, J.Z., Zambaldi, V., Lanctot, M., Marecki, J., & Graepel, T. (2017, May). Multi-agent reinforcement learning in sequential social dilemmas. In proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems (pp. 464-473). International Foundation for Autonomous Agents and Multiagent Systems, Sai Paulo, Brazil.
Li, M., Li, Z., & Vasilakos, A.V. (2013). A survey on topology control in wireless sensor networks: Taxonomy, comparative study, and open issues. Proceedings of the IEEE, 101(12), 2538-2557.
Liu, X. (2015). A deployment strategy for multiple types of requirements in wireless sensor networks. IEEE Transactions on Cybernetics, 45(10), 2364-2376.
Liu, Y.Y., & Yoo, S.J. (2017, July). Dynamic resource allocation using reinforcement learning for LTE-U and WiFi in the unlicensed spectrum. In 2017 Ninth International Conference on Ubiquitous and Future Networks (ICUFN) (pp. 471-475). IEEE. Milan, Italy.
Niyato, D., Hossain, E., Rashid, M.M., & Bhargava, V.K. (2007). Wireless sensor networks with energy harvesting technologies: A game-theoretic approach to optimal energy management. IEEE Wireless Communications, 14(4), 90-96.
Polastre, J., Hill, J., & Culler, D. (2004, November). Versatile low power media access for wireless sensor networks. In Proceedings of the 2nd international conference on Embedded networked sensor systems (pp. 95-107). ACM. Baltimore MD, USA.
Renold, A.P., & Chandrakala, S. (2017). MRL-SCSO: multi-agent reinforcement learning-based self-configuration and self-optimization protocol for unattended wireless sensor networks. Wireless Personal Communications, 96(4), 5061-5079.
Semnani, S.H., & Basir, O.A. (2014). Semi-flocking algorithm for motion control of mobile sensors in large-scale surveillance systems. IEEE Transactions on Cybernetics, 45(1), 129-137.
Singh, S., Jaakkola, T., Littman, M.L., & Szepesvári, C. (2000). Convergence results for single-step on-policy reinforcement-learning algorithms. Machine Learning, 38(3), 287-308.
Sun, Y., Du, S., Gurewitz, O., & Johnson, D.B. (2008, May). DW-MAC: a low latency, energy efficient demand-wakeup MAC protocol for wireless sensor networks. In Proceedings of the 9th ACM international symposium on Mobile ad hoc networking and computing (pp. 53-62). ACM. New York, US.
Sun, Y., Gurewitz, O., & Johnson, D.B. (2008, November). RI-MAC: a receiver-initiated asynchronous duty cycle MAC protocol for dynamic traffic loads in wireless sensor networks. In Proceedings of the 6th ACM Conference on Embedded Network Sensor Systems (pp. 1-14). ACM. Raleigh NC, USA.
Tang, L., Sun, Y., Gurewitz, O., & Johnson, D.B. (2011, May). EM-MAC: a dynamic multichannel energy-efficient MAC protocol for wireless sensor networks. In Proceedings of the Twelfth ACM International Symposium on Mobile Ad Hoc Networking and Computing (pp. 1-11). ACM. Paris, France.
Tang, L., Sun, Y., Gurewitz, O., & Johnson, D.B. (2011, April). PW-MAC: An energy-efficient predictive-wakeup MAC protocol for wireless sensor networks. In 2011 Proceedings IEEE INFOCOM (pp. 1305-1313). IEEE. Shanghai, China.
Wei, G., Ling, Y., Guo, B., Xiao, B., & Vasilakos, A.V. (2011). Prediction-based data aggregation in wireless sensor networks: Combining grey model and Kalman Filter. Computer Communications, 34(6), 793-802.
Xiao, Y., Peng, M., Gibson, J., Xie, G.G., Du, D.Z., & Vasilakos, A.V. (2011). Tight performance bounds of multihop fair access for MAC protocols in wireless sensor networks and underwater sensor networks. IEEE Transactions on Mobile Computing, 11(10), 1538-1554.
Yao, Y., Cao, Q., & Vasilakos, A.V. (2015). EDAL: An energy-efficient, delay-aware, and lifetime-balancing data collection protocol for heterogeneous wireless sensor networks. IEEE/ACM Transactions on Networking, 23(3), 810-823.
Ye, W., Heidemann, J., & Estrin, D. (2002, June). An energy-efficient MAC protocol for wireless sensor networks. In Proceedings. Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies (Vol. 3, pp. 1567-1576). IEEE. New York, USA.
Ye, W., Heidemann, J., & Estrin, D. (2004). Medium access control with coordinated adaptive sleeping for wireless sensor networks. IEEE/ACM Transactions on Networking, 12(3), 493-506.
Ye, W., Silva, F., & Heidemann, J. (2006, October). Ultra-low duty cycle MAC with scheduled channel polling. In Proceedings of the 4th International Conference on Embedded Networked Sensor Systems (pp. 321-334). ACM. Boulder Colorado, USA.
Zhang, D., Ma, Y., Zhang, Y., Lin, S., Hu, X.S., & Wang, D. (2018, April). A real-time and non-cooperative task allocation framework for social sensing applications in edge computing systems. In 2018 IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS) (pp. 316-326). IEEE.Porto, Portugal.
Zhao, Y., Liu, Y., Duan, Z., & Wen, G. (2016). Distributed average computation for multiple time‐varying signals with output measurements. International Journal of Robust and Nonlinear Control, 26(13), 2899-2915.
Zheng, R., Hou, J.C., & Sha, L. (2003, June). Asynchronous wakeup for ad hoc networks. In Proceedings of the 4th ACM International Symposium on Mobile ad hoc Networking & Computing (pp. 35-45). ACM. Maryland, USA.
Zhu, S., Chen, C., Li, W., Yang, B., & Guan, X. (2013). Distributed optimal consensus filter for target tracking in heterogeneous sensor networks. IEEE Transactions on Cybernetics, 43(6), 1963-1976.