{"\ufeffHIGH PERFORMANCE THROUGH WALL HUMAN ACTIVITY RECOGNITION USING WIFI \n\nFahd Abuhoureyah1, Yan Chiew Wong1*, Ahmad Sadhiqin bin Mohd Isira1, Joon Huang Chuah2\n\n1Centre for Telecommunication Research and Innovation (CETRI)\u00a0\nFakulti Kejuruteraan Elektronik dan Kejuruteraan Komputer (FKEKK), Universiti Teknikal Malaysia Melaka (UTeM), 76100 durian Tunggal, Melaka, Malaysia\n\n2Department of Electrical Engineering, Faculty of Engineering, University of Malaya, 50603 Kuala Lumpur, Malaysia\n\n*Corresponding Author's Email: ycwong@utem.edu.my\n\nArticle History: Received March 10, 2023": null, " Revised May 12, 2023": null, " \nAccepted May 27, 2023\n\nABSTRACT: Passive human activity recognition without requiring a device is crucial in various fields, including smart homes, health care, and identification. However, current systems for human activity recognition require a dedicated device, or they need to be more suitable for scenarios where signals are transmitted through walls. To address this challenge, we propose a device-free, passive recognition system of human activity that utilizes CSI-based WiFi signals and does not require any dedicated devices. The proposed approach uses two techniques to distinguish different human activities. First, we introduce an opposite robust method to eliminate the influence of the background environment on correlation extraction and to obtain the correlation between human activity and its resulting changes in channel state information values. Second, we propose a normalized variance sliding windows algorithm to segment the time of human action from the waveforms, which can differentiate human actions' start and end times. We also implemented a CSI-based model using Nexmon with an LSTM algorithm with commodity WiFi devices and evaluated it in several environments. Our experimental results demonstrate that we achieve an average accuracy of 95% when signals pass through concrete walls.\n\nKEYWORDS: Through wall HAR, WiFi Sensing, Passive Device-Free HAR, CSI HAR.\n\n    1.0 INTRODUCTION\nHuman activity recognition is a fundamental research topic in pervasive computing and human-computer interaction. It has the potential to support many emerging applications such as smart homes, augmented reality, identification, and health care. Different human activity recognition systems have been proposed with various techniques, including wearable sensor-based, computer vision-based, and ambient device-based approaches [1]. Wearable sensor-based methods utilize sensors, RFID [2], smartphones, and other devices to recognize human activities [3]. These systems use active detection techniques and require the device to be always on the body. On the other hand, computer vision-based approaches use cameras to capture image sequences and recognize human activities using activity classification algorithms [4]. However, the illumination of lights and shelters, such as walls, affects the camera, which limits its effectiveness.\nAmbient device-based approaches attempt to recognize human activities using radar, infrared, audio, and other sensors. While these techniques do not require wearing or carrying anything on the body, some dedicated devices should be deployed. Recently, WiFi signals-based human activity recognition techniques have been widely studied. These techniques exploit Channel State Information (CSI) and multi-antennas techniques to distinguish different activities, such as E-eyes [5], CARM [6], WifiU, RT-Fall [7], and others. These systems have the advantages of passive detection and easy deployment, but they did not consider human activity recognition under scenarios where WiFi signals pass through walls [8].\nHAR through walls using WiFi is a promising method because it provides a non-invasive and low-cost way to monitor human activities. WiFi signals penetrate walls and objects, allowing them to be used to sense the presence and movement of humans in indoor environments. The technique relies on analyzing the changes in the wireless signals as they interact with the human body and surrounding objects. Various signal processing and machine learning techniques are applied to extract features from the received signal, which classify human activities such as walking, running, sitting, or standing [4].\nTo realize the system, we face two main technical challenges. The first challenge is to obtain beneficial human activity CSI correlation from the raw CSI measurements. It is a challenging problem to get this correlation because the WiFi signals are affected severely by the wall and the indoor physical environment (e.g., reflection, diffraction, and scattering). These aspects cause the received signal to become weak and mix a large amount of complicated background environmental information. The second technical challenge is to segment activity from CSI waveforms. Existing work shows that the activity can be segmented easily. However, the changes in CSI waveform caused by some human activities are not noticeable when the WiFi signals pass through the wall.\nWe propose a robust LSTM approach to overcome these challenges to obtain the correlation between human activity and its resulting changes in CSI values. This proposal includes the correlation of the indoor background environment and noise, and if weakly affected by the environment. The contributions of our work are summarized as follows:\n    \u2022 The present work introduces a novel approach for human activity recognition without using any dedicated device by leveraging WiFi signals that can penetrate through walls. We employ CSI amplitude to recognize human activities with commodity WiFi devices through walls.\n    \u2022 The proposed system has been implemented using commercial WiFi devices with a singular transmitter and receiver. The proposed model achieves an average classification accuracy of 95% for HAR based on CSI signals passing through the wall.\n    2.0 RELATED WORKS\nIn recent decades, wireless detection has gained popularity as a contactless sensing technique that appeals to simplicity and manageable availability [9]. Likewise, signals can spread through walls, furniture, and doors, and they don't need a line of sight (LoS), allowing for more significant detection regions [8]. Numerous CSI and machine learning-based algorithms have been presented to correctly identify and count individuals in each coverage area of a WiFi sense. Most of these studies use 1-3 transmitters (Tx) and three reception antennas (Rx) and utilize different wireless network cards.\nHowever, the network interface card NIC frequently has various constraints due to the natural physical effects of the transmission process. A quick review of the literature shows research that utilizes CSI data derived from standard WiFi equipment, such as NICs, that provide a restricted number of data subcarriers and severely influence classification performance [10]. Moreover, NICs are typically utilized for networking tasks": null, " therefore, their dependability might be greatly harmed if they were also employed for sensing.\nSeveral research studies have investigated the field of human activity recognition through walls using WiFi signals, commonly referred to as \"through wall human activity recognition using WiFi,\" resulting in various proposed systems such as WiVi [11], WiSee [12], E-eyes [5], CARM [6], and  RT-Fall [7]. However, most of these systems are limited in their ability to recognize human activity through walls, as they either require specialized equipment or need to consider the scenarios where signals pass through obstacles.\nWiVi [11], for instance, uses WiFi signals through walls to detect objects and human actions. Still, it requires using PCs instead of commercial WiFi devices and performs poorly in activity classification. E-eyes [5] uses commercial WiFi devices to monitor human activity with one transmitter and three receivers, which achieve in-place and walking activity recognition. CARM [6] exploits CSI-speed and CSI-activity models to recognize different activities. In contrast, RT-Fall[7], on the other hand, exploits both the amplitude and phase of CSI measurements to detect falls.\nDespite the advantages of these systems, including passive detection and easy deployment, they do not consider the scenarios where signals pass through walls. In indoor environments, where multiple rooms may access one wireless signals AP, walls may block all the direct and reflected propagation paths between the receiver and the transmitter. The system exploits the differences in the CSI values measured to classify the activities of interest. This approach has the potential to significantly improve the accuracy and reliability of HAR systems in scenarios where signals pass through walls, making it suitable for a wide range of applications, including home automation, security, and healthcare.\n    3.0 METHODLOGY\n        3.1 Primarily and problem analysis\nCSI is widely used to capture activity data as it provides high sensitivity to channel link variations and has a fine-grained nature and small size, as shown in equation (3.1) and a schematic network diagram.\n\n(3. 1)\nIn the equation, H is a complex matrix of CSI values, and n is the channel noise [13]. The MIMO system utilizes multiple channels to increase transmission rates by constructing a matrix of connection links represented as: -\n\n(3. 2)\nThe CSI measures the amplitudes and phases influenced by the paths and the number of amplitudes and phase shifts Therefore, each CSI entry corresponds to the channel frequency response, as shown in equation (3.3).\n\n(3. 3)\nThe CSI ratio of WIFI indicates the impact of objects in the environment on the transmission of OFDM signals, causing signal weakening and scattering [14], [15]. This rapidly amplifies human motion's effect on the phase variation, as expressed mathematically in equation (3.4) as:\n\n(3. 4)\nWhereby (t) represents the intensity of impulsive noise, and (t) represents the time-varying phase offset. L represents the total number of propagating routes,  represents the wavelengths, and A1(t) and d1(t) represent the signal's attenuation and the L pathway's length, respectively.\n\n\nFigure 1: The utilization of WiFi for human activity recognition through walls demonstrates the presence of weak reflected signals\nThe analysis of activities, as depicted in Figure 1, confirms exceptional diversity in packet transmission through the wall. One of the main challenges is the complex and unpredictable propagation of wireless signals through the wall, which can cause severe signal attenuation, reflection, and multipath effects. Another challenge is collecting and processing high-quality CSI data due to commercial WiFi devices' limited bandwidth and sensitivity. Additionally, there is a need to develop robust and accurate algorithms to extract meaningful features from the CSI data and classify different human activities under various scenarios and environments.\n\n(3. 5)\nThe provided equation considers the received power (Pr), transmitted power (PT), the gain of transmission (GT), and receiving (GR), as well as the distance (R) that signals need to travel across the wall and the amount of power that is reflected into the adjacent room, where only a small fraction of power is reflected to the receiver with a cross-sectional area of \u03c3. Moreover, environmental factors, represented by F, significantly affect the equation. In the proposed model, the reflected power within the same room is removed, and attention is given to the body effects in the adjacent room.\n        3.2 System Overview\nThe proposed system has two WiFi devices serving as a transmitter and receiver. A commercially available WiFi access point with one antenna is the transmitter, while a common wireless NIC is the receiver. The experiments used a TP-Link EC230-g1 as the transmitter and a Raspberry Pi 4b as the receiver. The AP and receiver are placed in different adjacent rooms, and the wall obstructs all direct and reflected propagation paths between them. If human activity occurs in either the transmitter or receiver room, the system can automatically detect it. Figure 2 illustrates the system's principle of operation.\n\nFigure 2: System illustration of human activity recognition when the signals pass through the wall\nThe methodology of the system is described in this work through a flow chart consisting of four main stages: data collection, pre-processing, feature extraction, and classification. The data collection process involves using a Raspberry Pi device connected to Nexmon firmware to capture raw data of activities in the adjacent room. The proposed approach is focused on predicting activities through walls using wide-angle views. The captured data is subjected to various techniques for extraction and filtering, such as FFT, Hampel, and median, to eliminate noise for visualization and training purposes. The feature extraction stage involves extracting meaningful features from the pre-processed data to represent the activities accurately. Finally, a three-layer LSTM network with sequential connections and SoftMax function is employed for activity recognition and classification. \n\nFigure 3: System Architecture\n\n        3.3 Location Correlation with Extraction\nWe employed the median of Hampel filter processes to denoise matrix signals. The median filter is a nonlinear digital filtering technique that replaces each pixel value in a signal with the median value of its neighboring values. This filter effectively removes salt-and-pepper noise, a common type of noise in the signal. On the other hand, the Hampel filter process is a robust method for outlier detection that uses a sliding window to detect outliers in a movement. It replaces the outlier value with the median value of the window. Combining these two filters effectively reduces noise in matrix signals while preserving the integrity of the underlying data. By applying these denoising techniques, we improved the accuracy of signal analysis and interpretation shown in Figure 4.\n\nFigure 4: Denoising signal\nAs a person moves, their body reflects and scatters the wireless signals, causing variations in the CSI values. By analyzing these variations, it is possible to identify specific human activities such as walking, sitting, or standing shown in Figure 5. One advantage of using CSI signals for activity recognition is that they are non-intrusive and do not require wearable sensors. Additionally, they can be collected using commodity WiFi devices, making the approach cost-effective and widely accessible. By leveraging the amplitude changes in CSI signals caused by body movement\n\nFigure 5: Human activity recognition when the signals pass through the wall\nIt was observed that the activity recognition system proposed in the research could not capture reflected signals from the next room, as evidenced by equation 5. As a result, it was necessary to separate the activity from the initial room signal reflections to achieve accurate activity recognition. This separation was achieved using a signal processing technique that effectively filters unwanted reflections and isolates the relevant activity signal shown in Figure 6.\n\nFigure 6: Raw CSI amplitude environmental separation results\n\n    4.0 IMPLEMENTATION AND EVALUATION\n        4.1 Implementation \nIn this work, the Raspberry Pi 4B was employed to capture data and to evaluate the performance of the activity recognition model for through-wall sensing of six different activities (empty, lying, walking, running, standup, and falling). The Raspberry Pi was set to monitoring mode at 20 MHz and 80 MHz at 2.4 GHz/5 GHz, respectively, using a transmitter (Tx) (TP-LINK router) and a receiver (Rx) Raspberry Pi 4b, both of which utilized omnidirectional antennas. Furthermore, data injection was necessary, for which the Lenovo G580 laptop was used to inject the router in these experiments to achieve flow data in monitoring mode. The data is collected in the first stage using Wireshark/TCPDUMP in the monitoring mode of Nexmon installed on the Raspberry Pi.\n        4.2 Experimental Setup\nThe evaluation was implemented in the lab environment shown in Figure 7 to investigate the wider-angle model in different aspects, such as frequency, distance, and angle. The figure is divided into five positions, each subject to six regular activities. The study is conducted using both 2.4 GHz frequencies. The distance between the transmitter and the router is 2 meters, and the spaces between the receiver and the different positions are labeled in Figure 7.\n\nFigure 7: Layout of evaluation location\n\n        4.3 Human Activity Recognition\nThe experimental results demonstrate that the proposed system can recognize human activities through a wall, as illustrated in Figure 8. The model achieved this by classifying the separated signal using machine learning techniques, including low-rank matrix decomposition, feature extraction, and an LSTM neural network. These techniques enabled the model to extract relevant features from the raw signal and learn the temporal dependencies in the signal, resulting in accurate recognition of the target activities. \n\nFigure 8: Activity classification\nWith high accuracy, the system could classify six activities, including empty, lying, walking, running, standing up, and falling, with a few samples shown in the confusion matrix in Figure 9. \n\nFigure 9: Confusion matrix of through wall HAR using 2.4GHz WiFi\nThe models exhibited remarkable proficiency in identifying human activities through walls via WiFi signals in five distinct locations, as presented in Table 1. The implications of these findings are substantial, particularly for fields such as security and surveillance, healthcare, and home automation.\n\n\nTable 1: The efficacy of activity detection through a wall in five positions is shown in Figure 7.\nFreq\n2.4Ghz\nPositions\nPosition 1\nPosition 2\nPosition 3\nPosition 4\nPosition 5\nEmpty\n100\n100\n100\n100\n100\nFall\n100\n40\n100\n100\n100\nLie Down\n77.5\n92.5\n80\n100\n85\nRun\n100\n96.7\n96.7\n96.7\n96.7\nSit-down\n96.7\n96.7\n96.7\n80\n96.7\nWalk\n100\n100\n96.7\n96\n100\nAcc.%\n94 %\n92.4%\n85%\n95%\n95.6%\n    5.0 CONCL U S ION \nThe proposed model provides a narrative approach to improve the forecasting of through-wall human activities through signal processing in human activity recognition systems. Using the CSI signals captured through the wall, we showed that the proposed system accurately classified six activities (empty, lying, walking, running, sitting down, and falling). We also demonstrated the effectiveness of the system in different positions and angles, which can provide valuable insights for real-world deployment. The proposed approach is low-cost and easy to implement, making it suitable for various applications. The results of this study suggest that CSI-based HAR using WiFi and LSTM can be a viable solution for activity recognition through walls, with potential applications in healthcare, security, and intelligent home systems. Further research can explore the feasibility of using this system in larger spaces with more complex activities. Future research will focus on refining the localization and data extraction algorithm with enhanced technologies such as beamforming and array antennas.\n    6.0 ACKNOWLEDGMENTS \nThis work was initially accepted for and presented at ICTEC2022. The authors acknowledge the technical and financial support by Universiti Teknikal Malaysia Melaka (UTeM), Malaysia, and the Ministry of Higher Education, Malaysia, under the research grant no FRGS/1/2020/ICT02/UTEM/02/1\n\n\n\n    7.0 REFERENCES \n[1]\tC. Shi et al., \"Demo: Device-free Activity Monitoring Through Real-time Analysis on Prevalent WiFi Signals,\" 2019 IEEE Int. Symp. Dyn. Spectr. Access Networks, DySPAN 2019, pp. 1\u20132, 2019.\n[2]\tH. Obeidat, W. Shuaieb, O. Obeidat, and R. Abd-Alhameed, A Review of Indoor Localization Techniques and Wireless Technologies, vol. 119, no. 1. Springer US, 2021.\n[3]\tY. Zhang, Y. Yin, Y. Wang, J. Ai, and D. Wu, \"CSI-based location-independent Human Activity Recognition with parallel convolutional networks,\" Comput. Commun., vol. 197, no. November 2022, pp. 87\u201395, 2023.\n[4]\tX. Wu, Z. Chu, P. Yang, C. Xiang, X. Zheng, and W. Huang, \"TW-See: Human activity recognition through the wall with commodity WiFi Devices,\" IEEE Trans. Veh. Technol., vol. 68, no. 1, pp. 306\u2013319, 2019.\n[5]\tY. Wang, J. Liu, Y. Chen, M. Gruteser, J. Yang, and H. Liu, \"E-eyes: Device-free location-oriented activity identification using fine-grained WiFi signatures,\" Proc. Annu. Int. Conf. Mob. Comput. Networking, MOBICOM, no. October 2015, pp. 617\u2013628, 2014.\n[6]\tW. Wang, A. X. Liu, M. Shahzad, K. Ling, and S. Lu, \"Device-Free Human Activity Recognition Using Commercial WiFi Devices,\" IEEE J. Sel. Areas Commun., vol. 35, no. 5, pp. 1118\u20131131, 2017.\n[7]\tT. Z. Chowdhury, \"Using WiFi channel state information (CSI) for human activity recognition and fall detection,\" no. April, pp. 24\u201350, 2018. \n[8]\tS. Di Domenico, M. De Sanctis, E. Cianca, and M. Ruggieri, \"WiFi-based through-the-wall presence detection of stationary and moving humans analyzing the doppler spectrum,\" IEEE Aerosp. Electron. Syst. Mag., vol. 33, no. 5\u20136, pp. 14\u201319, 2018.\n[9]\tJ. Li, K. Yin, and C. Tang, \"Slideaugment: A simple data processing method to enhance human activity recognition accuracy based on WiFi,\" Sensors, vol. 21, no. 6, pp. 1\u201313, 2021.\n[10]\tQ. Li, R. Gravina, Y. Li, S. H. Alsamhi, F. Sun, and G. Fortino, \"Multi-user activity recognition: Challenges and opportunities,\" Inf. Fusion, vol. 63, no. March, pp. 121\u2013135, 2020.\n[11]\tL. Zhang, X. Ruan, and J. Wang, \"WiVi: A Ubiquitous Violence Detection System with Commercial WiFi Devices,\" IEEE Access, vol. 8, pp. 6662\u20136672, 2020.\n[12]\tS. Ajmera, N. Chouhan, and N. Audeechya, \"Wisee Technology,\" pp. 264\u2013267, [Online]. Available: www.ijert.org\n[13]\tM. Liyanage, C. Chang, S. Srirama, and S. Loke, \"Indoor people density sensing using WiFi and channel state information,\" Adv. Model. Anal. A, vol. 61, no. 1, pp. 37\u201347, 2018.\n[14]\tY. Zeng, D. Wu, J. Xiong, and D. Zhang, \"Boosting WiFi Sensing Performance via CSI Ratio,\" IEEE Pervasive Comput., vol. 20, no. 1, pp. 62\u201370, 2021.\n[15]\tY. He, Y. Chen, Y. Hu, and B. Zeng, \"WiFi Vision: Sensing, Recognition, and Detection with Commodity MIMO-OFDM WiFi,\" IEEE Internet Things J., vol. 7, no. 9, pp. 8296\u20138317. 2020.": null}