The approach and results of identifying the most accurate collection of attributes from data acquired by embedded smartphone sensors to detect five different daily activities. In this project, we are using a LSTM feature extraction approach with 784 features to distinguish standing, sitting, walking, walking upstairs and downstairs. This approach is getting an accuracy of 92.4% and F1-score of 92.46% as an average for test, train, and validation from the data set created.

Transform the data

The 3-axis raw signals tAcc-XYZ and tGyro-XYZ from the accelerometer and gyroscope were used to create this data set. At a constant rate of 50 Hz, the time domain signals (prefix ’t’ to signify time) were collected. To reduce noise, they were filtered with a median filter and a 3rd order low pass Butterworth filter with a 20 Hz corner frequency. Using a low pass Butterworth filter with a corner frequency of 0.3 Hz, the acceleration signal was split into body and gravity acceleration signals (tBodyAcc-XYZ and tGravityAcc-XYZ).

The 3-axis signals tAcc-XYZ and tGyro-XYZ depicted in Figure 1 and 2 are the raw data from the accelerometer and gyroscope.

Image showing the raw acceleration data

Figure 1: Plot of raw acceleration data (50Hz) within 128 readings (2.56 sec). Andreas Lien, CC0

Image showing the raw gyroscope data

Figure 2: Plot of raw gyroscope data (50Hz) within 128 readings (2.56 sec). Andreas Lien, CC0

To reduce noise, it where filtered with a median filter, as shown in Figures 3 and 4, and a 3rd order low pass Butterworth filter with a 20 Hz corner frequency, like shown in Figures 5 and 6. In both acceleration and gyroscope data, a median filter with a filter length of 5 was employed.

Image showing the median acceleration data

Figure 3: Plot acceleration data every 128 readings (2.56 sec) with median filter. Andreas Lien, CC0

Image showing the median gyroscope data

Figure 4: Plot of raw gyroscope data (50Hz) every 128 readings (2.56 sec) with median filter. Andreas Lien, CC0

Image showing the median butterworth acceleration data

Figure 5: Plot acceleration data within 128 readings (2.56 sec) with median filter and Butterworth filter. Andreas Lien, CC0

Image showing the median butterworth gyroscope data

Figure 6: Plot gyroscope data within 128 readings (2.56 sec) with median filter and Butterworth filter. Andreas Lien, CC0

Another low pass Butterworth filter with a corner frequency of 0.3 Hz was used to split the acceleration signal into body and gravity acceleration signals (tBodyAcc-XYZ and tGravityAcc-XYZ), like shown in Figures 8 and 7. And separating acceleration signal into body and gravity acceleration signals by each window, like shown in Figure 9.

Image showing the gravity acceleration data

Figure 7: Plot gravity acceleration data within 128 readings (2.56 sec) with median filter and Butterworth filter. Andreas Lien, CC0

Image showing the body acceleration data

Figure 8: Plot body acceleration data within 128 readings (2.56 sec) with median filter and Butterworth filter. Andreas Lien, CC0

Image showing the gravity acceleration data

Figure 9: Plot gravity acceleration window data within 128 readings (2.56 sec) with median filter and Butterworth filter. Andreas Lien, CC0

To acquire Jerk signals, the body linear acceleration and angular velocity were calculated in time (tBodyAccJerk-XYZ and tBodyGyroJerk-XYZ), as shown in Figure 10 and 11.

Image showing the acceleration Jerk signals

Figure 10: Plot acceleration Jerk signals. Andreas Lien, CC0

Image showing the angular velocity Jerk signals

Figure 11: Plot angular velocity Jerk signals. Andreas Lien, CC0

Lastly a Fast Fourier Transform (FFT) was applied to some of these signals producing fBodyAcc-XYZ, fBodyAccJerk-XYZ, fBodyGyro-XYZ, fBodyAccJerkMag, fBodyGyroMag, fBodyGyroJerkMag, as shown in Figure 12. The ‘f’ is to indicate frequency domain signals.

Image showing the acceleration with hamming

Figure 12: Plot acceleration with hamming. Andreas Lien, CC0

See DiFronzo/LSTM-for-Human-Activity-Recognition-classification for access to the repository with all the data.