Kintense: A Robust, Accurate, Real-Time and Evolving System for Detecting Aggressive Actions from Streaming 3D Skeleton Data

Kintense is a robust, accurate, real-time, and evolving system for detecting aggressive actions such as hitting, kicking, pushing, and throwing from streaming 3D skeleton joint coordinates obtained from Kinect sensors. Kintense uses a combination of: (1) an array of supervised learners to recognize a predefined set of aggressive actions, (2) an unsupervised learner to discover new aggressive actions or refine existing actions, and (3) human feedback to reduce false alarms and to label potential aggressive actions. This paper describes the design and implementation of Kintense and provides empirical evidence that Kintense is 11% – 16% more accurate and 10% – 54% more robust to changes in distance, body orientation, speed, and person when compared to standard techniques such as dynamic time warping (DTW) and posture based gesture recognizers. We deploy Kintense in a multiperson household and demonstrate how it evolves to discover and learn unseen actions, and reduces false alarms via 4 – 13 times fewer user interactions than a typical system.

smarthomeCPS_Kintense

People:

  • Shahriar Nirjon (smn8z@Virginia.edu)
  • Chris Greenwood (cmg7t@Virginia.edu)
  • Carlos Torres (ct5ab@Virginia.edu)
  • Stefanie Zhou (xz5xm@Virginia.edu)
  • John A. Stankovic (stankovic@Virginia.edu)
  • Hee Jung Yoon (heejung8@dgist.ac.kr
  • Ho-Kyeong Ra (hk@dgist.ac.kr)
  • Can Basaran (cbasaran@dgist.ac.kr)
  • Taejoon Park (taejoon@hanyang.ac.kr)
  • Sang H. Son (son@dgist.ac.kr)

Reference:

  1. Nirjon, Shahriar, Greenwood, Chris, Torres, Carlos, Zhou, Stefanie, Stankovic, John A., Yoon, Hee Jung, Ra, Ho-Kyeong, Basaran, Can, Park, Taejoon, Son, Sang H., “Kintense: A robust, accurate, real-time and evolving system for detecting aggressive actions from streaming 3D skeleton data.”Pervasive Computing and Communications (PerCom), 2014 IEEE International Conference on. IEEE, 2014.
  2. Shahriar Nirjon, Chris Greenwood, Carlos Torres, Stefanie Zhou, John A. Stankovic, Hee Jung Yoon, Ho-Kyeong Ra, Can Basaran, Taejoon Park, Sang H. Son, “Kintense: a robust, accurate, real-time and evolving system for detecting aggressive actions from streaming 3D skeleton data,” Poster and Demo Session at ACM SenSys 2013, Nov. 2013.