Diffusion-enabled 3D Human Pose Tracking, Data Augmentation, Completion, and Acceleration
Shiwen Mao, Auburn University, United States
Available Soon
Joel Rodrigues, College of Computer Science and Technology, China University of Petroleum (East China), Qingdao 266555, China; Instituto de Telecomunicações, Portugal
Diffusion-enabled 3D Human Pose Tracking, Data Augmentation, Completion, and Acceleration
Shiwen Mao
Auburn University
United States
Brief Bio
Shiwen Mao is a Professor and Earle C. Williams Eminent Scholar and Director of the Wireless Engineering Research and Education Center at Auburn University. Dr. Mao's research interest includes wireless networks, multimedia communications, RF sensing and IoT, smart health, and smart grid. He is the editor-in-chief of IEEE Transactions on Cognitive Communications and Networking, a member-at-large on the Board of Governors of IEEE Communications Society, and Vice President of Technical Activities of IEEE Council on Radio Frequency Identification (CRFID). He is a co-recipient of several technical and service awards from the IEEE. He is a Fellow of the IEEE.
Abstract
In recent years, 3D human activity recognition and tracking has become an important topic in human-computer interaction. To preserve the privacy of users, there is considerable interest in techniques without using a video camera. In this talk, we first present RFID-Pose, a vision-assisted 3D human pose estimation system based on deep learning (DL). The performance of DL models depends on the availability of sufficient high-quality radio frequency (RF) data, which is more difficult and expensive to collect than other types of data. To overcome this obstacle, in the second part of this talk, we present generative AI approaches to generate labeled synthetic RF data for multiple wireless sensing platforms, such as WiFi, RFID, and mmWave radar, including a conditional Recurrent Generative Adversarial Network (R-GAN) approach and diffusion/latent diffusion based approaches. Next, we propose a novel framework that leverages latent diffusion transformers to synthesize high quality RF data, as well as a latent diffusion transformer with cross-attention conditioning to accurately infer missing joints in skeletal poses, completing full 25-joint configurations from partial (i.e., 12-joint) inputs utilizing received RF sensory data. Finally, we present our recent work TF-Diff, a novel training-free diffusion framework for cross-domain radio frequency (RF)-based human activity recognition (HAR) system, which enables effective adaptation with minimal target-domain data.
Available Soon
Joel Rodrigues
College of Computer Science and Technology, China University of Petroleum (East China), Qingdao 266555, China; Instituto de Telecomunicações
Portugal
![]()
Brief Bio
Joel J. P. C. Rodrigues [S’01, M’06, SM’06, F’20] is a professor at the Federal University of Piauí (UFPI), Brazil; and senior researcher at the Instituto de Telecomunicações, Portugal. He received the Academic Title of Aggregated Professor in informatics engineering from UBI, the Habilitation in computer science and engineering from the University of Haute Alsace, France, a PhD degree in informatics engineering and an MSc degree from the UBI, and a five-year BSc degree (licentiate) in informatics engineering from the University of Coimbra, Portugal. His main research interests include IoT and sensor networks, e-health technologies vehicular communications, and mobile and ubiquitous computing.
Prof. Rodrigues is the leader of the Next Generation Networks and Applications research group (CNPq), IEEE Distinguished Lecturer [2018-2021], Director for Conference Development - IEEE ComSoc Board of Governors [2018-2019], Technical Activities Committee Chair of the IEEE ComSoc Latin America Region Board [2018-2019], the President of the scientific council at ParkUrbis – Covilhã Science and Technology Park, a Past-Chair of the IEEE ComSoc Technical Committee on eHealth, a Past-chair of the IEEE ComSoc Technical Committee on Communications Software, Steering Committee member of the IEEE Life Sciences Technical Community and Publications co-Chair, and Member Representative of the IEEE Communications Society on the IEEE Biometrics Council.
He is the editor-in-chief of the International Journal on E-Health and Medical Communications and editorial board member of several high-reputed journals (IEEE Network, IEEE Wireless Communications, IEEE IoT Journal, IEEE Open Journal of the Communications Society, etc.). He has been general chair and TPC Chair of many international conferences, including IEEE ICC, IEEE GLOBECOM, IEEE HEALTHCOM, and IEEE LATINCOM. He has authored or coauthored over 850 papers in refereed international journals and conferences, 3 books, 2 patents, and 1 ITU-T Recommendation. He had been awarded several Outstanding Leadership and Outstanding Service Awards by IEEE Communications Society and several best papers awards. Prof. Rodrigues is a licensed professional engineer (as senior member), member of the Internet Society, a senior member of ACM, and an IEEE Fellow.