Towards Deep Learning Based Robot Automatic Choreography System

Ruiqi Wu; Wenyao Peng; Changle  Zhou; Fei Chao; Longzhi Yang; Chih-Min Lin; Changjing Shang

doi:10.1007/978-3-030-27538-9

Towards Deep Learning Based Robot Automatic Choreography System

Overview

Standard

Towards Deep Learning Based Robot Automatic Choreography System. / Wu, Ruiqi; Peng, Wenyao; Zhou, Changle ; Chao, Fei; Yang, Longzhi; Lin, Chih-Min; Shang, Changjing.

Intelligent Robotics and Applications: 12th International Conference, ICIRA 2019, Shenyang, China, August 8–11, 2019, Proceedings, Part IV. ed. / Haibin Yu; Jinguo Liu; Lianqing Liu; Zhaojie Ju; Yuwang Liu; Dalin Zhou. Springer Nature, 2019. p. 629-640 (Lecture Notes in Computer Science; Vol. 11743).

Research output: Chapter in Book/Report/Conference proceeding › Conference Proceeding (Non-Journal item)

Harvard

Wu, R, Peng, W, Zhou, C, Chao, F, Yang, L, Lin, C-M & Shang, C 2019, Towards Deep Learning Based Robot Automatic Choreography System. in H Yu, J Liu, L Liu, Z Ju, Y Liu & D Zhou (eds), Intelligent Robotics and Applications: 12th International Conference, ICIRA 2019, Shenyang, China, August 8–11, 2019, Proceedings, Part IV. Lecture Notes in Computer Science, vol. 11743, Springer Nature, pp. 629-640, 12th International Conference on Intelligent Robotics and Applications, Shenyang, China, 08 Aug 2019. https://doi.org/10.1007/978-3-030-27538-9

APA

Wu, R., Peng, W., Zhou, C., Chao, F., Yang, L., Lin, C-M., & Shang, C. (2019). Towards Deep Learning Based Robot Automatic Choreography System. In H. Yu, J. Liu, L. Liu, Z. Ju, Y. Liu, & D. Zhou (Eds.), Intelligent Robotics and Applications: 12th International Conference, ICIRA 2019, Shenyang, China, August 8–11, 2019, Proceedings, Part IV (pp. 629-640). (Lecture Notes in Computer Science; Vol. 11743). Springer Nature. https://doi.org/10.1007/978-3-030-27538-9

Vancouver

Wu R, Peng W, Zhou C, Chao F, Yang L, Lin C-M et al. Towards Deep Learning Based Robot Automatic Choreography System. In Yu H, Liu J, Liu L, Ju Z, Liu Y, Zhou D, editors, Intelligent Robotics and Applications: 12th International Conference, ICIRA 2019, Shenyang, China, August 8–11, 2019, Proceedings, Part IV. Springer Nature. 2019. p. 629-640. (Lecture Notes in Computer Science). https://doi.org/10.1007/978-3-030-27538-9

Author

Wu, Ruiqi ; Peng, Wenyao ; Zhou, Changle ; Chao, Fei ; Yang, Longzhi ; Lin, Chih-Min ; Shang, Changjing. / Towards Deep Learning Based Robot Automatic Choreography System. Intelligent Robotics and Applications: 12th International Conference, ICIRA 2019, Shenyang, China, August 8–11, 2019, Proceedings, Part IV. editor / Haibin Yu ; Jinguo Liu ; Lianqing Liu ; Zhaojie Ju ; Yuwang Liu ; Dalin Zhou. Springer Nature, 2019. pp. 629-640 (Lecture Notes in Computer Science).

Bibtex - Download

@inproceedings{9d9fed8b3bd7435b96c9246a8317a225,

title = "Towards Deep Learning Based Robot Automatic Choreography System",

abstract = "It is a challenge task to enable a robot to dance according to different types of music. However, two problems have not been well resolved yet: (1) how to assign a dance to a certain type of music, and (2) how to ensure a dancing robot to keep in balance. To tackle these challenges, a robot automatic choreography system based on the deep learning technology is introduced in this paper. First, two deep learning neural network models are built to convert local and global features of music to corresponding features of dance, respectively. Then, an action graph is built based on the collected dance segments; the main function of the action graph is to generate a complete dance sequence based on the dance features generated by the two deep learning models. Finally, the generated dance sequence is performed by a humanoid robot. The experimental results shows that, according to the input music, the proposed model can successfully generate dance sequences that match the input music; also, the robot can maintain its balance while it is dancing. In addition, compared with the dance sequences in the training dataset, the dance sequences generated by the model has reached the level of artificial choreography in both diversity and innovation. Therefore, this method provides a promising solution for robotic choreography automation and design assistance",

keywords = "artificial intelligence, fuzzy sets, genetic algorithms, Human-Computer Interaction (HCI), image processing, image reconstruction, image segmentation, learning algorithms, manipulators, mobile robots, motion control, motion planning, neural networks, path planning, robotics, robots, sensors, signal processing, software engineering, wireless telecommunication systems",

author = "Ruiqi Wu and Wenyao Peng and Changle Zhou and Fei Chao and Longzhi Yang and Chih-Min Lin and Changjing Shang",

year = "2019",

month = aug,

day = "3",

doi = "10.1007/978-3-030-27538-9",

language = "English",

isbn = "978-3-030-27537-2",

series = "Lecture Notes in Computer Science",

publisher = "Springer Nature",

pages = "629--640",

editor = "Haibin Yu and Liu, {Jinguo } and Lianqing Liu and Zhaojie Ju and Yuwang Liu and Dalin Zhou",

booktitle = "Intelligent Robotics and Applications",

address = "Switzerland",

note = "12th International Conference on Intelligent Robotics and Applications, ICIRA ; Conference date: 08-08-2019 Through 11-08-2019",

}

RIS (suitable for import to EndNote) - Download

TY - GEN

T1 - Towards Deep Learning Based Robot Automatic Choreography System

AU - Wu, Ruiqi

AU - Peng, Wenyao

AU - Zhou, Changle

AU - Chao, Fei

AU - Yang, Longzhi

AU - Lin, Chih-Min

AU - Shang, Changjing

PY - 2019/8/3

Y1 - 2019/8/3

N2 - It is a challenge task to enable a robot to dance according to different types of music. However, two problems have not been well resolved yet: (1) how to assign a dance to a certain type of music, and (2) how to ensure a dancing robot to keep in balance. To tackle these challenges, a robot automatic choreography system based on the deep learning technology is introduced in this paper. First, two deep learning neural network models are built to convert local and global features of music to corresponding features of dance, respectively. Then, an action graph is built based on the collected dance segments; the main function of the action graph is to generate a complete dance sequence based on the dance features generated by the two deep learning models. Finally, the generated dance sequence is performed by a humanoid robot. The experimental results shows that, according to the input music, the proposed model can successfully generate dance sequences that match the input music; also, the robot can maintain its balance while it is dancing. In addition, compared with the dance sequences in the training dataset, the dance sequences generated by the model has reached the level of artificial choreography in both diversity and innovation. Therefore, this method provides a promising solution for robotic choreography automation and design assistance

AB - It is a challenge task to enable a robot to dance according to different types of music. However, two problems have not been well resolved yet: (1) how to assign a dance to a certain type of music, and (2) how to ensure a dancing robot to keep in balance. To tackle these challenges, a robot automatic choreography system based on the deep learning technology is introduced in this paper. First, two deep learning neural network models are built to convert local and global features of music to corresponding features of dance, respectively. Then, an action graph is built based on the collected dance segments; the main function of the action graph is to generate a complete dance sequence based on the dance features generated by the two deep learning models. Finally, the generated dance sequence is performed by a humanoid robot. The experimental results shows that, according to the input music, the proposed model can successfully generate dance sequences that match the input music; also, the robot can maintain its balance while it is dancing. In addition, compared with the dance sequences in the training dataset, the dance sequences generated by the model has reached the level of artificial choreography in both diversity and innovation. Therefore, this method provides a promising solution for robotic choreography automation and design assistance

KW - artificial intelligence

KW - fuzzy sets

KW - genetic algorithms

KW - Human-Computer Interaction (HCI)

KW - image processing

KW - image reconstruction

KW - image segmentation

KW - learning algorithms

KW - manipulators

KW - mobile robots

KW - motion control

KW - motion planning