When we snap strings playing with a CMOS camera, the strings seems to vibrate in a wobbly slow motion pattern. Because a CMOS sensor scans one line of video in sequence, fast moving objects are distorted during the scanning sequence. The morphing and distorting are called a rolling shutter effect, which is considered to be an artistic photographic techniques like strip photography and slit-scan photography. However, the effect can only be seen on a camera finder or a PC screen, the guitar player and audience are quite unlikely to notice it by the naked-eye.

To cope with this limitation, we developed Wobble Strings, a system that allows for the rolling shutter effect to be observed by naked-eye in real time using spatially divided stroboscopic projection. The system can produce wobbly slow motion effect in real time by generating the animation of sweep lines using a projector in accordance with the pitch of strings. Furthermore, our system can also alter the color and texture of strings using a projection of the color and texture sweep lines. Thanks to our system, the guitar player can monitor the strings’ oscillation and, the audience can experience an artistic visual effect with the guitar sound.

The stroboscopic effect is a temporal aliasing with a spatially uniform flashing light, and it can create stop-motion and slow-motion effects. In addition, we opened up a novel spatially divided stroboscopic system that has feedback from musical instruments, and we also developed a method to create new visual effects like wobbly motion distortion and color and texture coating.

論文 Transaction Paper

Shogo Fukushima, Takeshi Naemura,“Wobble Strings: Spatially Divided Stroboscopic Effect for Augmenting Wobbly Motion of String Instruments”, Elsevier Journal on Entertainment Computing, 27 pages, 2016 (under review).

国際学会 International Conference with Peer Review

Shogo Fukushima, Takeshi Naemura, ” Wobble Strings: Spatially Divided Stroboscopic Effect for Augmenting Wobbly Motion of Stringed Instruments”, In ACM SIGGRAPH 2015 Talks (SIGGRAPH ’15), 2015.8.
Shogo Fukushima, Takeshi Naemura, ” Wobble Strings: Spatially Divided Stroboscopic Effect for Augmenting Wobbly Motion of Stringed Instruments”, In ACM SIGGRAPH 2015 Emerging Technologies (SIGGRAPH ’15), 2015.8.
Shogo Fukushima, Takeshi Naemura, ” Wobble Strings: Spatially Divided Stroboscopic Effect for Augmenting Wobbly Motion of Stringed Instruments”, In ACM SIGGRAPH 2015 Posters (SIGGRAPH ’15), 2015.8.

Playing the guitar and the bass guitar

Wave number control, color and texture coated effect, and shape deformation effects

Technical details