Juggling in Virtual Reality! Project Neutrino!

Please see my new site at www.benjaminoutram.com

I have brought this project back to life!  And now working on a virtual reality juggling training and gaming application!  Check it out.

www.benjaminoutram.com

I have a new homepage which covers current and past projects!
www.benjaminoutram.com

Building the block-diagram of a force-haptic interface for virtual reality

I am developing a force-haptic device for virtual reality.  This post follows on from the last post, which contained more description and pictures and I recommend you look at that first.

I gave a talk today at Keio Media Design, describing the system and current issues with it.  The slides are available here, or click below.

Haptic device force feedback control optimisation using PID

I am developing a force-haptic device that displays the force of virtual simulated juggling balls to a user's hands.  The system tracks the users hands using a Wii remote infrared camera, and the tracked positions of the hands control hand game-objects that interact with virtual juggling balls.  The force is calculated and fed into a feedback control system, the output of which controls a motor which is coupled to the user's hands via chords.  

Here is a block diagram to make it clearer.

The feedback control is particularly tricky.  A proportional-integral-differential (PID) negative feedback controls the motor.  In addition, the position of the hand also feeds into the system.

The motor can be seen attached to a 3D printed mount with pulley and string, mounted on a weight which is on top of some hacked kitchen scales.  The scales sensor output is amplified by an instrumental amplifier before being fed to an Arduino, and ultimately to the virtual reality (VR) simulation program.  The amplifier gain is controlled using a variable resistor, and uses capacitors to remove high-frequency noise.  The simulation calculates the required signal and outputs via the Arduino and motor controller to the motor. 

In the picture below, the Nintendo Wii remote (IR camera) and infrared illuminator can be seen balancing on an oscilloscope, which is used to look at the output of the force-sensor.

The user holds a (low weight) ball with retroreflector for tracking via the Wii remote and string attached for coupling to the haptic feedback via the motor.

Thanks to Sean Shao-An Tsai, a masters student at KMD, for designing and 3D printing the motor's mount.

At the moment there are several places that require improvement.  The feedback system's gain parameters need to be optimised.  Laggy components in the system must be minimised.  The main laggy components are currently, I think, the springyness of the string coupling the motor to the user's hands, the weight/force-sensor which acts like a mass-spring combination, and the Unity software and frame rate (which is limited by serial communications between Arduino and Unity).  I plan to mitigate by getting a stiffer string (suggestions welcome, please comment below), designing the weight out of the force-sensor system, and moving some of the feedback loop calculations into the Arduino itself, thereby increasing the rate at which the output to the motor is calculated, and allowing some software-level control of averaging and smoothing.

Any suggestions, etc, please comment below!

Negative feedback force-haptic device with hacked kitchen scales

Want to jump straight to the video... ? Click here.

I'm developing a system that presents virtual balls to a user, who can catch and throw those balls and feel their momentum and weight via a haptic interface that applied forces to the user's hands.

Previously, I was using an open-loop control, calculating the signal to apply to the motor of the haptic device based on the force on the hands simulated from the virtual balls, and the tracked positions of the users real hands.  However, I realised that this is an inadequate control system for a haptic device where the force applied to the user is the primary concern.

I realised I need a negative feedback system, where the force is measured and compared with the simulated force.  I initially tried deriving the force from the current drawn/generated at the motor, but this is not sensitive enough.  Therefore I decided to measure the force directly (thanks to Oliver Coles' suggestion) using some kitchen scales.

I bought Tanita KD-192-GR kitchen scales, rated to measure up to 2kg with sensitivity of 1g.  There is also a 0.1g mode.  Inside the scales, there is a sensor (a wheatstone bridge load-cell), which is wired to the scales' circuit board.  The signal direct from the sensor is too small to use directly with a micro-contoller like the Arduino, so it need amplification via an instrumental amplifier.  The amplifier is likely in the black blob IC that can be seen in the picture, which makes it impossible to get the amplified signal from the on-board hardware directly.  The signal to the LCD is a complicated multiplex-encoded signal that I wasn't about to try and decode, and besides it looked like there was a lot of lag on that signal.  Therefore I went to Tokyo's Akihabara and bought an instrumental amplifier LT1167 (datasheet) to amplify the signal from the load-cell myself.

In the photo below, the instrumental amplifier (the small black chip with a gain-control resistor) uses the Arduino's 5V and ground, and the output of the amplifier is wired into an analog-in pin on the Arduino. The green thing in the picture below is the motor controller, which is receiving a signal from the Arduino.

Bit of a mess, but you can see the motor with pulley and string on the scales in the foreground, the circuit, and an oscilloscope for checking stuff.

The motor that applies the force is mounted on a heavy weight which is balanced on the scale.  The string would be attached to the user's hands, and the force is measured and fed back to the control system.  The measured force is subtracted from the simulated force, and the difference multiplied by a gain factor and passed to the motor controller.  In this way, the motor always tries to maintain the force calculated in the simulation (see wiki about negative feedback).

https://www.youtube.com/watch?v=VVub5v4KwCI

Unfortunately, the system is far from stable at the moment and I think I am going to have to delve into negative feedback control system theory to understand how to make it work nicely.  If you have any advice, please send me a message!  Have a look at this video of how the system is now!

Sound Synesthesia Project

A little unrelated to the juggling project, which is doing well, I also am developing sound visualisation software, and will be speaking at an event in Shibuya, Tokyo, on the 31st of January 2015 [edit: has already passed!], so please come and check it out!

You can download a version of the sound visualisation software at my other blog:
http://bensoundcolour.blogspot.jp/

https://sites.google.com/site/benjaminoutram/

"Benjamin Outram is a physicist, artist, hacker and engineer, with a PhD in liquid crystal science from the University of Oxford. He is interested in the use of media technology to transform our experience. While currently working on virtual reality at Keio Media Design, he is also creating a unique sound synesthesia project, combining physics, the science of perception, and art. He will be sharing his insights into our senses and showcasing an interactive artwork, a demo of which will be made available to download."

Facebook event page:
https://www.facebook.com/events/424041554438699/

Haptic Virtual Juggling (More Information)

More information is available in this talk I gave recently at Keio Media Design, Tokyo:
It shows a little about the hardware and idea.

You can see the videos that don't play in the PDF above in the following links:

http://youtu.be/ZXwMSYPgREE

http://youtu.be/nrJUQRlT0NY

http://youtu.be/nbi2Qk_R--0

A bit more detailed description, including the maths, can be found in a report I have written on the project here.   Please email me if you have any idea or would like to speak to me about the project! I'm very keen to get feedback, ideas, and learn from you.