Teaching myself how to juggle using VR

I’ve always wanted to be able to juggle, its one of those things that when other people do it, it looks really easy and you can’t work out why you can’t just do it immediately. I’m currently reading Jaron Lanier’s ‘Dawn of the New Everything’ (great book by the way), in the book, he briefly talks about an experiment conducted a while ago where people who couldn’t juggle were put into VR where a juggling experience was slowed down for them. After some time in the simulation, they were tested again in real life and their juggling performance had improved!

The details were sketchy, but the principle shocked me. Could it really be possible to learn how to juggle in VR and then be able to do it in real life? I had to try this myself.

The experiment

So how was this going to work? Well, in true research fashion we should always start with our aims and objectives. The aim being “To gauge whether a participant can improve their ability to juggle through the use of virtual reality.”

Now let’s break this down into our objectives:

1. Test the participant’s (me) ability to juggle prior to using VR.
2. Develop a VR application that slows the speed of juggling in intervals.
3. Immerse the participant in the application.
4. Intermittently test the subject in real life after some time immersed in the application to gauge whether their ability to juggle has improved.

If all goes according to plan, we should see some improvement to my performance!

To tick off the first objective, I tested myself in real time to gauge my ability to juggle. It doesn’t matter what units you use or how you measure it – I sucked. Could barely get past throwing one of them!

The application

Next step is the application. In my head, the app would work like this:

There would be a slider to the right to adjust the speed at which the balls were juggled at, and on the left there would be a reset button in case the balls were ever dropped or lost. I jumped into Unity to hash it out. If you have followed along with our previous tutorial you pretty much have a real time juggling simulator ready to go! Here’s a few things/additions to consider for making an app like this.

Lessons Learnt No.1: Changing gravity does NOT change the speed.

This might seem pretty obvious (and it probably is) but it wasn’t to me at first. We see films in space with everyone moving slowly without gravity, therefore lowering the gravity should make this slower to juggle, right? Wrong. All this does is make your juggling balls fly off into the stratosphere!

To achieve a slowed down scene, I used the following script.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.UI;

public class Adjusters : MonoBehaviour
{
    private float sliderValue;
    public GameObject slider;
    public float originalVelocityThrow;

    private float timeOnStart;

    public GameObject ball1;
    public GameObject ball2;
    public GameObject ball3;

    public GameObject textValue;

    private Vector3 origin1;
    private Vector3 origin2;
    private Vector3 origin3;

    // Start is called before the first frame update
    void Start()
    {
        origin1 = ball1.transform.position;
        origin2 = ball2.transform.position;
        origin3 = ball3.transform.position;
        timeOnStart = Time.timeScale;
    }

    public void ResetBalls()
    {
        ball1.transform.position = origin1;
        ball2.transform.position = origin2;
        ball3.transform.position = origin3;
    }

    private void Update()
    {
        sliderValue = ((slider.GetComponent<Slider>().value) / 10f);
        textValue.GetComponent<Text>().text = sliderValue.ToString();
        //Time.timeScale = Time.timeScale * sliderValue;
        Time.timeScale = timeOnStart * sliderValue;
        Time.fixedDeltaTime = 0.02F * Time.timeScale;

        ball1.GetComponent<OffsetGrabInteractable>().throwVelocityScale = (originalVelocityThrow * sliderValue);
        ball2.GetComponent<OffsetGrabInteractable>().throwVelocityScale = (originalVelocityThrow * sliderValue);
        ball3.GetComponent<OffsetGrabInteractable>().throwVelocityScale = (originalVelocityThrow * sliderValue);
    }
}

This works by using a function called ‘timescale’ in Unity. This function is how fast the game runs at, for example setting this to 0 will pause your game. This script does the following:

• Gets the time at the start of the game (1.0)
• Multiplies the start time with the UI slider value (1.0 * 0.6 for example)
• Then sets the fixed delta time to 0.02 * the value above. (I won’t pretend that I know why the value 0.02 is used, it just is! If you know the reason please let me know…)

‘Fixed delta time’ provides smooth frames, without it, slowing down your game will just slow down the framerate, not the physics.

Something else you may have noticed in the code is that I also adjust the ‘throw velocity scale’ as the speed changes. This is because our hands are still working in real time even when the game is slowed down. If I were to throw the ball and the game was set to run at half speed, the velocity of me throwing the ball would be double, this code just accommodates for that by also multiplying by the slider value.

Lessons Learnt No.2: Use an offset grab, it makes catching so much easier.

For those that aren’t aware offset grab allows you to pick up an object at any point of the mesh and not just its anchor attachment. This comes in handy when it comes to catching the juggling balls as without it, you’ll probably miss every time.

(I won’t go into detail about offset grab scripts but there are many tutorials out there that go through it if you’re interested)

Lessons Learnt No.3: A VRUI interface is so much easier to use as well as set up.

My original sketch had buttons and sliders all over a desk, this is much harder to set up than a simple UI interface put in front of your face. Add a simple UI raycast object to your hand and voila, you’ve got a pointer. This is what we’re accustomed too as well, you wouldn’t have been able to open the application on your headset without pointing your hand at a menu!

(again I won’t go into detail about this but UI is something we’re interested in, perhaps a future tutorial?! Either way, there are also lots of tutorials out there that go through that.)

Test time!

The app is finally ready, so how am I going to test myself? While building the app, I got a good feel for how many ‘juggles’ was a good amount. So my challenge was to juggle consecutively for 20 cycles/reps/catches(?)… I’ll go with cycles. Once I juggled this, I would go up in speed. I broke the speed down into tenths, 0.1 being the lowest, and 1 being real time juggling.

Starting at the slowest speed 0.1, I juggled for 20 cycles and went up to 0.2, testing myself at points 0.3 and 0.6. This seemed fine up until 0.7 where my attempts grew exponentially, and I mean exponentially, just look at the data!

Doing some quick math, I didn’t fancy attempting another 1000 times, albeit, it may just take that many attempts, either way If I wanted to get their quickly, I needed a different approach. I noticed that attempts in the real world at points 0.3 and 0.6, although unsuccessful, helped improve my ability juggle in game.  I wondered if the same was true in game, i.e. attempt for a few mins to juggle at a higher speed level than I can do, then come back to what I was attempting.

This seemed to work quite well, and helped me juggle, although not consistently for 20sec, at least longer than I was previously.I kept this up for levels 0.7 and above, testing myself once I got to 20catches at that level. When it got to levels 0.7 above, I also noticed that getting height on the juggling balls was important, as well as looking upwards rather than in front of me.

Here’s a video of how I got on:

Results

So what do you think? Would you say that my attempts at 0.9 was good juggling, maybe, maybe not… but I think we can both agree that I’d definitely improved, which was the hypothesis of this research.

But what else did I found out? How about some more lessons learnt…

Lessons Learnt No.4: Slowing things down in VR helped make the problem more intuitive.

By breaking the juggling process down step by step, it helped me understand the principles of juggling (i.e. the cascade method and good posture) far quicker than if I was attempting in real life. It might even be the case that this is why some people find juggling so hard, because they haven’t had the opportunity to break it down and learn about good technique.

Lessons Learnt No.5: Attempting harder things made the previously challenging things easier.

As mentioned previously, if I was struggling to do something at a given speed level, I would attempt it a few levels up and then dial back the speed and it was often the case that I could do it for longer. Not sure if this is a ‘me thing’ or not but it makes sense, its how our muscles work when we work out in the gym, why would it be any different for muscle memory.

Lessons Learnt No.6: There’s some things VR can’t teach you.

In VR, I pick the third ball up off the table to throw it, in reality I’m already holding the ball. Juggling tutorial websites talk about the difficulty of ‘throwing that third ball’ and they’re right, juggling in VR just doesn’t give you that confidence, there’s a whole separate barrier there.

Lessons Learnt No.7: Use VR to do your research logging for you.

If I had more time, or wanted to conduct this experiment on others, I would put in some timers/attempt counters into VR, this can then log and I won’t have to rely on stopping and starting to log my attempts.

Conclusions

Lets scale this up for a moment, say I’m working on a construction site and I have to install a something complicated/not using traditional methods – by following the principles in this experiment, I’d be able to break the learning of that installation methods into ‘bitesize’ pieces in VR. This should then, at least, improve my abilities to do this in real life after some time spent in VR training for it.

One thing we always say at MAVRiC is that VR and AR is more than just visualization – it’s a tool. It can help us do things in the real world that are too expensive/time consuming/dangerous to train on. This is why the medical industry is well ahead of the game when it comes to immersive tech, for this exact reason.

In conclusion, I’d say I got pretty good at juggling! Although I’m not perfect at it, to be able to jump in and out of VR and actually see improvements is incredible. So I challenge you to make something similar and teach yourself to juggle using the same methods, like any researcher, I’d love to know if it was just a one off or whether this is repeatable stuff!

At least if VR doesn’t get big, I’ll have the circus to fall back on.