WEBVTT

00:00:00.000 --> 00:00:04.750 align:middle line:90%


00:00:04.750 --> 00:00:05.450 align:middle line:90%
Hello.

00:00:05.450 --> 00:00:08.290 align:middle line:84%
In this video, we will
look at mobile technologies

00:00:08.290 --> 00:00:12.550 align:middle line:84%
for studying music
and body motion.

00:00:12.550 --> 00:00:14.170 align:middle line:84%
In the laboratory
setting, we are

00:00:14.170 --> 00:00:19.090 align:middle line:84%
able to use highly advanced
infrared optical motion capture

00:00:19.090 --> 00:00:24.100 align:middle line:84%
systems to study very precisely
how people move to music.

00:00:24.100 --> 00:00:27.400 align:middle line:84%
But sometimes, we need
to go outside the lab

00:00:27.400 --> 00:00:30.670 align:middle line:84%
and study how people
move in the real world.

00:00:30.670 --> 00:00:35.170 align:middle line:84%
How people move while listening
to music on the metro,

00:00:35.170 --> 00:00:38.740 align:middle line:84%
on the bus, or in
the dance club.

00:00:38.740 --> 00:00:41.290 align:middle line:84%
For this reason, we
need to have some sort

00:00:41.290 --> 00:00:47.140 align:middle line:84%
of mobile technology for
studying music and body motion.

00:00:47.140 --> 00:00:49.150 align:middle line:84%
In front of me, I have
a couple of examples

00:00:49.150 --> 00:00:51.050 align:middle line:90%
of such technologies.

00:00:51.050 --> 00:00:52.900 align:middle line:84%
One of the advantages
of this system

00:00:52.900 --> 00:00:55.640 align:middle line:84%
is that it can be
very, very small,

00:00:55.640 --> 00:00:58.330 align:middle line:90%
such as this accelerometer.

00:00:58.330 --> 00:01:02.290 align:middle line:84%
Imagine studying body motion
in a dance club, where

00:01:02.290 --> 00:01:05.740 align:middle line:84%
you can put this in the pocket
of every dancer in the club.

00:01:05.740 --> 00:01:07.960 align:middle line:84%
And you get some data
on how they move,

00:01:07.960 --> 00:01:11.260 align:middle line:84%
and can see how this
relates to the music

00:01:11.260 --> 00:01:15.520 align:middle line:84%
and how they compare
to each other.

00:01:15.520 --> 00:01:19.870 align:middle line:84%
Accelerometers and gyroscopes,
and also magnetometers,

00:01:19.870 --> 00:01:25.430 align:middle line:84%
are inside basically every
mobile phone these days.

00:01:25.430 --> 00:01:30.730 align:middle line:84%
And so everybody, in practise,
has their own motion capture

00:01:30.730 --> 00:01:35.260 align:middle line:84%
technology that
they carry around.

00:01:35.260 --> 00:01:37.520 align:middle line:84%
More specialised
systems also exist,

00:01:37.520 --> 00:01:43.330 align:middle line:84%
such as this bracelet which
contains sensors that measure

00:01:43.330 --> 00:01:44.590 align:middle line:90%
muscle tension.

00:01:44.590 --> 00:01:48.070 align:middle line:84%
In addition to this
accelerometer and gyroscope

00:01:48.070 --> 00:01:53.260 align:middle line:84%
that lets you measure the
direction of the bracelet,

00:01:53.260 --> 00:01:58.750 align:middle line:84%
and also how fast
it accelerates.

00:01:58.750 --> 00:01:59.410 align:middle line:90%
Like this.

00:01:59.410 --> 00:02:02.500 align:middle line:84%
So it will tell you
which direction I point

00:02:02.500 --> 00:02:06.400 align:middle line:84%
and how I turn my hand,
because of the muscle sensors.

00:02:06.400 --> 00:02:10.669 align:middle line:90%


00:02:10.669 --> 00:02:15.560 align:middle line:84%
This suit is a motion capture
suit from Xsens Technologies.

00:02:15.560 --> 00:02:20.770 align:middle line:84%
It contains 17 of
these orange sensors.

00:02:20.770 --> 00:02:25.060 align:middle line:84%
And inside each of these
there is a magnetometer,

00:02:25.060 --> 00:02:29.080 align:middle line:84%
and a gyroscope,
and accelerometer,

00:02:29.080 --> 00:02:32.170 align:middle line:84%
which measures inertia,
and acceleration,

00:02:32.170 --> 00:02:34.570 align:middle line:90%
and the magnetic flux.

00:02:34.570 --> 00:02:38.140 align:middle line:90%
Or basically, a compass.

00:02:38.140 --> 00:02:43.330 align:middle line:84%
Combining 17 of these sensors
with a model of a human body,

00:02:43.330 --> 00:02:47.110 align:middle line:84%
saying that, for instance,
this sensor and this sensor

00:02:47.110 --> 00:02:49.840 align:middle line:84%
can move further
apart than this.

00:02:49.840 --> 00:02:54.760 align:middle line:84%
And this sensor and this sensor
can only move in this angle.

00:02:54.760 --> 00:02:58.480 align:middle line:84%
It's possible to get
an estimation of how

00:02:58.480 --> 00:03:04.400 align:middle line:84%
the pose of the
person is at any time.

00:03:04.400 --> 00:03:08.020 align:middle line:84%
And the result you get
is quite good, actually,

00:03:08.020 --> 00:03:10.940 align:middle line:84%
from thinking that
this type of technology

00:03:10.940 --> 00:03:13.630 align:middle line:90%
there are no cameras involved.

00:03:13.630 --> 00:03:17.710 align:middle line:84%
The system is portable,
with two senders like this.

00:03:17.710 --> 00:03:21.700 align:middle line:84%
And it's possible
to move quite far

00:03:21.700 --> 00:03:27.100 align:middle line:84%
away from the transmitter, which
is right next to our computer.

00:03:27.100 --> 00:03:30.970 align:middle line:84%
So looking at the screen
now, my hand is upright

00:03:30.970 --> 00:03:33.700 align:middle line:84%
and the avatar is
following my movements.

00:03:33.700 --> 00:03:36.355 align:middle line:84%
We tested it up to 50
metres without problems.

00:03:36.355 --> 00:03:39.550 align:middle line:90%


00:03:39.550 --> 00:03:42.790 align:middle line:84%
One small problem with this
system is that it's drifting.

00:03:42.790 --> 00:03:46.330 align:middle line:84%
So if you move around a lot
and go back to the same place,

00:03:46.330 --> 00:03:50.740 align:middle line:84%
you will possibly have a
result that diverges slightly.

00:03:50.740 --> 00:03:56.440 align:middle line:84%
And also, it's not able to
give you a millimetre position

00:03:56.440 --> 00:03:59.740 align:middle line:84%
like you'll get in
the optical systems.

00:03:59.740 --> 00:04:04.930 align:middle line:84%
Still, it's remarkably good
and absolutely a good way

00:04:04.930 --> 00:04:07.780 align:middle line:90%
of measuring body movements.

00:04:07.780 --> 00:04:10.300 align:middle line:84%
What you can see here is that
there's quite a lot of cables.

00:04:10.300 --> 00:04:15.340 align:middle line:84%
So using it in, for
instance, a stage performance

00:04:15.340 --> 00:04:18.339 align:middle line:84%
requires that you are
aware of your system,

00:04:18.339 --> 00:04:21.190 align:middle line:84%
and you will not be
able to move as freely

00:04:21.190 --> 00:04:23.290 align:middle line:90%
as without the system.

00:04:23.290 --> 00:04:25.660 align:middle line:84%
But we have used it in
several performances,

00:04:25.660 --> 00:04:30.040 align:middle line:84%
and you can use it as
a DJ to control DJ sets

00:04:30.040 --> 00:04:31.390 align:middle line:90%
or as a musical instrument.

00:04:31.390 --> 00:04:33.540 align:middle line:90%
[MUSIC PLAYS]

00:04:33.540 --> 00:04:52.000 align:middle line:90%