IEEE Robotics & Automation Magazine - September 2020 - 94

Figure 9(b) depicts the overview of this experiment, and we
can see that the car began to move when the traffic light turned
blue and the car could turn right at the crossing by 90°. The
sequence of the steering
wheel operation is dis-
played in Figure 9(c). The
By making use of
sequence is rotating the
steering wheel by both
flexibility, variablearms as much as possible,
releasing the left hand,
stiffness structure, and
returning the left hand
to the original position,
several sensors, we
releasing the right hand,
returning the right hand to
succeeded in the steering
the original position, and
rotating again. The transi-
wheel operation with
tion of the steering angle is
presented in Figure 9(d).
both arms and human
The robot could turn the
steering wheel by about
recognition in the
180° in 70 s. Figure 9(e)
shows the recognition
side mirror.
result of the traffic light.
The light is cropped by the
object detection module,
and the blue or red of the traffic light is recognized based on the
ratio of red and blue pixels. The module could recognize the
moment when the light turns from blue to red. At the same
time, as shown in Figure 9(f), the left ankle pitch joint moved to
release the brake pedal, and the car began to move. The prob-
lem of this experiment is the slowness of the steering wheel

t=0s

operation. Currently, turning at the crossing takes about two
minutes, and we must make the motions faster and smoother.
Limitations and Future Works
Pedal Operation
The pedal operation is one of the tasks with various remaining
issues. In this study, we assumed a flat and smooth road and
developed a method to achieve the target car velocity quickly
by representing the state equation between the car velocity and
joint angle of the ankle pitch. However, there are three issues.
First, in the actual driving environment, the road is not
smooth; the ground rises at a crossing, and the road is some-
times uphill or downhill. In those cases, the state equation
trained at a flat road is different from that of the actual envi-
ronment, and the robot cannot adjust the car velocity well. To
solve this problem, we need to conduct online learning or add
the image of road condition and IMU information in the
body to the initial task state s taskl. However, online learning
becomes difficult with additional network input, and an effi-
cient learning system with only a few data is desired.
Second, the robot adjusts only the acceleration pedal and
cannot adjust the brake pedal. When driving more slowly than
creep velocity, the robot must adjust the car velocity by stepping
on the brake pedal. Also, in this study, although not required at
the slow car velocity, the robot needs to acquire how to smooth-
ly step on the brake pedal as the car velocity becomes fast.
Third, the car velocity is currently obtained from the
COMS software, but it should be obtained using image
information, IMUs, and so forth. Since the car veloci-
ty obtained from visual odometry is too noisy for pedal

t = 21 s

t = 66 s

t = 110 s

Brake

Traffic Light: Blue

(b)

Yes
Acceleration
Steering Wheel Operation

Blue

Joint Angle (rad)

0.5
0
-0.5
-1
-1.5
-2
-2.5
-3

(c)

Detected

Angle (rad)

(a)

None
Red

60
Time (s)
(d)

100

120

10
Time (s)
(e)

0.5
0.4
0.3
0.2
0.1
0
-0.1
0

10
Time (s)
(f)

Figure 9. The steering wheel operation with recognition: the (a) experimental motion flow, (b) experimental appearance, (c)
sequence of steering wheel operation, (d) transition of the steering wheel angle, (e) traffic light recognition result, and (f) joint angle
of the left ankle pitch for brake pedal operation.

IEEE ROBOTICS & AUTOMATION MAGAZINE

SEPTEMBER 2020

IEEE Robotics & Automation Magazine - September 2020

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2020