IEEE Robotics & Automation Magazine - December 2021 - 48

for the invariances. The integrated error bounds are
! 5mm for translations, ! 15° for roll and pitch, and ! 45°
for yaw rotations.
Pose Prediction From Deep Learning
PBTS control requires an estimate of the sensor pose
FPtS
()
in the contact frame of an edge or surface feature from tactile
measurements ()m t at time t. In related work [11], we
investigated how deep learning can be used to train accurate
" PoseNet " neural network models for estimating the
3D pose from tactile images. This involved a systematic
approach for selecting the best network hyperparameters
using Bayesian optimization applied to the validation loss,
rather than the more common approach of hand-tuning.
This work uses the methods in [11] to gather training/validation
data and train the neural network. We emphasize
that the data collection involved a transverse sliding motion
before recording each tactile image via an (unlabeled) random
displacement of the sensor. This motion is important
because contact-dependent shear affects measurements from
optical tactile sensors, such as the one used here [10], [25].
We refer to [11] for the details of the methods and report
the differences from that previous analysis. These were
●
●
larger training/validation data sets of 5,000/5,000 samples
over the same ranges as previously (Table S1 in " Training
Data, Control, and PoseNet Parameters " )
fewer iterations (100) for the hyperparameter optimization by
fixing values that were consistently optimized [11], such as
Training Data, Control, and PoseNet Parameters
The training data, control, and PoseNet parameters used in this work are given in Tables S1-S3.
Table S1. The training/validation data parameters.
Object Feature
3D surface
Samples
10,000
3D edge
10,000
Pose Range (Labeled)
00 51 15 15 0!!-(,
,[ ,];, ,)
!50 51 15 15 45!!!-(,
,[ ,];, ,)
Displacement Range (Unlabeled)
(, ,; ,,)55 0 555!! !!!
(, ,; ,,)55 0 555!! !!!
In each pose, the first three values are given in millimeters and the second three values in degrees.
Table S2. The pose-based servo control parameters.
Sensor Pose FPS Reference Pose FPR
Object
Feature
3D surface (, ,; ,, )ab 0
00 z
3D edge
(, ,; ,, )xz0 ab c
(, ,; ,, )
01 30 00(,
,; ,, )
01 30 00`11
`
Proportional
Gain
diag (KP)
,, ;, ,
2
1
2
1 1
,, ;, ,
2
1
2
1
2
1
2
1
2
1 1
j
2
1
j
Integral Gain diag (KI)
`00
,, ;, , 0j
10
3
`
10
3
,, ;, ,
In each pose, the first three values are given in millimeters and the second three values in degrees.
10
3
10
1
10
1
10
1
10
1
10
1
Integral Bounds B
(, ,; ,, )
! 00 515150
j (, ,; ,, )
! 50 5151545
Table S3. The PoseNet parameters.
Object
Feature
3D
surface
Number of
Convolutional
Layers
5
3D edge 5
Number
of Dense
Layers
1
1
Batch
Norm.
None
None
Number of
Convolutional
Kernels
256
256
Number
of Dense
Units
256
256
Batch
Size
16
16
Activation
Function
ELU
ReLU
Dropout
Coefficient
0.0047
0.07
L1-Reg.
Coefficient
0.0001
0.0001
L2-Reg.
Coefficient
0.0159
0.0019
The first six parameters were fixed (using values from [11]); the final four parameters were optimized. Norm.: normalization; Reg.: regularization;
ELU: exponential linear unit; ReLU: rectified linear unit.
48 * IEEE ROBOTICS & AUTOMATION MAGAZINE * DECEMBER 2021
IEEE Robotics & Automation Magazine - December 2021

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2021
