Content Gazette - October 2020 - 28

Contents

Volume 37 | Number 5 | September 2020

SPECIAL SECTION

109

NONCONVEX OPTIMIZATION
FOR SIGNAL PROCESSING
AND MACHINE LEARNING
15

FROM THE GUEST EDITORS

18

UNDERSTANDING NOTIONS OF
STATIONARITY IN NONSMOOTH
OPTIMIZATION

GEOMETRY, MANIFOLDS,
AND NONCONVEX
OPTIMIZATION
Jonathan H. Manton

120

SUBMODULARITY
IN ACTION

Ehsan Tohidi, Rouhollah Amiri,
Mario Coutino, David Gesbert,
Geert Leus, and Amin Karbasi

Anthony Man-Cho So,
Prateek Jain, Wing-Kin Ma,
and Gesualdo Scutari

Jiajin Li, Anthony Man-Cho So,
and Wing-Kin Ma

32

ADAPTIVE STOCHASTIC
OPTIMIZATION
Frank E. Curtis and
Katya Scheinberg

43

A PRIMER ON ZEROTH-ORDER
OPTIMIZATION IN SIGNAL
PROCESSING AND MACHINE
LEARNING

Sijia Liu, Pin-Yu Chen,
Bhavya Kailkhura, Gaoyuan Zhang,
Alfred O. Hero III,
and Pramod K. Varshney

COLUMNS

ON THE COVER

6

This special issue introduces the essential elements of
nonconvex optimization to the broader signal processing
and machine learning communities, provides insights
into how structures of the nonconvex formulations of
various practical problems can be exploited in algorithm
design, showcases some notable successes in this line
of study, and identifies important research issues that
are motivated by existing or emerging applications.

NONCONVEX MIN-MAX OPTIMIZATION

67

NONCONVEX STRUCTURED
PHASE RETRIEVAL

Meisam Razaviyayn, Tianjian Huang,
Songtao Lu, Maher Nouiehed,
Maziar Sanjabi, and Mingyi Hong

New Society Officers, Editors-in-Chief
Elected for 2020
Election of Regional Directors-at-Large
and Members-at-Large
Ali H. Sayed

8

Reader's Choice

11

Special Reports

COVER IMAGE: ©ISTOCKPHOTO.COM/GAIHONG DONG

55

Society News

Top Downloads in IEEE Xplore

Three New Imaging Technologies
That Are Worth a Look
John Edwards

Namrata Vaswani

78

COMPUTING LARGE-SCALE MATRIX
AND TENSOR DECOMPOSITION WITH
STRUCTURED FACTORS
Xiao Fu, Nico Vervliet, Lieven De
Lathauwer, Kejun Huang,
and Nicolas Gillis

95
PG. 11

Spain
2%

Israel
2%

Sweden
2%

Other
15%

United States
24%

Switzerland
2%
Canada
2%
South Korea
2%
Taiwan
3%

THE GLOBAL LANDSCAPE OF
NEURAL NETWORKS

Ruoyu Sun, Dawei Li, Shiyu Liang,
Tian Ding, and Rayadurgam Srikant

India
4%
United Kingdom
4%

China
21%
Germany
5%
Japan
6%

France
6%

PG. 146

IEEE SIGNAL PROCESSING MAGAZINE (ISSN 1053-5888) (ISPREG) is published bimonthly by the Institute of Electrical and Electronics Engineers, Inc., 3 Park Avenue, 17th Floor, New York,
NY 10016-5997 USA (+1 212 419 7900). Responsibility for the contents rests upon the authors and not the IEEE, the Society, or its members. Annual member subscriptions included in Society fee.
Nonmember subscriptions available upon request. Individual copies: IEEE Members US$20.00 (first copy only), nonmembers US$246 per copy. Copyright and Reprint Permissions: Abstracting
is permitted with credit to the source. Libraries are permitted to photocopy beyond the limits of U.S. Copyright Law for private use of patrons: 1) those post-1977 articles that carry a code at the bottom of the first page, provided the per-copy fee is paid through the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923 USA; 2) pre-1978 articles without fee. Instructors are
permitted to photocopy isolated articles for noncommercial classroom use without fee. For all other copying, reprint, or republication permission, write to IEEE Service Center, 445 Hoes Lane,
Piscataway, NJ 08854 USA. Copyright © 2020 by the Institute of Electrical and Electronics Engineers, Inc. All rights reserved. Periodicals postage paid at New York, NY, and at additional mailing offices.
Postmaster: Send address changes to IEEE Signal Processing Magazine, IEEE, 445 Hoes Lane, Piscataway, NJ 08854 USA. Canadian GST #125634188
Printed in the U.S.A.

Digital Object Identifier 10.1109/MSP.2020.3006644

www.signalprocessingsociety.org 28 OCTOBER 2020
IEEE
2020| |
IEEESIGNAL
SIGNALPROCESSING
PROCESSINGMAGAZINE
MAGAZINE| |September
March 2020

1


https://ieeexplore.ieee.org/document/9194028 https://ieeexplore.ieee.org/document/9186137 https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9186141 https://ieeexplore.ieee.org/document/9186389 https://ieeexplore.ieee.org/document/9186135 https://ieeexplore.ieee.org/document/9194022 https://ieeexplore.ieee.org/document/9186148 https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9186143 http://www.ISTOCKPHOTO.COM/GAIHONG https://ieeexplore.ieee.org/document/9186144 https://ieeexplore.ieee.org/document/9186152 https://ieeexplore.ieee.org/document/9186151 https://ieeexplore.ieee.org/document/9194351 https://ieeexplore.ieee.org/document/9194023 https://ieeexplore.ieee.org/document/9186152 http://www.signalprocessingsociety.org

Content Gazette - October 2020

Table of Contents for the Digital Edition of Content Gazette - October 2020

Content Gazette - October 2020 - Cover1
Content Gazette - October 2020 - Cover2
Content Gazette - October 2020 - 1
Content Gazette - October 2020 - 2
Content Gazette - October 2020 - 3
Content Gazette - October 2020 - 4
Content Gazette - October 2020 - 5
Content Gazette - October 2020 - 6
Content Gazette - October 2020 - 7
Content Gazette - October 2020 - 8
Content Gazette - October 2020 - 9
Content Gazette - October 2020 - 10
Content Gazette - October 2020 - 11
Content Gazette - October 2020 - 12
Content Gazette - October 2020 - 13
Content Gazette - October 2020 - 14
Content Gazette - October 2020 - 15
Content Gazette - October 2020 - 16
Content Gazette - October 2020 - 17
Content Gazette - October 2020 - 18
Content Gazette - October 2020 - 19
Content Gazette - October 2020 - 20
Content Gazette - October 2020 - 21
Content Gazette - October 2020 - 22
Content Gazette - October 2020 - 23
Content Gazette - October 2020 - 24
Content Gazette - October 2020 - 25
Content Gazette - October 2020 - 26
Content Gazette - October 2020 - 27
Content Gazette - October 2020 - 28
Content Gazette - October 2020 - 29
Content Gazette - October 2020 - 30
Content Gazette - October 2020 - 31
Content Gazette - October 2020 - 32
Content Gazette - October 2020 - 33
Content Gazette - October 2020 - 34
Content Gazette - October 2020 - 35
Content Gazette - October 2020 - 36
Content Gazette - October 2020 - 37
Content Gazette - October 2020 - 38
Content Gazette - October 2020 - 39
Content Gazette - October 2020 - 40
Content Gazette - October 2020 - Cover3
Content Gazette - October 2020 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202312
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202311
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202310
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202309
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202308
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202307
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202306
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202305
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202304
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202303
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202302
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202301
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202212
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202211
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202210
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202209
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202208
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202207
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202206
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202205
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202204
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202203
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202202
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202201
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202112
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202111
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202110
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202109
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202108
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202107
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202106
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202105
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202104
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202103
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202102
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202101
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202012
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202011
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202010
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202009
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202008
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202007
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202006
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202005
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202004
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202003
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202002
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_202001
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_201912
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_201911
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_201910
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_201909
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_201908
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_201907
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_201906
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_201905
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_201904
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_201903
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_201902
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_201901
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_201812
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_201811
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_201810
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_201809
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_201808
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_201807
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_201806
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_201805
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_201804
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_201803
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_201802
https://www.nxtbook.com/nxtbooks/ieee/contentgazette_201801
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_1217
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_1117
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_1017
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0917
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0817
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0717
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0617
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0517
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0417
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0317
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0217
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0117
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_1216
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_1116
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_1016
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0916
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0816
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0716
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0616
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0516
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0416
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0316
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0216
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0116
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_1215
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_1115
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_1015
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0915
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0815
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0715
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0615
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0515
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0415
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0315
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0215
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0115
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_1214
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_1114
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_1014
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0914
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0814
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0714
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0614
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0514
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0414
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0314
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0214
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0114
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_1213
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_1113
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_1013
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0913
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0813
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0713
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0613
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0513
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0413
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0313
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0213
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0113
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_1212
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_1112
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_1012
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0912
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0812
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0712
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0612
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0512
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0412
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0312
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0212
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0112
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_1211
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_1111
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_1011
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0911
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0811
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0711
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0611
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0511
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0411
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0311
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0211
https://www.nxtbook.com/nxtbooks/ieee/content_gazette_0111
https://www.nxtbookmedia.com