Processing

Please wait...

Settings

Settings

Goto Application

1. WO2021007100 - CROSS-MODAL SENSOR DATA ALIGNMENT

Publication Number WO/2021/007100
Publication Date 14.01.2021
International Application No. PCT/US2020/040670
International Filing Date 02.07.2020
IPC
G06T 7/30 2017.01
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
7Image analysis
30Determination of transform parameters for the alignment of images, i.e. image registration
G06T 17/05 2011.01
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
173D modelling for computer graphics
05Geographic models
G06N 3/08 2006.01
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
08Learning methods
G06N 3/04 2006.01
GPHYSICS
06COMPUTING; CALCULATING OR COUNTING
NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
3Computer systems based on biological models
02using neural network models
04Architecture, e.g. interconnection topology
CPC
G06K 9/00201
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
9Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
00201Recognising three-dimensional objects, e.g. using range or tactile information
G06K 9/00791
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
9Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
00624Recognising scenes, i.e. recognition of a whole field of perception; recognising scene-specific objects
00791Recognising scenes perceived from the perspective of a land vehicle, e.g. recognising lanes, obstacles or traffic signs on road scenes
G06K 9/6251
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
9Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
62Methods or arrangements for recognition using electronic means
6217Design or setup of recognition systems and techniques; Extraction of features in feature space; Clustering techniques; Blind source separation
6232Extracting features by transforming the feature space, e.g. multidimensional scaling; Mappings, e.g. subspace methods
6251based on a criterion of topology preservation, e.g. multidimensional scaling, self-organising maps
G06K 9/6271
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
9Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
62Methods or arrangements for recognition using electronic means
6267Classification techniques
6268relating to the classification paradigm, e.g. parametric or non-parametric approaches
627based on distances between the pattern to be recognised and training or reference patterns
6271based on distances to prototypes
G06K 9/6289
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
9Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
62Methods or arrangements for recognition using electronic means
6288Fusion techniques, i.e. combining data from various sources, e.g. sensor fusion
6289of input or preprocessed data
G06T 2207/10028
GPHYSICS
06COMPUTING; CALCULATING; COUNTING
TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
2207Indexing scheme for image analysis or image enhancement
10Image acquisition modality
10028Range image; Depth image; 3D point clouds
Applicants
  • WAYMO LLC [US]/[US]
Inventors
  • BRALEY, Colin Andrew
  • IVANCHENKO, Volodymyr
  • ZHANG, Yu
Agents
  • TREILHARD, John
  • PORTNOV, Michael
Priority Data
16/509,15211.07.2019US
Publication Language English (EN)
Filing Language English (EN)
Designated States
Title
(EN) CROSS-MODAL SENSOR DATA ALIGNMENT
(FR) ALIGNEMENT DE DONNÉES DE CAPTEUR INTERMODAL
Abstract
(EN)
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining an alignment between cross-modal sensor data. In one aspect, a method comprises: obtaining (i) an image that characterizes a visual appearance of an environment, and (ii) a point cloud comprising a collection of data points that characterizes a three-dimensional geometry of the environment; processing each of a plurality of regions of the image using a visual embedding neural network to generate a respective embedding of each of the image regions; processing each of a plurality of regions of the point cloud using a shape embedding neural network to generate a respective embedding of each of the point cloud regions; and identifying a plurality of region pairs using the embeddings of the image regions and the embeddings of the point cloud regions.
(FR)
L'invention concerne des procédés, des systèmes et un appareil, comprenant des programmes informatiques encodés sur un support de stockage informatique, permettant de déterminer un alignement entre des données de capteur intermodal. Selon un aspect, un procédé consiste à : obtenir (i) une image qui caractérise un aspect visuel d'un environnement et (ii) un nuage de points comprenant une collection de points de données qui caractérise une géométrie tridimensionnelle de l'environnement; traiter chaque région parmi une pluralité de régions de l'image à l'aide d'un réseau neuronal d'incorporation visuelle pour générer une incorporation respective de chacune des régions d'image; traiter chaque région parmi une pluralité de régions du nuage de points à l'aide d'un réseau neuronal à incorporation de forme pour générer une incorporation respective de chacune des régions de nuage de points; et identifier une pluralité de paires de régions à l'aide des incorporations des régions d'image et des incorporations des régions de nuage de points.
Also published as
Latest bibliographic data on file with the International Bureau