Volume 12, Number 22, December 2022

Visually Similar Products Retrieval for Shopsy

  Authors

Prajit Nadkarni and Narendra Varma Dasararaju, Flipkart Internet Pvt. Ltd, India

  Abstract

Visual search is of great assistance in reseller commerce, especially for non-tech savvy users with affinity towards regional languages. Product attributes available in e-commerce have potential for building better visual search systems [2, 20, 29]. We design a visual search system for reseller commerce using a multi-task learning approach and address challenges like image compression, cropping, etc, faced in reseller commerce. Our model consists of three tasks: attribute classification, triplet ranking and variational autoencoder (VAE). We introduce an offline triplet mining technique which utilizes information from multiple attributes to capture relative order within data. This technique displays better performance compared to traditional triplet mining [27] baseline. We compare and report incremental gain achieved by our unified multi-task model over each individual task separately. The efsfectiveness of our method is demonstrated using in-house dataset of images from the Lifestyle business-unit of Flipkart. To efficiently retrieve images in production, we use Approximate Nearest Neighbor (ANN) index.

  Keywords

Content based image retrieval, Visual search, Multi-task Learning, Triplet loss, Variational autoencoder.