Current Research

Smiles is a research lab in the Department of Electronic and Information Engineering at Xi'an Jiaotong University.

Current Research Interests

ocial Mobile Multimedia Mining Learning and Search

Photo Taken Place Estimation from Social Media

By leveraging the social user contributed photos in social community, we can carry out image taken place estimation by fast visual search.  We propose a fast global feature clustering and local feature refinement based image location estimation approach. Moreover, a fast invert file index is built for organizing the offline dataset. The online estimation for an image is less than 1 ms for both our GOLD and GOLDEN datasets.

说明: http://smiles.xjtu.edu.cn/images/knowledge%20mining2.jpgMobile Sensing Based Travelogue Generation, Travel Guide, and Localtion based Serverce Recommendation  

Smart phone is very popular. Users often utilize them to take photos in travel. When user shares the photos in their social media websites, such as Flickr, we can automatically map the photos to the map, and generate a travelogue for the travel. For the places of interest, the descriptions of each spot are mining from Wiki and Social media websites to carry out mobile based travel guide. By mining the metadata, we can mine users prefer and recommend them location based services.

说明: http://smiles.xjtu.edu.cn/images/rs.jpgPersonalized Recommendation Combining User Interest, Social Circle and Location

For social user, we recommend users favorite items, products and services by mining users interests (prefers) and social circles, moreover, the location information of users and services are also taken into account to recommend user personalized services.

说明: http://smiles.xjtu.edu.cn/images/image%20annotation.pngSocial Image Tagging and Content Understanding

For social images, recommend content relevant textual descriptions for the use shared photos is important. Our solutions to image tagging: 1) We model the tags by a graph, and the tag enrichment problem is converted to a graph-cut based optimization problem. 2) A similar compatible model based approach. 3) Tagging image using users’ own vocabulary by mining social user tagging behavior, and incorporating image taken time, location and visual information.


Past Research Interests

说明: http://smiles.xjtu.edu.cn/images/sports%20video.jpgSport Video Content Understanding

Our Sport video content understanding sytem consists of following steps: 1) video shot boundary detection, 2) adaptive event clip segmentation, 3) extract middle level semantics and temporal overall features of each event clip, 4) Learning based event modeling using enhanced HMM, and Hidden Conditional Random Field.

说明: http://smiles.xjtu.edu.cn/images/sports%20video.jpgCompressed Domain based Video Analysis  

We carry out fast video text detection, global motion estimation, shot boundary detection from the bit-streams of compressed domain. We extract DC coefficients for fade-in/out and Flashlight detection, AC coefficients for fast text detection, localization, and tracking. For the D1 format video, the Recall and Precision are all above 90%, and the computational costs is about 200 frames per second. These approaches are successfully utilized in the products of Huawei.

说明: http://smiles.xjtu.edu.cn/images/sports%20video.jpgCorrupted Video Stream Recovery  

During video transmission, when a package is lost or corrupted, this will make the constructed video quality well poor. To solve the problem, we proposed to use global/local motion based error concealment approach, and determining the concealment order by analyzing the spatial information of corrupted regions. The performance is better than that utilized in H.26L.




Our research is supported by:



NSFC No.61772407:Key studies on object based surveillance video retrieval method.(in Chinese: 基于对象的监控视频检索方法研究,负责人)

NSFC No.61732008:Key studies on heterogeneous media search for complex queries.(in Chinese: 面向复杂查询的异质媒体搜索,子课题负责人)

Microsoft Research Asia: Learning salient feature to enhance the mobile image retrieval,负责人







NSFC No.61171309:Key studies on mobile based product recommendation approach.(in Chinese: 基于移动互联网的图标广告推荐方法研究,2012.1~2015.12)

NSFC No.61171309:Key studies on mobile based product recommendation approach.(in Chinese: 基于移动互联网的图标广告推荐方法研究,2012.1~2015.12)

NSFC No.60903121: Key study on a unified scalable sport video content understanding approach. ( in Chinese:

MSRA: Mobile Sensing based travel guide and travelogue generation.( in Chinese:

Huawei: Fast video text detection.( in Chinese: