Home->Research

Current Research


Smiles is a research lab in the Department of Electronic and Information Engineering at Xi'an Jiaotong University.


Current Research Interests

S
ocial Mobile Multimedia Mining Learning and Search

Photo Taken Place Estimation from Social Media

By leveraging the social user contributed photos in social community, we can carry out image taken place estimation by fast visual search.  We propose a fast global feature clustering and local feature refinement based image location estimation approach. Moreover, a fast invert file index is built for organizing the offline dataset. The online estimation for an image is less than 1 ms for both our GOLD and GOLDEN datasets.




说明: http://smiles.xjtu.edu.cn/images/knowledge%20mining2.jpgMobile Sensing Based Travelogue Generation, Travel Guide, and Localtion based Serverce Recommendation  

Smart phone is very popular. Users often utilize them to take photos in travel. When user shares the photos in their social media websites, such as Flickr, we can automatically map the photos to the map, and generate a travelogue for the travel. For the places of interest, the descriptions of each spot are mining from Wiki and Social media websites to carry out mobile based travel guide. By mining the metadata, we can mine users prefer and recommend them location based services.


说明: http://smiles.xjtu.edu.cn/images/rs.jpgPersonalized Recommendation Combining User Interest, Social Circle and Location

For social user, we recommend users favorite items, products and services by mining users interests (prefers) and social circles, moreover, the location information of users and services are also taken into account to recommend user personalized services.





说明: http://smiles.xjtu.edu.cn/images/image%20annotation.pngSocial Image Tagging and Content Understanding

For social images, recommend content relevant textual descriptions for the use shared photos is important. Our solutions to image tagging: 1) We model the tags by a graph, and the tag enrichment problem is converted to a graph-cut based optimization problem. 2) A similar compatible model based approach. 3) Tagging image using users’ own vocabulary by mining social user tagging behavior, and incorporating image taken time, location and visual information.

 

Past Research Interests


说明: http://smiles.xjtu.edu.cn/images/sports%20video.jpgSport Video Content Understanding

Our Sport video content understanding sytem consists of following steps: 1) video shot boundary detection, 2) adaptive event clip segmentation, 3) extract middle level semantics and temporal overall features of each event clip, 4) Learning based event modeling using enhanced HMM, and Hidden Conditional Random Field.




说明: http://smiles.xjtu.edu.cn/images/sports%20video.jpgCompressed Domain based Video Analysis  

We carry out fast video text detection, global motion estimation, shot boundary detection from the bit-streams of compressed domain. We extract DC coefficients for fade-in/out and Flashlight detection, AC coefficients for fast text detection, localization, and tracking. For the D1 format video, the Recall and Precision are all above 90%, and the computational costs is about 200 frames per second. These approaches are successfully utilized in the products of Huawei.


说明: http://smiles.xjtu.edu.cn/images/sports%20video.jpgCorrupted Video Stream Recovery  

During video transmission, when a package is lost or corrupted, this will make the constructed video quality well poor. To solve the problem, we proposed to use global/local motion based error concealment approach, and determining the concealment order by analyzing the spatial information of corrupted regions. The performance is better than that utilized in H.26L.

 


 

Acknowledgement


Our research is supported by:


国家自然科学基金面项目,基于对象的监控视频检索方法研究, 2018.1-2021.12. No. 61772407,

国家自然科学基金重点项目, 面向复杂查询的异质媒体搜索, 2018.1-2022.12. No. 61732008

Microsoft Research Asia, Learning salient feature to enhance the mobile image retrieval,2016.3-2016.12.

NSFC No.61171309:Key studies on mobile based product recommendation approach.(in Chinese: 基于移动互联网的图标广告推荐方法研究,2012.1~2015.12)

NSFC No.60903121: Key study on a unified scalable sport video content understanding approach. ( in Chinese:
一种统一的具有可分级的体育视频内容理解方法研究,2010.1~2012.12)

MSRA: Mobile Sensing based travel guide and travelogue generation.( in Chinese:
基于移动位置感知的旅游导游及有机生成方法,2011.5~2013.5)

中车集团,音视频编码、调制、解调器,参与人

中车集团,液晶屏控制器,参与人

腾讯公司,“新闻、视频封面生成方法研究”,2016.11

亿阳信通集团股份有限公司,“基于运营商运维大数据的自动网络优化方法研究”,2017.10~2017.12

亿阳信通集团股份有限公司,“以图搜图算法”,2016.3~2016.12

热工院合作项目智能化颗粒物分析装置大数据分析软件设计与开发,2017.9-2017.12.

Huawei: Fast video text detection.( in Chinese:
快速视频字幕检测,2008.9~2009.12)


 
ҳģҳģ