Image retargeting based on self-learning 3D saliency for content-aware data analysis

Published on Apr 1, 2020in Multimedia Tools and Applications2.313
· DOI :10.1007/S11042-017-4436-0
Qiang Lu2
Estimated H-index: 2
(Hefei University of Technology),
Gang Tao1
Estimated H-index: 1
Yanxiang Chen7
Estimated H-index: 7
(Hefei University of Technology)
Image retargeting is a process to change the resolution of image while preserve interesting regions and avoid obvious visual distortion. In other words, it focuses on image content more than anything else that applies to filter the useful information for data analysis. Existing approaches may encounter difficulties on the various types of images since most of these approaches only consider 2D features, which are sensitive to the complexity of the contents in images. Researchers are now focusing on the RGB-D information, hoping depth information can help to promote the accuracy. However it is not easy to obtain the RGB-D image we need anywhere and how to utilize depth information is still at the exploration stage. In this paper, instead of using RGB-D data captured by 3D camera, we employ an iterative MRF learning model to predict depth information from a single still image. Then we propose our self-learning 3D saliency model based on the RGB-D data and apply it on the seam carving framework. In seam caving, the self-learning 3D saliency is combined with L1-norm of gradient for better seam searching. Experimental results demonstrate the advantages of our method using RGB-D data in the seam carving framework.
📖 Papers frequently viewed together
1 Author (Oleg Muratov)
1 Citations
8 Citations
#1Jianbing Shen (BIT: Beijing Institute of Technology)H-Index: 53
#2Dapeng Wang (BIT: Beijing Institute of Technology)H-Index: 1
Last. Xuelong LiH-Index: 115
view all 3 authors...
Image seam carving algorithm should preserve important and salient objects as much as possible when changing the image size, while not removing the secondary objects in the scene. However, it is still difficult to determine the important and salient objects that avoid the distortion of these objects after resizing the input image. In this paper, we develop a novel depth-aware single image seam carving approach by taking advantage of the modern depth cameras such as the Kinect sensor, which captu...
50 CitationsSource
Jul 15, 2013 in ICME (International Conference on Multimedia and Expo)
#1Semir Elezovikj (TU: Temple University)H-Index: 2
#2Haibin Ling (TU: Temple University)H-Index: 69
Last. Xiufang Chen (Rowan University)H-Index: 1
view all 3 authors...
In this paper, we propose the use of depth-information to protect privacy in person-aware visual systems while preserving important foreground subjects and scene structures. We aim to preserve the identity of foreground subjects while hiding superfluous details in the background that may contain sensitive information. We achieve this goal by using depth information and relevant human detection mechanisms provided by the Kinect sensor. In particular, for an input color and depth image pair, we fi...
6 CitationsSource
#1Yun Liang (SCAU: South China Agricultural University)H-Index: 5
#2Zhuo Su (SYSU: Sun Yat-sen University)H-Index: 12
Last. Xiaonan Luo (SYSU: Sun Yat-sen University)H-Index: 17
view all 5 authors...
Image retargeting is a critical technique in displaying images on devices with different resolutions. This study presents a new image retargeting algorithm based on aesthetic-based cropping and scaling. A composite measurement is first constructed under the guidelines of composition aesthetics in photographing. An aesthetic-based cropping is proposed to yield an optimal candidate retargeted image with maximum aesthetic value computed via a constructed composite measurement. The optimal candidate...
8 CitationsSource
#1Meir Johnathan Dahan (TAU: Tel Aviv University)H-Index: 1
#2Nir ChenH-Index: 1
Last. Daniel Cohen-Or (TAU: Tel Aviv University)H-Index: 96
view all 4 authors...
As depth cameras become more popular, pixel depth information becomes easier to obtain. This information can clearly enhance many image processing applications. However, combining depth and color information is not straightforward as these two signals can have different noise characteristics, differences in resolution, and their boundaries do not generally agree. We present a technique that combines depth and color image information from real devices in synergy. In particular, we focus on combin...
42 CitationsSource
Oct 7, 2012 in ECCV (European Conference on Computer Vision)
#1Congyan Lang (NUS: National University of Singapore)H-Index: 15
#2Tam V. Nguyen (NUS: National University of Singapore)H-Index: 16
Last. Shuicheng Yan (NUS: National University of Singapore)H-Index: 120
view all 6 authors...
Most previous studies on visual saliency have only focused on static or dynamic 2D scenes. Since the human visual system has evolved predominantly in natural three dimensional environments, it is important to study whether and how depth information influences visual saliency. In this work, we first collect a large human eye fixation database compiled from a pool of 600 2D-vs-3D image pairs viewed by 80 subjects, where the depth information is directly provided by the Kinect camera and the eye tr...
182 CitationsSource
#1Seung-Won Jung (Samsung)H-Index: 18
#2Sung-Jea Ko (KU: Korea University)H-Index: 30
In this paper, we present a novel depth sensation enhancement algorithm considering the behavior of human visual system toward stereoscopic image displays. On the basis of the recent studies on the just noticeable depth difference (JNDD), which represents a threshold at which a human can perceive the depth difference between objects, we modify the depth image such that neighboring objects in the depth image can have a depth value difference of at least the JNDD. This modification is modeled via ...
37 CitationsSource
Jul 5, 2012 in QoMEX (Quality of Multimedia Experience)
#1Matthieu Urvoy (University of Nantes)H-Index: 5
#2Marcus Barkowsky (University of Nantes)H-Index: 21
Last. Narciso Garcia (UPM: Technical University of Madrid)H-Index: 28
view all 8 authors...
Research in stereoscopic 3D coding, transmission and subjective assessment methodology depends largely on the availability of source content that can be used in cross-lab evaluations. While several studies have already been presented using proprietary content, comparisons between the studies are difficult since discrepant contents are used. Therefore in this paper, a freely available dataset of high quality Full-HD stereoscopic sequences shot with a semiprofessional 3D camera is introduced in de...
100 CitationsSource
Jun 16, 2012 in CVPR (Computer Vision and Pattern Recognition)
#1Varsha Hedau (Nokia)H-Index: 6
#2Derek Hoiem (UIUC: University of Illinois at Urbana–Champaign)H-Index: 47
Last. David Forsyth (UIUC: University of Illinois at Urbana–Champaign)H-Index: 81
view all 3 authors...
In this paper we consider the problem of recovering the free space of an indoor scene from its single image. We show that exploiting the box like geometric structure of furniture and constraints provided by the scene, allows us to recover the extent of major furniture objects in 3D. Our “boxy” detector localizes box shaped objects oriented parallel to the scene across different scales and object types, and thus blocks out the occupied space in the scene. To localize the objects more accurately i...
106 CitationsSource
Jun 13, 2010 in CVPR (Computer Vision and Pattern Recognition)
#1Matthias Grundmann (Georgia Institute of Technology)H-Index: 14
#2Vivek Kwatra (Google)H-Index: 18
Last. Irfan Essa (Georgia Institute of Technology)H-Index: 60
view all 4 authors...
We introduce a new algorithm for video retargeting that uses discontinuous seam-carving in both space and time for resizing videos. Our algorithm relies on a novel appearance-based temporal coherence formulation that allows for frame-by-frame processing and results in temporally discontinuous seams, as opposed to geometrically smooth and continuous seams. This formulation optimizes the difference in appearance of the resultant retargeted frame to the optimal temporally coherent one, and allows f...
105 CitationsSource
Nov 7, 2009 in ICIP (International Conference on Image Processing)
#1Radhakrishna Achanta (EPFL: École Polytechnique Fédérale de Lausanne)H-Index: 14
#2Sabine Süsstrunk (EPFL: École Polytechnique Fédérale de Lausanne)H-Index: 41
Content aware image re-targeting methods aim to arbitrarily change image aspect ratios while preserving visually prominent features. To determine visual importance of pixels, existing re-targeting schemes mostly rely on grayscale intensity gradient maps. These maps show higher energy only at edges of objects, are sensitive to noise, and may result in deforming salient objects. In this paper, we present a computationally efficient, noise robust re-targeting scheme based on seam carving by using s...
180 CitationsSource
Cited By19
#1Henan Sun (NWAFU: Northwest A&F University)
#2Haowei Xu (NWAFU: Northwest A&F University)
Last. Nan Geng (NWAFU: Northwest A&F University)
view all 0 authors...
Abstract null null Alternaria blotch, Brown spot, Mosaic, Grey spot, and Rust are 5 common apple leaf diseases that severely impact apple production and quality. At present, although many CNN methods have been proposed for apple leaf diseases, there are still lack of apple leaf disease detection models that can be applied on mobile devices, which limits their application in practical production. This paper proposes a light-weight CNN model that can be deployed on mobile devices to detect apple l...
#1Muhammad Fahad Khan (QAU: Quaid-i-Azam University)H-Index: 6
#2Khalid Saleem (QAU: Quaid-i-Azam University)H-Index: 8
Last. Shariq Bashir (Florida State University College of Arts and Sciences)H-Index: 11
view all 4 authors...
Block cipher has been a standout amongst the most reliable option by which data security is accomplished. Block cipher strength against various attacks relies on substitution boxes. In literature, extensively algebraic structures, and chaotic systems-based techniques are available to design the cryptographic substitution boxes. Although, algebraic and chaotic systems-based approaches have favorable characteristics for the design of substitution boxes, but on the other side researchers have also ...
#1Swarnajit Ray (IAU: Islamic Azad University)H-Index: 7
#2Arunita Das (KGEC: Kalyani Government Engineering College)H-Index: 4
Last. Prabir Kumar Naskar (Government College)H-Index: 3
view all 5 authors...
Pathological color image segmentation is an exigent procedure due to the existence of imperceptibly correlated, and indistinct multiple regions of concern. Multi-level thresholding has been introduced as one of the most significant image segmentation procedures for pathological analysis. However, finding an optimal set of threshold values is an extremely time-consuming task, and crucially depends on the objective function criterion. In order to solve these problems, this paper presents a multi-l...
1 CitationsSource
#1Anson Pinhero (SCMS School of Engineering and Technology)
#2Anupama M L (SCMS School of Engineering and Technology)
Last. AnanthaKrishnan S (SCMS School of Engineering and Technology)
view all 7 authors...
Abstract With the fast growth of malware’s volume circulating in the wild, to obtain a timely and correct classification is increasingly difficult. Traditional approaches to automatic classification suffer from some limitations. The first one concerns the feature extraction: static approaches are hindered by code obfuscation techniques, while dynamic approaches are time consuming and evasion techniques often impede the correct execution of the code. The second limitation regards the building of ...
2 CitationsSource
#1R. MonikaH-Index: 2
Last. Rahul Kumar (SRM University)H-Index: 1
view all 3 authors...
Internet of Underwater Things (IoUT) consists of a large number of interconnected resource-constrained underwater devices that are capable of monitoring vast unexplored water bodies. Specifically, these devices are equipped with cameras to capture the underwater scenes and communicate them with each other and also with the cloud. However the data generated is very high which limits the performance of the IoUT devices in terms of computational capabilities and battery lifetime. Block Compressed S...
1 CitationsSource
#1Lu Sun (U of O: University of Ottawa)
#2Hussein Al Osman (U of O: University of Ottawa)H-Index: 13
Last. Jochen Lang (U of O: University of Ottawa)H-Index: 18
view all 3 authors...
Our augmented reality online assistance platform enables an expert to specify 6DoF movements of a component and apply the geometrical and physical constraints in real-time. We track the real components on the expert’s side to monitor the operations of an expert. We leverage a remote rendering technique that we proposed previously to relieve the rendering burden of the augmented reality end devices. By conducting a user study, we show that the proposed method outperforms conventional instructiona...
#1Abhishek Samanta (KIIT: KIIT University)H-Index: 2
#2Aheli Saha (KIIT: KIIT University)H-Index: 2
Last. Hong Lin (University of Houston–Downtown)
view all 4 authors...
The location of discriminative features and reduction of model complexity are the two main research directions in fine-grained image classification. The manual annotation of object is very labor-intensive, and the commonly used model compression methods usually reduce the classification accuracy while compressing the model. In this paper, we propose a Sparse Focus Framework(SFF) based on Bilinear Convolutional Neural Network(BCNN), which includes self-focus module and sparse scaling factors. The...
With the continual development of deep learning, the image processing in Internet of Things is the key technology. Nevertheless, many deep learning methods cannot deal with the special needs of Internet of Things, for example, the Internet of vehicles and ships for the traffic haze image. Particularly, haze removal in the water area, because of the influence of water vapor, is more difficult than that in the ordinary scene. And the dehazing of water area has practical value in shipping and aeria...
2 CitationsSource