DOCTORAL THESIS: PRIVAWARE (NSF Grant No. 1527421)
Year: 2016 to Present | Clemson University | Collaborators: Dr. Kelly Caine, Dr. Bart Knijnenburg, Dr. Apu Kapadia, Dr. David Crandall, Dr. Hongxin Hu, Rakib Hasan, Nishant Vishwamitra | Skill: Experimental Design, Quantitative Data Analysis | Role: PhD Leader
Through my doctoral research, I aim to provide a better online photo privacy protection strategy than existing approaches such as photo self-censorship and recipient control on Online Social Networks (OSNs). To inform the building for an effective and usable photo privacy protection system on OSNs, I gain understanding on the two parameters that influence photo privacy–photo content and recipient–through a series of studies and provide design guidelines. My research will benefit privacy researchers, online social network designers, policymakers, and computer vision researchers. I plan to answer the following questions.
Study and select promising obfuscation methods :
What are the effective and usable obfuscations?
As an extreme privacy protection scheme, is photo self-censorship prevalent? Can obfuscations combat it and encourage photo sharing?
Understand what to obscure (content) and prevent from whom (recipient):
What is the sensitive content in photos to be obscured?
With different groups of recipients, what are users’ preferences of sharing different categories of sensitive content?
To create a more collaborative and usable privacy protection mechanism, I need to have a better understanding of users’ current photo privacy protection behaviors:
Other than identified self-censorship and recipient control that OSNs provide, are there other privacy protection schemes that they often use?
Study 1: Identifying effective and usable obfuscations Focusing on controlling photo content disclosure, my first study introduces privacy-enhancing obfuscations and conduct an online experiment with 271 participants to evaluate their effectiveness against human recognition and how they affect the viewing experience. The obfuscations we investigated are shown below. Results suggest that avatar and inpainting are two promising obfuscations which are effective and provide a good viewer experience, while the widely adopted blurring and pixelating are not effective. Two pilot studies are published in HFES and CV-COPS. The complete study is published and presented at CSCW 2018.
Research method: Controlled experiment; Online survey Sample: 271 participants recruited on Amazon MTurk Independent variable: 14 obfuscation conditions (including as-is as the baseline condition) Dependent variable:
Study 2: Avatar and inpainting are robust when de-identifying both unfamiliar and familiar people In this study, one limitation is that we only explored obfuscations’ effectiveness for de-identifying unfamiliar people (people in stimuli were unknown to the participants). We conducted an experiment where participants identified both familiar and unfamiliar people applied different obfuscation methods. We find that avatar and inpainting are robust to the increased likelihood of recognition associated with familiarity so they would be useful privacy enhancement tools on OSNs. We are preparing the manuscript for a journal publication. Research method: Controlled experiment; Online survey Sample: 230 participants recruited on Amazon MTurk Independent variable: 7 obfuscation conditions by 2 familiarity levels (familiar vs. unfamiliar) Dependent variable:
Users' experience: Photo satisfaction; Perceived Photo information sufficiency; Photo enjoyment; Perceived social presence; Obfuscation likability
Quantitative data analysis method:
Logistic mixed-effects models
Linear mixed-effects models
Study 3: Obscuring scene elements My prior work only considers human as the most sensitive content to be obscured, however, various incidental information could also harm privacy, such as monitor or indoor scene. We studied 11 filters applied to obfuscate 20 different objects and evaluated how effectively they protect privacy and preserve image quality for human viewers . This work has been published and was presented at CHI 2018.
Research method: Controlled experiment; Online survey Sample: 570 participants recruited on Amazon MTurk Independent variable:
Study 4: Obfuscation may combat photo self-censorship Prior studies have identified image obfuscation methods (e.g., blurring) to enhance privacy, but many of these methods adversely affect viewers’ satisfaction with the photo, and people may not use such transforms because of their appearance. In this paper, we study the novel hypothesis that it may be possible to restore viewers’ satisfaction by ‘boosting’ or enhancing the aesthetics of an obscured image, thereby compensating for the negative effects of a privacy transform. Using a between-subjects online experiment, we studied the effects of three artistic transformations on images that had objects obscured using three popular obfuscation methods validated by prior research. Our findings suggest that using artistic transformations can indeed mitigate the negative effects of obfuscation methods in some circumstances, thus allowing for both privacy andviewer satisfaction. This work has been accepted at CHI 2019.
Research method: Between subject controlled experiment; Online survey Sample: 653 participants recruited on Amazon MTurk Independent variable:
Users' experience: Photo satisfaction; Perceived Photo information sufficiency; Visual aesthetics
Quantitative data analysis method:
Dunn's post hoc test with Bonferroni correction
This paper will be available in ACM Digital Library soon.
Study 5: Obfuscation may combat photo self-censorship We quantified the prevalence of self-reported photo self-censorship and associated this with gender, age, and privacy preference. We find that over half of the participants have self-censored photos due to privacy concerns and privacy-conscious people were more likely to censor photos. Among the participants who reported they had self-censored photos, half of them were willing to share the previously censored photo if they would be able to obfuscate portions of the photo to enhance privacy. Hence, privacy-preserving obfuscations may be useful for combating photo self-censorship. We are preparing the manuscript for a journal publication.
Privacy preference/consciousness about personal information
If the user has declined to upload a photo to an OSN for privacy reasons
If the user is willing to upload the photo which he/she has previously refused to share if they are able to obscure the sensitive portion
Quantitative data analysis method:
Logistic regression model
Study 6: Identifying sensitive content and users’ sharing preference Focusing on the content parameter, we must know what portions are considered sensitive and should be obscured. We aim to 1) identify sensitive content in photos from a human-centered perspective and 2) study people’s sharing preference of different sensitive content with 20 recipient groups (e.g., friend, colleague). We plan to provide a taxonomy of sensitive content categories associated with users’ sharing preference with different recipients. We collected sensitive photos and/or descriptions of sensitive photos from 116 MTurk participants and asked them to identify which elements of the photo made each photo sensitive. Afterwards, they rated the likelihood to share this sensitive content with each of the 20 recipient groups. We have published the result from our pilot study in CV-COPS, and we are preparing the manuscript of the main study for a journal publication.
Research method: Controlled experiment; Online survey; Card sorting Sample:
116 participants recruited on Amazon MTurk for photo elicitation and collecting sharing preference
14 participants recruited on campus for grouping the collected sensitive content via card sorting
Hasan, R., Li, Y., Caine, K., Crandall, D. J., Hoyle, R., & Kapadia, A. (2019). Can Privacy Be Satisfying? On Improving Viewer Satisfaction for Privacy-Enhanced Photos Using Aesthetic Transforms. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM.
Li, Y.. (2018). Photo Privacy Protection on Online Social Networks. In Companion of the 2018 ACM Conference on Computer Supported Cooperative Work and Social Computing. ACM.
Li, Y., & Caine, K. (2018). Applying A Behavioral Theory of Privacy to Online Photo Sharing. In Proceedings of the ACM Conference on 2018 Networked Privacy Workshop at CSCW. ACM.
Li, Y., Troutman, Y., Knijnenburg, B. P., and & Caine, K. (2018). Human perceptions of sensitive content in photos. In Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE.
Hasan, R., Hassan, E., Li, Y., Caine, K., Crandall, D., Hoyle, R., & Kapadia, A. (2018). Viewer experience of obscuring scene elements in photos to enhance privacy. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 47.
Currently we are working on the follow up studies to build an effective and usable online photo privacy protection system.