You will recall that the goal of the secret coding I discussed in my last post was to set the stage to test a variety of questions about PostSecret. Specifically, is PostSecret sensationalized?

Again, PostSecret may be selecting only the most provocative of secrets to boost website hits or book sales. So this seems an important issue to explore. But how could we go about testing this question?

Basically, what we needed was a comparison sample of unselected secrets. Again, all the PostSecrets the public ever gets to see have been selected by Frank Warren and his associates. What my research group needed was a sampling of secrets that had not undergone a selection procedure. Once this unselected sample was obtained we could code those secrets and compare the content between the unselected and selected (PostSecret) samples to see if there were any differences in content. For the purposes of this study we defined "sensationalized" as content with sexual or mental health (e.g., abuse, drug use, trauma, suicide) themes. That is, PostSecret would be considered sensationalized if it published more sexual or mental health content relative to a comparison sample where no picking and choosing had occurred.

Great idea, but where to get this sample of unselected secrets? We tried three approaches.

First, I sent letter to Frank Warren describing the project and asking for his participation. Basically, we asked if he might supply us copies of a few hundred of the secrets he had received that he never published. In short, we wanted to see a sample of the secrets that didn't make the posting/publishing cut to see if the content of those secrets differed from those that did get posted/published.

Outcome: Warren never answered. On to the next idea.

Second, while we waited for Warren to answer we tried to collect our own sample of secrets. We started a project called YourSecret, launched its Facebook group, and asked acquaintances of ours and people who attended a presentation of ours to send in their secret to a P.O. Box. We did get some participation, aided by a school newspaper article, but not enough for a comparative sample. The secrets we obtained can be seen at the YourSecret Facebook site.

So, we were foiled again. On to the next idea.

Third, at the start of the project the students and I joined the official PostSecret Facebook group. There we noted that people were uploading hundreds of digital postcards sharing their secrets. Best of all for our purposes, these uploads are unselected. Any member of the Facebook group can upload a digital postcard. We had our comparative sample.

Given that Facebook is generally populated by high school and college students, we selected the My Secret publication as our sample of PostSecrets. (My Secret is devoted to the secrets of highschool and college age persons.) The My Secret book had 188 secrets so we randomly selected 188 secrets from the PostSecret fan site. We then coded the Facebook secrets just as we had all the My Secret secrets. We then ran statistical comparisons across all content themes, particularly looking at the sexual and mental health codes.

The Verdict? PostSecret doesn't appear to be sensationalized.

That is, when we compared the selected secrets in My Secret to the freely uploaded content on the Facebook site we saw no more sexual or mental health content in the for-profit PostSecret publication relative to the unselected sample.

We did, however, see a different kind of selection bias. Specifically, we found significantly more Relational codes in the Facebook sample compared to the PostSecret publication. An analysis of the Relational subcodes revealed that many of these relational codes were of the Romantic Anxiety and Unrequited variety. In short, the Facebook secrets were more romantically sappy and maudlin. If Frank Warren has a selection bias it seems he's keeping a cap on this kind of submitted content. Perhaps for good reason. Teens can be a bit, well, romantically histrionic.

To conclude, we found no evidence that PostSecret is consciously selecting the most sensationalized secrets to create more web hits or sales. Of course, this conclusion is tempered by one major limitation. Namely that the Facebook fan site is populated by people who regularly consume PostSecret. Consequently, these fans may be imitating the content they find in PostSecret. That is, a copycat effect may be in play. This possibility, a very reasonable one, should be kept in mind.

Next Post: PostSecret, Experimental Confession and Virtual Confession Boxes