PostSecret: Part 3, Is PostSecret Sensationalized?

You will recall that the goal of the secret coding I discussed in my last post was to set the stage to test a variety of questions about PostSecret. Specifically, is PostSecret sensationalized?

Again, PostSecret may be selecting only the most provocative of secrets to boost website hits or book sales. So this seems an important issue to explore. But how could we go about testing this question?

Basically, what we needed was a comparison sample of unselected secrets. Again, all the PostSecrets the public ever gets to see have been selected by Frank Warren and his associates. What my research group needed was a sampling of secrets that had not undergone a selection procedure. Once this unselected sample was obtained we could code those secrets and compare the content between the unselected and selected (PostSecret) samples to see if there were any differences in content. For the purposes of this study we defined "sensationalized" as content with sexual or mental health (e.g., abuse, drug use, trauma, suicide) themes. That is, PostSecret would be considered sensationalized if it published more sexual or mental health content relative to a comparison sample where no picking and choosing had occurred.

Great idea, but where to get this sample of unselected secrets? We tried three approaches.

First, I sent letter to Frank Warren describing the project and asking for his participation. Basically, we asked if he might supply us copies of a few hundred of the secrets he had received that he never published. In short, we wanted to see a sample of the secrets that didn't make the posting/publishing cut to see if the content of those secrets differed from those that did get posted/published.

Outcome: Warren never answered. On to the next idea.

Second, while we waited for Warren to answer we tried to collect our own sample of secrets. We started a project called YourSecret, launched its Facebook group, and asked acquaintances of ours and people who attended a presentation of ours to send in their secret to a P.O. Box. We did get some participation, aided by a school newspaper article, but not enough for a comparative sample. The secrets we obtained can be seen at the YourSecret Facebook site.

So, we were foiled again. On to the next idea.

Third, at the start of the project the students and I joined the official PostSecret Facebook group. There we noted that people were uploading hundreds of digital postcards sharing their secrets. Best of all for our purposes, these uploads are unselected. Any member of the Facebook group can upload a digital postcard. We had our comparative sample.

Given that Facebook is generally populated by high school and college students, we selected the My Secret publication as our sample of PostSecrets. (My Secret is devoted to the secrets of highschool and college age persons.) The My Secret book had 188 secrets so we randomly selected 188 secrets from the PostSecret fan site. We then coded the Facebook secrets just as we had all the My Secret secrets. We then ran statistical comparisons across all content themes, particularly looking at the sexual and mental health codes.

The Verdict? PostSecret doesn't appear to be sensationalized.

That is, when we compared the selected secrets in My Secret to the freely uploaded content on the Facebook site we saw no more sexual or mental health content in the for-profit PostSecret publication relative to the unselected sample.

We did, however, see a different kind of selection bias. Specifically, we found significantly more Relational codes in the Facebook sample compared to the PostSecret publication. An analysis of the Relational subcodes revealed that many of these relational codes were of the Romantic Anxiety and Unrequited variety. In short, the Facebook secrets were more romantically sappy and maudlin. If Frank Warren has a selection bias it seems he's keeping a cap on this kind of submitted content. Perhaps for good reason. Teens can be a bit, well, romantically histrionic.

To conclude, we found no evidence that PostSecret is consciously selecting the most sensationalized secrets to create more web hits or sales. Of course, this conclusion is tempered by one major limitation. Namely that the Facebook fan site is populated by people who regularly consume PostSecret. Consequently, these fans may be imitating the content they find in PostSecret. That is, a copycat effect may be in play. This possibility, a very reasonable one, should be kept in mind.

This entry was posted by Richard Beck. Bookmark the permalink.

6 thoughts on “PostSecret: Part 3, Is PostSecret Sensationalized?”

  1. "Teens can be a bit, well, romantically histrionic"...HA! I love it! ;)

    I so enjoy Postsecret; thus your posts discussing the topic always interest me! I thoroughly enjoyed seeing the secrets y'all collected via the facebook group as well! That was like it being a new group of Sunday secrets posted all over again for me! HA!

  2. The only other issue I would raise is whether there ends up being a self-selection issue. Specifically, people submitting cards now are often familiar with what the post secret content entails, and thus submit similar secretes resulting in a reinforced content in the current direction. The only way to get around this however would be to see a sample of the first batch of post-secret cards before the books were out.


  3. Kimberly,
    Count me as a fan as well. Your comment awhile back is one of the reasons I'm doing this series.

    Thanks for dropping by. I hope you find the research we've done the highest form of compliment. The students I worked with have been profoundly affected by your work.

    I agree. We need a large and diverse collection of secrets from people who have never been exposed to PostSecret. The papers we presented were initial forays into the area. So much more is waiting to be done or improved upon.

  4. I have had only brief contact with PostSecret and when you posted a sample earlier they certainly did NOT resonate with me. I view many of these things as not so much 'secrets' but simply thoughts that arise at the convergence circumstances. Some certainly are 'secrets' in the sense of how they intentionally kept from parties that arguably should know this knowledge, but overall from my perspective I view these not as untrue but yes as somewhat sensationalized.

  5. Not sure I recall what comment that was that I left in the past...HA HA! I think I'm getting old! ;)

    That's pretty cool that Frank stopped in! I remember thinking how cool it was when he responded to an email I sent him awhile back. Not to mention how happy I was when I got his autograph on a picture of him I printed out and took to one of his book signings. He said I was the first person to ever have him sign a picture of himself...HA HA! I keep it in my binder with all my other autographed pictures of individuals who have been influential in the area of mental health.

Leave a Reply