rudimentary hip data visualizations

rudimentary hip data visualizations - Printable Version

+- PunBB (https://punbb.in)
+-- Forum: General Discussion - Hipbase.com (https://punbb.in/forumdisplay.php?fid=4)
+--- Forum: Tragically Hip Discussion (https://punbb.in/forumdisplay.php?fid=7)
+--- Thread: rudimentary hip data visualizations (/showthread.php?tid=13612)

rudimentary hip data visualizations - catechism - 11-23-2009

Without boring anyone too much with the details, I decided I wanted to play around with data visualizations. I've been wanting to do that for a while now, but I could never really find data I cared enough about to get my hands dirty -- until it occurred to me to use Hip concert data. So I now have the first in what I hope will be a series of visualizations of increasing interestingness, awesomeness and complexity. It's just a map of North America, color-coded to be darker in places where the Hip have played most often, which means it's dead simple and doesn't tell us anything we don't already know. But I think it's a good first step, and if you guys have ideas for other visualizations or stuff you'd like to see, I'd love to hear them. I think next I want to do some kind of google maps mashup that would show where they've played at a more granular level, and from there I can start pulling in dates and setlists. It could be awesome! At any rate, it's a good learning experience for me.

Anyway, the blog post where I wrote all this up is here: http://www.newsprint-fray.com/2009/11/23/data-visualizations-and-the-tragically-hip-part-0/. It's mostly code, but the map I generated is at the bottom.

Re: rudimentary hip data visualizations - Shane Kroeker - 11-23-2009

catechism Wrote:Without boring anyone too much with the details, I decided I wanted to play around with data visualizations. I've been wanting to do that for a while now, but I could never really find data I cared enough about to get my hands dirty -- until it occurred to me to use Hip concert data. So I now have the first in what I hope will be a series of visualizations of increasing interestingness, awesomeness and complexity. It's just a map of North America, color-coded to be darker in places where the Hip have played most often, which means it's dead simple and doesn't tell us anything we don't already know. But I think it's a good first step, and if you guys have ideas for other visualizations or stuff you'd like to see, I'd love to hear them. I think next I want to do some kind of google maps mashup that would show where they've played at a more granular level, and from there I can start pulling in dates and setlists. It could be awesome! At any rate, it's a good learning experience for me.

Anyway, the blog post where I wrote all this up is here: http://www.newsprint-fray.com/2009/11/23/data-visualizations-and-the-tragically-hip-part-0/. It's mostly code, but the map I generated is at the bottom.

I know a couple years back Hipbaser Laliber put together a google map of all of the locations which were referenced in Tragically Hip songs. For example Thompson, Manitoba - for Thompson Girl... That was interesting. You guys should collaborate!

Re: rudimentary hip data visualizations - catechism - 11-23-2009

Oh, interesting! I thought vaguely about trying to do something with lyrics, but didn't really come up with any ideas beyond word clouds, which I think would be less useful/cool than some kind of more serious linguistic analysis. I hadn't even thought about locations referenced, though. I'll have to see if I can find it. Thanks!

Re: rudimentary hip data visualizations - Tthip - 11-23-2009

catechism Wrote:I hadn't even thought about locations referenced, though. I'll have to see if I can find it. Thanks!

<a class="postlink-local" href="http://hipbase.com/forum/viewtopic.php?f=1&t=11998">viewtopic.php?f=1&t=11998</a>
<a class="postlink-local" href="http://hipbase.com/forum/viewtopic.php?f=1&t=12015">viewtopic.php?f=1&t=12015</a>

Re: rudimentary hip data visualizations - catechism - 11-23-2009

You are an amazing cross-referencing machine. Thanks!

Re: rudimentary hip data visualizations - VioletLite - 11-24-2009

This is VERY cool. I was just thinking the other day about place names referenced in Hip songs, being so many of them. I was crossing the Tacoma Narrows Bridge at the time so it wasn't just a completely random thought!!!

Re: rudimentary hip data visualizations - laliber - 11-24-2009

It has been awhile since I updated the GE file, there are many dead image links not to mention a new album and various tours since 2006 - a little Christmas break project perhaps....

btw catechism - great code - I never really thought about making choropleth maps based on Hip Related data

Re: rudimentary hip data visualizations - laliber - 12-17-2009

This is interesting...

Bruce Cockburn + Toronto a historical tour
<a class="postlink" href="http://cockburnproject.net/toronto/map.html">http://cockburnproject.net/toronto/map.html</a>

Re: rudimentary hip data visualizations - catechism - 12-18-2009

I have made a lyrics tree! And by "made," I mean I scrubbed the lyrics a little and then pasted them into many eyes. But it is actually a lot cooler to play with than I thought it would be.

The caveats are that I pasted the lyrics in as one big file and therefore it doesn't differentiate between songs; the end of one runs into the start of another. [Like: "we live to survive our paradoxes / first thing we'd climb a tree."] I didn't think about that until after the fact, so the file I pasted in has the lyrics in what I think is an entirely random order. At any rate, that means that some of the connections will be a little off, since "first" doesn't actually come after "paradoxes" at any point.

I also did a little scrubbing and standardizing -- swapped "yer" for "you're," for example, and I took out most punctuation not in the middle of words. I had a reason for that originally, and now I think it wasn't necessary, but whatever.

Anyway! You can go there, type in a word, and hit enter. It defaults to showing you the words that come after whatever you've typed, but if you select the 'end' radio button, it will show you the words that come before. My favorite one so far is what happens when you type "heart" and choose 'end.' I think that would make a pretty cool t-shirt, actually.

... have fun? It turns out actual data visualizations are kind of hard, and mostly I have learned that I have a lot to learn. In the meantime, many eyes is going to keep me entertained for a while.

Re: rudimentary hip data visualizations - let_me_out - 12-18-2009

I did "love" and start. Quite cool. Nice work.

Re: rudimentary hip data visualizations - laliber - 12-19-2009

This is great. Had fun comparing Ocean/Lake & Road/Street New Y/New O ect.. and a certain four letter word… :censored

Have you pasted the scrubbed the lyrics into Wordle?

Re: rudimentary hip data visualizations - catechism - 12-19-2009

laliber Wrote:This is great. Had fun comparing Ocean/Lake & Road/Street New Y/New O ect.. and a certain four letter word…

Water's a good one, too. He definitely likes his aquatic metaphors.

laliber Wrote:Have you pasted the scrubbed the lyrics into Wordle?

I have now! Sort of! The cool thing about many eyes is that it lets you do different visualizations on the same dataset, so I went just now and generated a wordle. It's not as pretty as the ones you can get from wordle itself because it has so few options for fonts and color schemes, but it does allow other people to mess with what's there, which I guess is sort of nice?

The other language-based visualizations available are:

tag cloud, which doesn't actually have any tags, but I like it a little better than the wordle one. You can search for specific words to narrow down the cloud, and hover over stuff to see where the words get used (though it'll only show you up to six occurrences). You can also generate a two-word cloud to see which two-word phrase is used most (looks like 'country day' beats out 'new orleans' by one) or how often he says "coconut cream" (four times).
phrase net, which shows connections between words. You can choose "and" to see the words it connects (kings/pretty, fingers/toes), but there are a bunch of other choices like "of the" (queen/furrows, end/rink) and even "'s," which I think is pretty cool (tonight/the, summer/killing). You can also enter your own connector. I really like the idea of this one, but the display is quite bad and it's difficult to read. If you tell it not to hide common words, then it looks way cooler, but seems to think Gord's favorite word is "m," or possibly "t." Pretty hit or miss, really.

Re: rudimentary hip data visualizations - catechism - 12-21-2009

many eyes continues to make visualizations really easy to make but less cool to look at than the ones in my head.

[Image: bubble-chart.png]

That's a screenshot of the bubble chart for songs played on the the most recent tour. If you actually go there, you can hover over the unlabeled bubbles to see what they are. In my ideal world, this chart would be color-coded by album rather than by song, but the site doesn't allow for more than two columns for bubble charts. Also, while I am dreaming, the bubbles would be big enough that I wouldn't have to use the clever code names for songs, rendering that chart completely incomprehensible to people who do not spend their time staring at Hip setlists. And there'd be more variation in bubble size (rather, the units would be larger). And and and.

(Data for this pulled from Sherpa's Shiny Spreadsheet of Statistical Spectacularness.)

(...I should sleep, ever. Or at least code instead of just copying stuff into other stuff and then complaining about how it doesn't do exactly what I want.)