Strongly Typed

Rhapsody, a subscription music service I've used for the last couple years, makes a lot of their data available through RSS feeds and XML. For a project I'm working on I wanted to parse one of their RSS feeds with Ruby and I thought I'd share how I did it.

The Rhapsody feeds are RSS 2.0 but also include Rhapsody specific elements in the 'rhap' namespace, so it was necessary to use a parser that could be extended to parse these elements. Ruby includes a library for parsing RSS but after reading the source I wasn't sure how to extend it. Fortunately, some searching turned up the Syndication library which the author designed to be easily extended. After doing a 'gem install syndication' you're all set to begin coding.

The Syndication parser works by defining objects that map (roughly) to elements in the RSS document. When the parser is parsing an element it maps element attributes to object attributes with namespace (if any) prepended to the object attribute with an underscore. For example, the attribute rhap:rcid would become rhap_rcid and if an attr_accessor named rhap_rcid existed on the object it would get set to the value of the rhap:rcid attribute. I wanted to extend the Syndication::RSS:Item class so I started out by defining a module called Syndication::Rhapsody::Item that had all the properties I was interested in. Then, with that defined, it was simply a matter of including this module in the Syndication::RSS:Item class:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

require 'syndication/rss'

module Syndication
  module Rhapsody
    module Item
      attr_accessor :rhap_rcid
      attr_accessor :rhap_artist
      attr_accessor :rhap_artist_rcid
      attr_accessor :rhap_album
      attr_accessor :rhap_album_rcid
      attr_accessor :rhap_album_art
      attr_accessor :rhap_album_release_date
      attr_accessor :rhap_album_original_release_date
      attr_accessor :rhap_album_type
      
      # Need to override the tag2method defined in class Container because it
      # doesn't deal with tags with dashes in them. Ruby can't handle method
      # names with dashes so we switch to underscores.
      def tag2method(tag)
        return tag.downcase.gsub(/[:-]/, '_') + '='
      end
    end
  end
  
  module RSS
      class Item
        include Rhapsody::Item
      end
    end
end

You'll notice that I also ended up overriding the definition of the tag2method method. This is because Ruby doesn't allow variable names with hyphens (quite sensibly) so the elements in the 'rhap' namespace that had a hyphen in them were getting ignored. To fix that I simply had tag2method substitute an underscore for a hyphen.

With the Syndication::RSS::Item modified it's now simply a matter of creating the parser and reading the feed:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

require 'rss/2.0'
require 'open-uri'
require 'syndication/rhapsody'

class RhapsodyReader
  def initialize(url = 'http://feeds.rhapsody.com/new-releases.rss')
    @url = url
  end

  def read
    parser = Syndication::RSS::Parser.new
    @feed = nil
    open(@url) do |s|
      content = s.read
      @feed = parser.parse content
    end
    
    @feed.items.each do |item|
      puts "Got rcid '#{item.rcid}'"
    end
  end
end

For more information on the Rhapsody web-service you can go here: http://webservices.rhapsody.com/. Like I mentioned above, Rhapsody also makes some of their data available as non-RSS XML and I'll write about using REXML to parse it in the future.

I have a few programming related things I'd like to do or am in the process of doing and I thought I'd write them down here to keep track of them.

1. Learn Scala. This is partly motivated by my desire to use the lift framework and partly by interest in the language itself. I used to shudder involuntarily when I heard the word 'functional' (unpleasant flashbacks of YAPL-to-C translators written in LISP in college) but the concept of a functional/OO hybrid language seems like a good way to restore my relationship with my highly-parenthesized brethren.
   
2. Write a JSON serializer in Java. I think there are probably a couple different approaches that I can take and I'm curious which is best. I might also try to write a deserializer. This is a project I'm undertaking for informational purposes and not to use the actual code. I might also try to write a Scala implementation just to see how they differ.
   
3. Learn Python. This is something I need to do for work. I've heard a lot of good things about Python so I'm looking forward to working with it. This is more of a passive learning since I'll learn as much as I need to know to do what I need to do. Probably not the best way to learn a language, but for now it'll have to suffice.
   
4. Write a JavaScript parser that will let me use Java style class and interface definitions and then convert it to JavaScript on the fly. I hate the way JavaScript does classes and member access and I want some way to clearly define what my objects look like and behave like. I would love to be able to write:
   

   
   package com.joelpm.widgets;
   
   public class WidgetFoo {
     protected var width;
     protected var height;
   
     public WidgetFoo(var width, var height) {
       this.width = width;
       this.height = height;
     }
   
     public var getWidth() { return width; }
     public var getHeight() { return height; }
   
     public var move(var xDist, var yDist) {
       // do move...
     }
   }
   
   

   Instead of having the equivalent JavaScript that might look something like this:

   
   var com = {};
   var com.joelpm = {};
   var com.joelpm.widgets = {};
   
   com.joelpm.widgets.WidgetFoo = function(width, height) {
     this.width = width;
     this.height = height;
   }
   
   com.joelpm.widgets.WidgetFoo.prototype.getHeight = function() {
     return this.height;
   }
   
   com.joelpm.widgets.WidgetFoo.prototype.getWidth = function() {
     return this.width;
   }
   
   com.joelpm.widgets.WidgetFoo.prototype.move = function(xDist, yDist) {
     // do move...
   }
   
   

   Clearly, the JS parser would just convert the much nicer looking Java style class declaration syntax into the equivalent JS syntax and evaluate it, but development would be much nicer and the code would be clearer to both myself and anyone else who has to read it. I know there are JS compilers like Google's GWT that will turn actual Java into JS, but I'm looking for something that doesn't require a compile cycle to pick up changes. Anyway, this is probably something I'll never get around to, but the thought was prompted by a discussion I had yesterday.
   
5. In theory I also want to keep working on my RoR picture application. I've wanted an application that would let me upload all my pictures to a directory on my webhost via FTP and then process them all with a web-interface. Once they're all uploaded I want to go to the web-interface, type in the directory name, and click a button. The app should then find all new pictures, load the details (EXIF, size, etc) into a DB, create thumbnails, and create a default album. From there I want to be able to add the photos to custom albums and set privacy. I've got some basic code to read a dir and insert photos into a DB but there's a lot left to do. Recently this has been less important than learning Scala, but I may bounce back and forth between the two.

Looking back over my list I guess I've got enough to keep me busy for a while, and that's not counting the non-programming projects I'd also like to tackle at some point, like setting up my Ubuntu workstation as a wireless router that connects to the internet using my EVDO card. So much to do, so little time...

Since I get to play with Java at work all day I decided to spend the free minutes I have on Ruby/Rails. Towards that end I decided I'd start with an app that already existed and peak at its innards while extending it. A blogging application seemed like the perfect start and Mephisto, based on what I've read, is much simpler than Typo so I chose it.

My goal was to get a full development environment set up so that I could develop and run locally, store code in Subversion, and deploy to the live site using Capistrano. My local development is done on OS X and the live site is on a shared host provided by Site5.

There were a couple articles that I found very helpful. The first is over at TheBitGuru titled Setting up Capistrano on Site5. This article will help you get your Subversion repository set up and your Rails application 'Capistranized.' After you've done that head over to fluct.isono.us and check out Moving Home... with Capistrano on Site5 and part two Deploy Mephisto on Rails RC1 with Capistrano.

After those articles you should have Mephisto checked into your Subversion repository on Site5 and deployable with a shared frozen rails via James' shared_rails rake task. It was at this point that I started running into some problems. I would deploy my code and nothing would happen - attempts to access joelpm.com would result in a 0 byte response and there were no results in the production log and nothing in the fastcgi crash log.

As a sanity check I ran Adam Greenfield's Mephisto install script and everything worked, so I knew that it was possible to get Mephisto running on Site5. After that I started on the dozen different attempts to fix things, one of which included rewriting the shared_rails task based on the code in the rails:freeze:edge task.

desc "deploy shared rails environment"
task :deploy_shared_rails do
  
  ENV['SHARED_PATH'] = '../../shared' unless ENV['SHARED_PATH']
  ENV['RAILS_PATH']  = File.join(ENV['SHARED_PATH'], 'rails')
  svn_root = "http://dev.rubyonrails.org/svn/rails/"
  symlink_path  = 'vendor/rails'

  if ENV['TAG']
    rails_svn = "#{svn_root}/tags/#{ENV['TAG']}"
    export_path = "#{ENV['RAILS_PATH']}/tag_#{ENV['TAG']}"
  else
    rails_svn = "#{svn_root}/trunk"
    if ENV['REVISION'].nil?
      ENV['REVISION'] = /^r(\d+)/.match(%x{svn -qr HEAD log #{svn_root}})[1]
      puts "REVISION not set. Using HEAD, which is revision #{ENV['REVISION']}."
    end
    export_path = "#{ENV['RAILS_PATH']}/rev_#{ENV['REVISION']}"   
  end
      
  # do we need to export this tag/revision?
  unless File.exists?(export_path)
    puts "setting up rails " + (ENV['TAG'] ? "tag #{ENV['TAG']}" : "rev #{ENV['REVISION']}")

    mkdir_p export_path

    get_framework do |framework|
      system "svn export #{rails_svn}/#{framework} #{export_path}/#{framework}" + (ENV['REVISION'] ? " -r #{ENV['REVISION']}" : "")
    end
  end

  puts 'linking rails'
  rm_rf   symlink_path
  mkdir_p symlink_path

  get_framework do |framework|
    ln_s File.expand_path("#{export_path}/#{framework}"), "#{symlink_path}/#{framework}"
  end
  
  touch symlink_path + (ENV['TAG'] ? "/TAG_#{ENV['TAG']}" :  "/REVISION_#{ENV['REVISION']}")
end

def get_framework
  %w( railties actionpack activerecord actionmailer activesupport actionwebservice ).each do |framework|
    yield framework
  end
end

I don't think rewriting the task actually fixed anything since the problem was elsewhere, but it was a good learning exercise and I'm posting it here for your reference.

When I tried running dispatch.fcgi from the command line I kept getting this error:

undefined method `downcase' for nil:NilClass
    //vendor/rails/actionpack/lib/action_controller/request.rb:20:in `method'

I thought surely this was the reason I couldn't get Mephisto running so I tried different things to solve the problem. However, it turned out that the problem was that when running from the command line one of the environment parameters doesn't get set. If you run the command like this everything works: REQUEST_METHOD=GET public/dispatch.fcgi.

After dispatch.fcgi was happily returning a response I decided that the problem must be somewhere between Apache and fcgi so I took a look at the Apache error logs. Sure enough, there were a bunch of errors about the permissions on the /public directory being too permissive. Chmod'ing to 755 fixed everything and Mephisto finally came up.

I'm very happy with the current setup. I'm able to develop locally using Locomotive/TextMate/Terminal and deploy the latest code just by running 'cap deploy'. Many thanks to TheBitGuru and fluct.isono.us for their very helpful articles. I'll be happy to answer any questions I can about getting things up and running on Site5.

Last night and tonight I've been playing around with my MacBook and getting it configured the way I want. This mostly involves learning how things work on OS X and finding a few pieces of software to compliment what comes with OS X.

Last night my goal was to install everything necessary for Ruby-on-Rails development. After some googling I was able to find this link, which does an excellent job of detailing how to install/set-up a RoR dev environment. I opted to install LightTPD 1.4.10 which caused some problems until I found a bug report which detailed a work-around. If you're like me and prefer a GUI for MySQL you can download MySQL Administrator, which runs fine using Rosetta.

After the RoR environment was set up I pulled down the latest Typo code and set it up locally so I can test themes and/or take a crack at developing my own. Looking at the Typo code seems like it would also be a good way to learn some more about Rails.

One of the things I've noticed is that Mac people still pay for software. On the PC, whether it's Windows or Linux, for the most part it's easy to find open-source/free tools (notepad2, SmartFTP, RSS Bandit, Reflector, etc, come to mind) while on the Mac most of the tools people recommend are shareware. So, after playing around with TextMate for a little bit I bought a license for it. I think one of the reasons that people pay for software in the OS X world is that the prices are more reasonable. I'll gladly pay ~$45 for TextMate when the alternative is $284 for SlickEdit.

Because if itself is an expression, you can get really obscure with statements such as

if artist == "John Coltrane"
  artist = "'Trane"
end unless use_nicknames == "no"

This path leads to the gates of madness.