Tech Blog

Some time ago, I wrote about putting HTTP status code to use for your Rails application. For my reinvigorated project, I wanted to apply the same technique. Instead of re-implementing it once again, I created a Rails plugin called http_status_exceptions to easily add this functionality and I have put it on Github. For more information on how to install and use the plugin, see the project’s wiki.

Last week we had to convert our existing Rails application, which uses Gettext to the new I18n API in combination with the SimpleBackend. I personally never liked Gettext, there was simply not enough control over translations as PO/MO files are not native ruby or at least can be easily accessed by Ruby (like YAML files).

We therefore decided to switch to the brand new, not even released, I18n API. But now we had a serious problem, our code base isn’t small and all that code had to be converted in some way. We could do it by hand, but hey, that’s a lot of work and especially very error-prone. Some convertor had to be written. Here it is as a rails plugin: GettextToI18n. 

What does the I18n convertor do?

It scrapes your whole application and searches for gettext calls, like this:

_("to be translated")

It will convert this gettext call to the newly I18n format: 

I18n.t :message_id

Then it builds up a big hash containing all the the translations. We decided it was handy to use the scopes that are introduced in the new I18n api. So it stores the translations in the following format: 

For models:

["model"]["model_name"]={:message_1 => "to be translated"}

For controller:

["controller"]["controller_name"]={:message_1 => "to be translated"}

After this hash of translations has been built up, the convertor writes it as a YAML file to:  config/locales/template.yml.That’s all!

What’s supported?

It supports basic gettext calls. We have run it over our code base and it converts all gettext calls we use without any problem. 

A normal gettext call

_("to be translated")

converts to:

I18n.t :message_0, :scope => [:txt, :controller, :controller_name])

A gettext call with variables

_("My name is %{name}" % {:name => "Jaap"})

converts to:

I18n.t :message_0, :name => "Jaap", :scope => [:txt, :controller, :controller_name])

A gettext call with variables that contain gettext calls

_("Click %{link} to go to the homepage" % {:link => link_to(_("Here"), root_path)})

converts to:

I18n.t :message_0, :link => link_to(I18n.t(:message_1, :scope => [:txt, :controller, :controller_name]), root_path), :scope => [:txt, :controller, :controller_name])

Installation

./script/plugin install git://github.com/japetheape/gettext_to_i18n.git

Usage

To convert your application:
rake gettext_to_i18n:transform

Please make sure you backup your complete application as it can screw things up.

Contribution

Please contribute to this plugin and make it better, as I won’t use it anymore, cause we are not going to convert another time (I think ;-)). Things that has to be done are:

unnamed variables:

_("I play the %s" % "saxophone")

Go to the development location at github and fork this plugin!

My friend Bart from movesonrails.com just blogged about Rails log analyzer, a command line tool to get performance statistics for your Rails application by parsing its log file.

What started as an exercise for me to write a command line ruby program, has been extended and improved by Bart to be actually useful! We decided to release it under an MIT license. You can found the source on github. The project’s wiki contains usage information and an example of the output it will produce.

Remember my post about easy OLAP queries in Rails? I rewrote it almost completely and published is as a Rails plugin for anyone to use on github! It is now called: Active OLAP.

Although it is a complete rewrite, the API I demoed in my previous post should still work with some small changes. The most important: you have to enable it for every class you want to use it on with the enable_active_olap method. You can provide a block to this method with dimension definitions, but is not mandatory:

class User < ActiveRecord::Base
 
  enable_active_olap do |olap|
 
    # create a simple dimension on the account_type field
    olap.dimension :account_type
 
    # create a dimension with custom categories
    # the order of the categories will be kept in the results 
    # if you use an array to define the categories.
    olap.dimension :nationality, :categories => [
      [:usa, { :country => 'US' }],
      [:china, { :country => 'CN' }]
      # other is automatically added
    ]
 
    # Easily create a trend dimension
    olap.dimension :created_daily, :trend => {
      :timestamp_field => :created_at,
      :period_length => 1.day, 
      :period_count => 20
    }
  end
end

Now, we can use these dimensions for our OLAP queries. Multiple dimensions are supported too!

# simple query
@result = User.olap_query(:nationality)
# @result[:usa] == 123, @result[:china] == 456, @result[:other] = 789
 
# do drilldown using will_paginate to paginate the results
# olap_drilldown is implemented as a named_scope
@users = User.olap_drilldown(:nationality => :china).paginate(:page => 1)
 
# multiple dimensions!
@result = User.olap_query(:nationality, :created_daily)
@users = User.olap_drilldown(:nationality => :china, 
                        :created_daily => :period_19)

I am working on a generic controller that can easily be added to your Rails project. Just define dimensions for your models and the controller will let you execute OLAP queries and display the results as a table or a graph.

Keep an eye on this weblog or the github project if you want to stay up-to-date! Or, contact me if you have questions, suggestions or want to help out.

A couple of minutes ago I released scoped_search, a Rails/ActiveRecord plugin that makes it easy to search your models. It is very easy to use:

1. Install the plugin in your vendor/plugins directory from http://github.com/wvanbergen/scoped_search
   Add the gem to your rails environment.rb:

   config.gem 'wvanbergen-scoped_search', :lib => 'scoped_search', 
       :source => 'http://gems.github.com'

   Call rake gems:install afterwards to ensure the gem is installed.

2. Define in what fields your model should be searched by calling
   searchable_on :some, :field, :names
3. Find your records by calling search_for("query keywords")

That’s all! A short example:

class Project < ActiveRecord::Base
  searchable_on :name, :description
end
 
Project.search_for("search keywords").each do |project| 
  puts project.name
end
 
# SELECT * FROM projects WHERE 
#      (name LIKE '%search%' OR description LIKE '%search%') 
#  AND (name LIKE '%keywords%' OR description LIKE '%keywords%')

This functionality is completely build upon named_scope. The search_for method is actually a named scope that was created by the call to searchable_on. Because these scopes can be chained, this offers some great possibilities.

For example, in Floorplanner, we only want you to search on the projects you have access to. We have implemented this access logic in another named scope. The calls can simply be chained:

class Project < ActiveRecord::Base
  searchable_on :name, :description
 
  named_scope :accessible_by, lambda { |user| ... }
  named_scope :published, :conditions => 'published_at IS NOT NULL'
end
 
@projects = Project.accessible_by(current_user).published.search_for('query')
@projects.each { |project| ... }

This plugin is released under the MIT license, so please use it for any purpose you see fit. There are some TODOs: you currently can not search on fields in other tables, and splitting the search string into keywords is very basic. Please contact me if you have implemented any of these features and you are willing to share them! Do not hesitate to contact me in case or problems either.

Update: I added support for quotes and the minus sign to the query language:
Project.search_for('willem -"van bergen"').count

Update #2: Wes Hays implemented the OR keyword:
Project.search_for('wes OR hays').count.
A big thanks to Wes for helping out on this project!!

Today I setup the first version of a RAPIDoc, a Rest API Rails Documentation Generator and we decided to open source this thing, from now on let’s call it RAPIDoc. It is a API code generator for Rails, describing your Rest resources. We we’re looking for a way to fully integrate documentation of our API into the code base, like Rdoc does. Rdoc didn’t suit our needs, cause it has got nothing to do with Rest and resource stuff, so we decided to hack something together and gave it a name: RAPIDoc.

What does it do?
It generates a API controller containing documenation maked up in a special language. It parses controllers you specify, and generates a ApiController with appropriate views for it. This makes it very easy to document a Rest API. For methods you use, and according to ERB templates you specify it generates the views for you. It doesn’t parse your routes.rb, it was not needed for us, but may be a nice extension for it.

Well how is that RAPI doc looking? Put this in front of a Restful method.

1
2
3
4
5
6
7
8
9
10
11

=begin rapidoc
url:: /projects
method:: GET
access:: FREE
return:: [JSON|XML] - some project
param:: page:int - the page
param:: per_page:int - max items per page
 
Get a list of projects. This method uses pagination. If you want to retreive project 1-10 for example:
/projects?page=1&per_page=10.
=end

For each resource you specify, a view is created and it is put into the index.

Opensource
It is available from now on on:

http://code.google.com/p/rapidoc/

If you find it useful and you want to change some things, become a submitter. Ow yeah, just a note: really experimental code

Today, I have added a server switch to Floorplanner.com. Now, it will load the floorplanner elements from the server that is nearest to you, which can yield a significant improvement in the initial loading time of your floorplans.

To determine your location, your IP address is matched against a table of locations. This worked fine in our development version, but it didn’t work at all on the production server. After some searching, I found that our server configuration was causing this. We use Apache as our web server, which uses mod_proxy to send the request to our Mongrel cluster. This intermediary step caused the IP address that Rails would receive to always be the IP address of the Apache server: 127.0.0.1. Therefore, the location matching did not work.

However, I found that mod_proxy adds an additional header to the request with the original IP address: HTTP_X_FORWARDED_FOR. This header can be used for our purpose. Now, I use the following function to determine the correct IP address:

1
2
3
4
5

def determine_ip(request)
  # use HTTP_X_FORWARDED_FOR if available
  # otherwise fall back to default header
  request.env["HTTP_X_FORWARDED_FOR"] || request.remote_addr
end

On a related note: to match an IP address against ranges of IP addresses in our location table, it must be converted from a string (”1.2.3.4″) to a number (16909060). I use the following oneliner, which uses some nice functional programming tricks and an application of bit-shifting:

1
2
3
4

def numeric_ip(ip_str)
  ip_str.split('.').inject(0) { |ip_num, part|
            ( ip_num << 8 ) + part.to_i }
end

Yes, I am really proud if this function!

UPDATE: I just found out that request.remote_ip does the same as my determine_ip-function. Unfortunately, it only works in Rails 2.0.

We are currently implementing an API for Floorplanner, so other sites can use the service Floorplanner offers for their own needs. The Floorplanner website is developed with Rails, so we are trying to be a good Rails citizen and create a API based on REST.

REST embraces the HTTP protocol by coupling URIs to resources and HTTP methods GET, PUT, POST and DELETE) to actions for manipulating them. Today, I tried to embrace the HTTP protocol even more by using its various status codes for Floorplanner’s particular needs.

Most people will know the HTTP 404 status code, which a server returns if you request a page that does not exist. But there are many more interesting codes that can be put to good use as well. The Forbidden status code (403) can for instance be returned if you try to access a another user’s floorplan. Moreover, Floorplanner offers paid subscriptions that include additional functionality. If you try to access this functionality with an account without the necessary privileges, an Upgrade Required status code (426) can be returned. And if you forgot to pay your subscription fee, a Payment Required code (402) can be returned to indicate this.

However, in the end, users wants to see a nice and informative page that tells them what is wrong without cryptic error codes. This is easily possible by leveraging some new features in Rails 2. First of all, we define some custom exceptions that can be raised if some of these conditions occur is wrong:

class PermissionDenied < Exception
  # no further implementation necessary
end
 
class AccountExpired < Exception
  # no further implementation necessary
end

Now, we can raise these exceptions when needed in our controllers:

def show
  @plan = Floorplan.find(params[:id])
  raise PermissionDenied unless @plan.user == current_user
  # ...
end
 
def login
  # ...
  raise AccountExpired if Time.now > current_user.account_paid_until
  # ...
end

Normally, raised exceptions will trigger an Internal server error (HTTP status code 500). The new exceptions will be handled manually to return the intended status code and serve a pretty page explaining what is wrong. Exceptions can be caught using the rescue_from method. If we put the code in the Application controller, it will automagically work for all our controllers. DRY.

class ApplicationController < ActionController::Base
 
  rescue_from PermissionDenied { |e| http_status_code(:forbidden, e) }
  rescue_from AccountExpired   { |e| http_status_code(:payment_required, e) }
 
  # Returns a HTTP status code, with a nice error page
  def http_status_code(status, exception)
    # store the exception so its message can be used in the view
    @exception = exception
 
    # Only add the error page to the status code if the reuqest-format was HTML
    respond_to do |format|
      format.html { render :template => "shared/status_#{status.to_s}", :status => status }
      format.any  { head status } # only return the status code
    end
  end
 
end

Now, it is easy to create super fancy pages by editing the views file like app/views/shared/status_forbidden.html.erb.

Note that you cannot and should not use this method for handling internal server errors with status code 500. These should not occur because we have thought of every possible way our code will be used and misused… in theory. In practice, we installed the exception notifier plugin, so we receive a message if one of these occurs and get our asses back to work.