Even Flow..

I know, it has been a long time since I've blogged and actually people have complained about the fact that I didn't do this for a long time. Well, in short, I'm fine. A lot has happened since the last time I've blogged, but I'll write about that when I make the time somewhere in the upcoming weeks. Since I've been coding a lot with Rails lately, I just wanted to share some geek stuff with the rest of the world today.

In one of my projects, I'm using the will_paginate plug-in, which is pretty cool. Without a lot of stuff, it will give you pagination in your views. It also works for generic Arrays. Anyway, there is one problem. The project I'm working with can contain a lot of records per table. Since will_paginate doesn't act like a named scope (or at least not in this case, since we're not directly calling paginate() on an AssociationProxy, but on an Array) and requires the whole set returned. This is, because it needs the size of the array as well, so it will calculate the total amount of pages and records in the collection. Obviously, this can be quite expensive if you have a lot of records in your table. It will retrieve all records, create objects in the Array and finally cut of just a very small part of it to display. Because of this, we came up with the following solution:

# The amount of objects per page we want to show
limit = $OBJECTS_PER_PAGE

# If the page was set to 0 (shouldn't happen), or nil (yeah, nil.to_i == 0),
# set the offset to 0
offset_page = (page.to_i > 0) ? page.to_i - 1 : 0
offset = limit * offset_page

# Create args hash with which we count
count_args = args.dup

# Add the limit and the offset to the arguments hash
args[:limit] = limit unless args[:limit]
args[:offset] = offset unless args[:offset]

# Count all object that would be there
total_count = objects.count(:all, count_args)

# Find all objects (this is limited)
objects = find(:all, args)

# Create an array filled with nil values, the size of the total collection
objects_array = Array.new(total_count, nil)

# Insert the found objects at the right place in the array
objects_array[(offset)..(offset + limit - 1)] = objects

# Call paginate() on the array with limit and page   
return objects_array.paginate(:per_page => limit, :page => page)

What this does is only getting the required records, using :limit and ffset in find() and an additional count() without :limit and ffset. To make sure that will_paginate gets a collection the size of the total collection, we create an Array filled with nil values, the size of count(). Then we insert the found records in this array.

The only problem is that the whole operation isn't atomic. The count could differ from the real objects array.

I haven't run any benchmarks, but the Rails logs tell me that only a very small subset of the complete table is selected.

Update: Pointed out by Habbie, using SQL OFFSET is very slow. Back to the drawing board.