home

Boost Blog

Posts Tagged ‘ruby’

July 5th, 2010

Working with Git

Posted by jeremy on July 5th, 2010

During a brief slow period on a Friday afternoon I started pondering how much work I actually do, and if it was even useful knowing. Obviously all our code is stored in a version control system (git), so in a way all of the data for finding out the quantity of work is readily available. A little investigation and I found that it’s quite easy to pull a list of commits from git showing total lines added and deleted per file:

git log --oneline --numstat

I’ve committed a lot of code that I didn’t write, such as plugins, the Rails framework etc. So a quick and dirty ruby script later I could get a list of all unique files in all repositories that I’ve committed to. It was pretty easy to go through the list and create an exclusion list. I then broke out Ruport to aggregate everything by extension. That gave me the following table:

I’ve cleaned this up a little and collapsed some alternative extensions down.

Commits per week

Just over 110,000 lines added and 50,000 deleted, of which about 100,000 are to Ruby files. Now I’m not claiming to have written all those lines myself, any part of any line changed counts towards the total. All this does is illustrate the general balance of work that I do. There have been two lines added for every line deleted. This year has seen a lot of refactoring work, so it’ll be interesting to run the same exercise next year and see if the results are similar (of course git holds historical data, but we only started using it about 18 months ago, and previously had everything stored in subversion).

It’s interesting to see that the proportion of additons to deletions is much higher in view (rhtml/haml) files than in ruby code. This could point to the way things look being changed much more than the way things work.

Now if only there was a way to measure the quality of work. (Actually there are tools; metric_fu is a good starting point and we use it a lot at Boost. However, that’s going a little too far for this post).

Another interesting bit of data I extracted from git is the number of commits I’ve done per week over the last 52 weeks.

I’ve posted my script as a github gist. You  can run it by modifying the @repositories array with a list of git repositories, @author with your email address and @excludes with a list of regular expressions for excluding files. Run the script as ruby gitcount.rb. If it is run with the argument “files” then it will list individual files, making it easier to build the exclude list.

 
Tags: git, ruby
Posted in: Development, Random thoughts
No Comments
 
April 15th, 2010

Custom application and CMS integration

Posted by jeremy on April 15th, 2010

When we release a SaaS web application, such as IntuitionHQ, it’s inevitable that there will be two parts that make it up. The main part is the application itself. The second part is the marketing site that goes with it. The marketing site includes the content, and usually a way to sign up. It normally requires some integration with the application.

We choose Radiant for our CMS when working on internal projects. The reason we like it is for it’s simplicity and power. In this post I’ll go through the different ways we’ve experimented with to integrate our SaaS applications with Radiant based websites.

more »

 
Tags: radiant, rails, ruby
Posted in: Ruby on Rails
2 Comments
 
October 29th, 2009

Hosting in the cloud

Posted by Nathan on October 29th, 2009

It’s been a busy time here at Boost, and we have just released our new web usability testing application IntuitionHQ. I’ll be writing a post about IntuitionHQ soon, but today I’d like to talk about hosting.

When we launched SonarHQ in April we decided to host on Slicehost. There were two main reasons we went with a virtualised hosting solution. The first was that we were not sure whether we would be scaling vertically (bigger, faster servers) or horizontally (splitting different functions onto different servers), and the second was the we wanted to be able to scale up and down in a fine-grained way.

Our initial approach with SonarHQ was to have 2 applications servers, 2 database servers and a utility server (for background tasks including mail). This approach gave us some redundancy and the ability to scale in either horizontally or vertically as needed. This has worked well, but we didn’t feel that the performance/price ratio is particularly good.

During our initial testing, we had IntuitionHQ setup at Slicehost in the same way. As we were preparing to launch, Engine Yard released Engine Yard Cloud. Built on top of the Amazon cloud infrastructure, Engine Yard Cloud provides a managed instance and configuration engine specifically optomised for hosting Ruby on Rails applications.

It was easy and painless to get IntuitionHQ up and running on Engine Yard Cloud – taking under an hour from start to finish. I don’t think it could have been any easier – everything just seems to work! We fired up a small instance and put through a quick series of load tests. It was evident even with casual clicking through the application that we were getting better response times. Working through the likely costs for hosting on Engine Yard Cloud, we found that we could use a 32bit, 5 ECU, 1.7GB RAM  instance for around the same cost as our previous setup and get a useful boost in both performance and manageability.

One of the most significant benefits is the ability to scale vertically all the way to a 64bit, 20 ECU, 68GB RAM instance with a simple restart. Scaling horizontally is just as easy, and Engine Yard Cloud really takes the pain out of this.

Another important benefit has been the streamlining of our deployment processes. Engine Yard takes care of everything needed for automated deployments, and any custom deployment tasks are easily handled with Chef recipes. Deploying multiple application instances is easy and works seamlessly, with Engine Yard implementing a proxying system with failover across all instances without the need for a separate proxy instance (and single point of failure) – a significant cost saving. Running a seperate instance (or set of instances) for the databases is as easy as ticking a checkbox.

The image used for the instance is kept up to date with the latest security and reliability patches and is used each time our application is deployed. This gives you a semi-managed hosting system without any of the associated costs.

The margin over standard Amazon EC2 is reasonably high for the small instances but reduces to a 10% premium on the biggest instances. This is very reasonable and for us makes a great deal of sense.

The one thing that would make this much more affordable is the ability to use Amazon reserved instances. These give you a discounted hourly rate for the instance once a one off payment is made. If you know what you need and can commit to a year, this can effectively half the hosting costs.

We launched IntuitionHQ a week ago and have been extremely happy with the performance and utility of Engine Yard Cloud. We are looking forward to growing IntuitionHQ and are confident that Engine Yard Cloud can grow with us.

 
Tags: Deployment, Development, Hosting, rails, ruby
Posted in: Ruby on Rails
No Comments
 
October 27th, 2009

How the west was clustered

Posted by jeremy on October 27th, 2009
Screen shot 2009-10-27 at 9.20.25 AM

A cluster on IntuitionHQ

Our new product, IntuitionHQ, shows clusters of clicks on an image. To generate these clusters we made use of a gem called Hierclust. The great thing about this gem is it’s simplicity – just input the points and a minimum cluster separation, and out come the clusters.

The problem with Hierclust was the performance. With fewer than 100 points to cluster Hierclust was running too slow to do it dynamically. This was no problem, we moved the clustering program into a cronjob and stored the data in a marshalled file.

However, in testing we found that Hierclust was still too slow. Once we had over 200 points being clustered it started taking minutes to process – an unsustainable amount of time for the data we expected. The graph below shows the timings, which I believe is O(n3). We had to disable cluster processing while looking at the problem due to issues it was causing on the server.

Screen shot 2009-10-27 at 10.14.36 AM

Graph of points v time taken

more »

 
Tags: rails, ruby
Posted in: Development
1 Comment
 
September 15th, 2009

What should break the test?

Posted by jeremy on September 15th, 2009

I recently came across FakeFS, which sounds like a great idea for helping to test file functions. However, I did pick up on the following off the website:

[...] it means replacing our mkdir call with a call to mkdir_p won’t break our tests. Because, really, it shouldn’t.

Now I understand that in the context of the example given, mkdir and mkdir_p should do the same thing. But mkdir and mkdir_p do perform different operations, and the tests should break if mkdir_p is changed to mkdir. If you change mkdir_p to mkdir and your passing tests don’t fail then your tests aren’t correct.

 
Tags: ruby, tdd
Posted in: Development
No Comments
 
September 11th, 2009

Squirrel and the custom nut matcher

Posted by jeremy on September 11th, 2009

I recently found squirrel, and I wanted to use it for a project we’re working on to simplify some complex finder statements. Squirrel allows turning something like this:

Task.find(:all,
  :conditions => [
    'active = ? and (updated_at > cache_version or cache_version IS NULL)', true
  ]
)

into:

Task.find(:all) do
  active == true
  any do
    updated_at > cache_version
    cache_version.nil?
  end
end

The problem is testing

Then I ran into a serious problem – how to test this piece of code using rspec? Here was my first attempt: more »

 
Tags: rails, rspec, ruby, tdd
Posted in: Development, Ruby on Rails
2 Comments
 
July 24th, 2009

DigitalNZ client library for ruby

Posted by jeremy on July 24th, 2009

We’ve just released a new client library for accessing the digitalnz.org’s search API. This library uses nokogiri for speedily parsing the XML, making it faster than an existing dnz client library using JSON parsing.

Installation

sudo gem install dnz-client

Or for the bleeding edge (if there is one):

sudo gem install boost-dnz-client

You’ll need to have an API key from digitalnz.org
more »

 
Tags: ruby
Posted in: Ruby on Rails
2 Comments
 
June 22nd, 2009

Writing tests/specs for existing code

Posted by jeremy on June 22nd, 2009

On several occasions we’ve looked at Rails projects with no tests, or very low test coverage. Trying to write tests when you didn’t write the original methods is difficult. Here are some tips that we’ve found useful.

Use your favourite framework.

If there is a large body of existing tests using a particular framework, write your new tests in that framework. If there are few or no tests then strip out anything already there and replace it with your preferred framework. At Boost we’ve decided to use RSpec for all our projects. Your objective here is to write tests, not to learn a new way of writing tests. more »

 
Tags: rspec, ruby, tdd, testing
Posted in: Ruby on Rails
No Comments
 
April 29th, 2009

Extracting Active Directory SIDs with Ruby

Posted by jeremy on April 29th, 2009

Getting user and group information from Active Directory via LDAP is nearly easy, but I came across one hurdle. If you retrieve a user, you might get something like this:

{
  :smaccountname => ['jeremy'],
  :dn => ‘["CN=Jeremy Wells,CN=Users,DC=domain,DC=boost,DC=co,DC=nz"],
  :primarygroupid => ['1114'],
  :objectsid => ["010500000000000525000000\210B ?\23302r\255ոT040000"]
}

If you then wanted to get the users primary group, you’d need to search using that primary group id. But you’d find there is no corresponding id on the group object, which looks as thus:

{
  :samaccountname=>["Students"],
  :dn=>["CN=Students,OU=Groups,DC=domain,DC=boost,DC=co,DC=nz"],
  :objectsid=>["010500000000000525000000\210B ?\23302r\255ոZ040000"],
  :name=>["Students"]
}

But notice the binary field :objectsid. This is the binary form of the string you may see sometimes when using AD, called SID, and it looks something like “S-1-5-21-123-456-789″. In order to find the users group you would take the :primarygroupid and the users :objectsid to generate the groups SID.

Space Intruder Detector (SID)

Space Intruder Detector (SID) UFO © ITV Global Entertainment

more »

 
Tags: active directory, ruby, windows
Posted in: Ruby on Rails
No Comments
 
  • Categories

    • Agile (23)
    • Agile Coaching (1)
    • Business (3)
    • Cool tools (5)
    • Design (7)
    • Development (17)
    • Drupal (1)
    • e-Learning (70)
    • Magic & Delight (6)
    • Publishing (3)
    • Random thoughts (7)
    • Ruby on Rails (9)
    • Scrum (9)
    • Social media (9)
    • Usabilty (4)
    • Writing (1)
  • Archives

    • January 2012 (3)
    • November 2011 (4)
    • August 2011 (5)
    • July 2011 (1)
    • June 2011 (2)
    • May 2011 (4)
    • April 2011 (1)
    • March 2011 (1)
    • February 2011 (1)
    • November 2010 (1)
    • October 2010 (1)
    • September 2010 (3)
    • August 2010 (4)
    • July 2010 (6)
    • June 2010 (2)
    • April 2010 (1)
    • March 2010 (1)
    • February 2010 (1)
    • January 2010 (3)
    • December 2009 (1)
    • November 2009 (1)
    • October 2009 (4)
    • September 2009 (2)
    • August 2009 (3)
    • July 2009 (6)
    • June 2009 (3)
    • May 2009 (1)
    • April 2009 (6)
    • March 2009 (6)
    • February 2009 (11)
    • December 2008 (4)
    • November 2008 (6)
    • October 2008 (12)
    • September 2008 (7)
    • August 2008 (7)
    • July 2008 (4)
  • Boost Loves Design

    • I love Typography
    • IntuitionHQ | easy website usability
    • OMG It even has a watermark
  • Follow me on Twitter
© Boost Limited.
All rights reserved.
CONTACT US
info@boost.co.nz
tel. (04) 939 0062
fax. (04) 939 0063

Level 6, 175 Victoria Street
PO Box 11504, Wellington
New Zealand