As an intern here at Boost I have had the privilege of working with a group of experienced and friendly developers. I have been here two weeks now and already feel settled in completely. One of the great things about working for Boost is that everybody is very approachable when you need to ask for help or when you need things to be elaborated. This has made the transition from university to industry easier than I initially expected.
Here at Boost we use the Agile software development methodology known as Scrum. So every day I have a stand up meeting with my scrum master (Jacob Creech) and product owner (Nathan Donaldson) where I go over what I have done, what I will be working on today, and discuss problems that I need to get out of my way. So far at Boost I have been working on bug fixes for our online usability testing tool IntuitionHQ, which is built in Ruby on Rails. Working in this way has helped me get my head around the large code base that I will gradually move towards developing features for.
The IntuitionHQ Scrum Board
Using Scrum means that I’m working in ‘sprints’, or two-week long development periods. Each bug in IntuitionHQ is written up as a user story. At the start of my first sprint I took part in a sprint planning meeting which included sizing the stories (assigning them points between 1 and 8 to indicate their complexity/the effort required to fix them) and then breaking the stories down into tasks. I had to indicate how many of the stories I could commit to in the two-week sprint, and how confident I was that I could complete them in that time frame.
As an intern I was not really sure how much I could complete, being new to Rails and working in the industry in general. But one of the cool things about Scrum is that at the end of each sprint you have a sprint retrospective. This is a chance to talk about how the sprint went, and what can be done to improve things. If I don’t complete all my stories in the first sprint, the next sprint will be adapted to deal with this, and so on and so forth. So in the case that I didn’t complete all my stories it will be known for the next sprint to take on less stories in relation to their size. Overall, this is about figuring out your ‘velocity’ – how many story points you can get done in a sprint.
For my first sprint I initially committed to seven stories. I ended up completing the stories early and brought in two more stories from the backlog. One of the fun things I have found about fixing bugs is it really stimulates the mind’s problem solving abilities. I developed most of my problem solving abilities while doing my Software Engineering degree at Victoria University. Although university is a great learning environment I have found that learning in a working environment has more merits. This is because you are working with a group of people towards a common goal so they are more likely to help you out and I find that social learning is the best way to learn. This differs from university as everybody in your papers are competing to get higher grades than you so they generally don’t feed you all the facts.
One of the cool things about coming out of university into a working environment is that once you get home from work it’s your time, not stressing-about-assignments time. I believe that the less stress you have weighing on your mind the more productive you can be. Not being stressed out has helped me to be more productive working here at Boost and has got me highly motivated and keen for my next sprint. To sum up my experiences so far I would say that working here has been awesome!
When we release a SaaS web application, such as IntuitionHQ, it’s inevitable that there will be two parts that make it up. The main part is the application itself. The second part is the marketing site that goes with it. The marketing site includes the content, and usually a way to sign up. It normally requires some integration with the application.
We choose Radiant for our CMS when working on internal projects. The reason we like it is for it’s simplicity and power. In this post I’ll go through the different ways we’ve experimented with to integrate our SaaS applications with Radiant based websites.
It’s been a busy time here at Boost, and we have just released our new web usability testing application IntuitionHQ. I’ll be writing a post about IntuitionHQ soon, but today I’d like to talk about hosting.
When we launched SonarHQ in April we decided to host on Slicehost. There were two main reasons we went with a virtualised hosting solution. The first was that we were not sure whether we would be scaling vertically (bigger, faster servers) or horizontally (splitting different functions onto different servers), and the second was the we wanted to be able to scale up and down in a fine-grained way.
Our initial approach with SonarHQ was to have 2 applications servers, 2 database servers and a utility server (for background tasks including mail). This approach gave us some redundancy and the ability to scale in either horizontally or vertically as needed. This has worked well, but we didn’t feel that the performance/price ratio is particularly good.
During our initial testing, we had IntuitionHQ setup at Slicehost in the same way. As we were preparing to launch, Engine Yard released Engine Yard Cloud. Built on top of the Amazon cloud infrastructure, Engine Yard Cloud provides a managed instance and configuration engine specifically optomised for hosting Ruby on Rails applications.
It was easy and painless to get IntuitionHQ up and running on Engine Yard Cloud – taking under an hour from start to finish. I don’t think it could have been any easier – everything just seems to work! We fired up a small instance and put through a quick series of load tests. It was evident even with casual clicking through the application that we were getting better response times. Working through the likely costs for hosting on Engine Yard Cloud, we found that we could use a 32bit, 5 ECU, 1.7GB RAM instance for around the same cost as our previous setup and get a useful boost in both performance and manageability.
One of the most significant benefits is the ability to scale vertically all the way to a 64bit, 20 ECU, 68GB RAM instance with a simple restart. Scaling horizontally is just as easy, and Engine Yard Cloud really takes the pain out of this.
Another important benefit has been the streamlining of our deployment processes. Engine Yard takes care of everything needed for automated deployments, and any custom deployment tasks are easily handled with Chef recipes. Deploying multiple application instances is easy and works seamlessly, with Engine Yard implementing a proxying system with failover across all instances without the need for a separate proxy instance (and single point of failure) – a significant cost saving. Running a seperate instance (or set of instances) for the databases is as easy as ticking a checkbox.
The image used for the instance is kept up to date with the latest security and reliability patches and is used each time our application is deployed. This gives you a semi-managed hosting system without any of the associated costs.
The margin over standard Amazon EC2 is reasonably high for the small instances but reduces to a 10% premium on the biggest instances. This is very reasonable and for us makes a great deal of sense.
The one thing that would make this much more affordable is the ability to use Amazon reserved instances. These give you a discounted hourly rate for the instance once a one off payment is made. If you know what you need and can commit to a year, this can effectively half the hosting costs.
We launched IntuitionHQ a week ago and have been extremely happy with the performance and utility of Engine Yard Cloud. We are looking forward to growing IntuitionHQ and are confident that Engine Yard Cloud can grow with us.
Our new product, IntuitionHQ, shows clusters of clicks on an image. To generate these clusters we made use of a gem called Hierclust. The great thing about this gem is it’s simplicity – just input the points and a minimum cluster separation, and out come the clusters.
The problem with Hierclust was the performance. With fewer than 100 points to cluster Hierclust was running too slow to do it dynamically. This was no problem, we moved the clustering program into a cronjob and stored the data in a marshalled file.
However, in testing we found that Hierclust was still too slow. Once we had over 200 points being clustered it started taking minutes to process – an unsustainable amount of time for the data we expected. The graph below shows the timings, which I believe is O(n3). We had to disable cluster processing while looking at the problem due to issues it was causing on the server.
I recently found squirrel, and I wanted to use it for a project we’re working on to simplify some complex finder statements. Squirrel allows turning something like this:
Task.find(:all,
:conditions => ['active = ? and (updated_at > cache_version or cache_version IS NULL)', true])
into:
Task.find(:all)do
active == true
any do
updated_at > cache_version
cache_version.nil?
endend
The problem is testing
Then I ran into a serious problem – how to test this piece of code using rspec? Here was my first attempt: (more…)
When writing a Rails application, how do you decide on the best indexes to add to your database? It might seem obvious, especially if you work on a project from scratch. The problem is a little harder when you come to optimize an existing codebase.
New Relic
Recently I’ve been using two methods to work out where to put indexes. Firstly I’d strongly recommend using New Relic RPM in development mode. When running your application you can visit /newrelic to get all kinds of useful information. Here you can see the most recent rails calls:
