Deleting all records in a Rails project

During the initial phase of development of a Rails application I don’t use migrations as migrations but as table definitions. Until I deploy I feel free to modify the migration files as much as I want and I have one per table.

The downside of that is that the only way to apply the changes is to destroy all the tables and re-build them. I’ve explained how to do that in my post really resetting the database. The nice side effect of doing this is that you end up with a task that sets sample data to work with.

Being able to quickly set up sample data or download production data is very important. It helps new developers getting started with the project but it also allows you to play much more freely with the project, do destructive actions and then in a quick command have system reset to a known state. Once you have sample data you’ll probably become as addictive as I am to reseting.

But the truth is that 90% of the time you reset your data, you don’t need to nuke the database and re-create all records, you just need to delete all records and this is the code I use to do that:

def destroy_data
  puts "==  Data: Destroying all data ".ljust(79, "=")
  sql = ActiveRecord::Base.connection()

  sql.execute "SET autocommit=0"
  sql.begin_db_transaction

  if sql.adapter_name == "MySQL"
    sql.execute("/*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */")
    sql.execute("/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */")
  end

  tables = sql.tables - ["schema_migrations"]

  tables.each do |table|
    puts "-- Deleting all for #{table}."
    # So far, disabling and enabling keys was not needed.
    #sql.execute("/*!40000 ALTER TABLE `#{table}` DISABLE KEYS */") if sql.adapter_name == "MySQL"
    record_count = sql.delete("DELETE FROM `#{table}`")
    #sql.execute("/*!40000 ALTER TABLE `#{table}` ENABLE KEYS */") if sql.adapter_name == "MySQL"
    puts "   -> done: #{record_count} reconds"
  end

  if sql.adapter_name == "MySQL"
    sql.execute("/*!40014 SET FOREIGN_KEY_CHECKS=@OLD_FOREIGN_KEY_CHECKS */")
    sql.execute("/*!40014 SET UNIQUE_CHECKS=@OLD_UNIQUE_CHECKS */")
  end

  sql.commit_db_transaction

  puts "==  Data: Destroying all data (done) ".ljust(79, "=") + "\n\n"
end

Note that I’m not deleting anything in the table schema_migration. The output is more or less like this:

==  Data: Destroying all data =================================================
-- Deleting all for blogs.
   -> done: 4 reconds
-- Deleting all for posts.
   -> done: 10 reconds
-- Deleting all for users.
   -> done: 11 reconds
==  Data: Destroying all data (done) ==========================================

I also have some nice code to generate sample data, but that is for another post.

Advertisements

undefined method `authenticate?'

If you are getting this error:

ActionView::Template::Error: undefined method `authenticate?' for nil:NilClass

in your call to Devise’s user_signed_in? or similar, you probably forgot to add this:

class ActionController::TestCase
  include Devise::TestHelpers
end

at the bottom of the test_helper.rb file. Not that that would ever happen to me…

I suddenly love my… ISP

People, including me, tend to make a big deal of bad stuff and not of good stuff, so I sometimes try to do the oposite. Recently my internet connection was slow, very slow. Several weeks passed and I’ve got really angry so I did complaint, expecting to be ignored. I wasn’t ignored, I’ve got an awesome reply:

Dear Mr. Fernandez

Thank you for contacting us.

I’ve checked your connection for factors that might lead to the service impairment you complain about and this is what I’ve found so far:

– your cablemodem passed all our tests, it shows no sign of wear and should therefore work perfectly
– the signal quality of your connection is very good
– our network has enough capacity to deliver the data rates of your subscription plan. In the evening peak hours there might be a reduction in bandwidth (Best Effort)

This was all I expected as an answer. Just them telling me there’s no problem. But they went on:

According to the information I have so far you might be using a Netgear router between the modem and your PC. Please keep in mind that a router may reduce your bandwith depending on several factors.

That’s true, I have a crappy cheap consumer cheap switch/router and they know it. Is their modem informing them? I don’t see another way for them to know. Interesting.

Please attach your PC directly to the cablemodem and perform a few reference tests on http://hsi.cablecom.ch . Should the results show a reduced bandwidth please send us these results by clicking on the “trouble ticket” link on that page and fill out the form.

Oh! They have their own measurement tool. Of course it measures only their own connection, so if my access to them is fast, but their access to the internet is slow, it’ll say my access is fast while for practical porpoises it’s going to be slow. It’s still good that we can find where the problem is.

Please also make sure that no other programs using the internet is running when performing these tests. The statistics of your connection show a very high upstream : downstream ratio – you seem to be sending up to twice as much data as you are receiving (your utilization of thelast four weeks: 190 GB sent, 85 GB received).

I’ve uploaded 190GB? Holly crap! I didn’t know. Of course an ISP have that data, but the fact that technical support has access to it and that they use it to tell you what’s going on it’s amazing for me.

This might be due to a badly tuned peer2peer application (emule, kazaa, bittorrent, etc) or due to some server application running in the background. To get the full Downstream bandwidth you will need to have at least 500kbit/s upstream bandwidth left. Most peer2peer applications allow you to limit the upstream – we recommend to set this threshold to 2000kbit/s with your current subscription plan.

During the tests any peer2peer activity must be disabled (we recommend to have them switched off for at least 5 minutes before you start the test).

This is where they went way beyond their duty. They’ve explained me how to configure my bittorrent client. If I’m using my connection wrongly it’s not their problem, except that it is, because even if it’s due to my ignorance, I’m perceiving their product as bad. This kind of mentality, “if the user believes we are bad we are bad” is what makes companies like Google and Apple so great and successful.

And actually, this part of the answer probably helped me find the issue, which was not bittorent: backups. I have two programs constantly backing up my machine, constantly uploading everything to various servers on the web. One of them tends to misbehave and use as much bandwidth and processor as there is, leaving nothing to the puny humans.

Once we have received your test results we can continue the analysis on the issue.

Yours sincerely,

Markus *******
Customer Service

That was awesome! Thank you Cablecom.

Better assert difference?

Rails come with some awesome assertion methods for writing tests:

assert_difference("User.count", +1) do
  create_a_user
end

That asserts that the count of user was incremented by one. The plus sign is not needed, that’s just an integer, I add it to make things clear. You can mix several of this expressions into one assert_difference:

assert_difference(["User.count", "Profile.count"], +1) do
  create_a_user
end

That works as expected, it asserts that both users and profiles were incremented by one. The problem I have is that I often found myself doing this:

assert_difference "User.count", +1 do
  assert_difference "Admin.count", 0 do
    assert_difference "Message.count", +3 do  # We send three welcome messages to each user, like Gmail.
      create_a_user
    end
  end
end

That looks ugly. Let’s try something different:

assert_difference("User.count" => +1, "Admin.count" => 0, "Message.count" => +3) do
  create_a_user
end

Well, that looks nicer, and straightforward, so I implemented it (starting from Rails 3 assert_difference):

def assert_difference(expressions, difference = 1, message = nil, &block)
  b = block.send(:binding)
  if !expressions.is_a? Hash
    exps = Array.wrap(expressions)
    expressions = {}
    exps.each { |e| expressions[e] = difference }
  end

  before = {}
  expressions.each {|exp, _| before[exp] = eval(exp, b)}

  yield

  expressions.each do |exp, diff|
    error = "#{exp.inspect} didn't change by #{diff}"
    error = "#{message}.\n#{error}" if message
    assert_equal(before[exp] + diff, eval(exp, b), error)
  end
end

Do you like it? If you do, let me know and I might turn this into a patch for Rails 3 (and then let them now, otherwise they’ll ignore it).

Update: this is now a gem.

Redirect to SSL in Rails applications

I’ve looked at the various ssl_requirement repositories out there. I concluded the most modern and maintained version is yardstick’s which is released as a gem called sslrequirement, but I’ve failed to use it properly. So I just did it by hand.

First, we need a simple method that will let us know whether SSL is enabled or not. We don’t want to redirect to SSL in development mode because it’ll fail. In the application controller I’ve created:

  def ssl_enabled?
    !(Rails.env.development? || Rails.env.test?)
  end

Switching to SSL is not only a matter of redirecting. If you show a login or signup form in your homepage, like I do in Restraq, you want that to point to https even if the page was loaded as http. So I’ve added this helper method in the application controller:

  def https
    ssl_enabled? ? "https://" : "http://"
  end
  helper_method :https

and then for the forms I just do this:

form_for ..., :url => session_url(resource_name, :protocol => https)

and

form_for ..., :url => registration_url(resource_name, :protocol => https)

And then the redirection part, which is a before filter in the application controller because I want to redirect when hitting Devise controllers:

  def enforce_ssl_if_needed
    if request.protocol == "http://" && ssl_enabled? &&
            (controller_name == "registrations" || controller_name == "sessions")
      redirect_to :protocol => https
    end
    return true
  end

and that’s it. I’m not actually testing it yet. For a similar solution with tests you might want to check out SSLShopper’s article about this.

My hypothesis of music appreciation

I believe most music we like we like by association. What I mean by that is that we don’t like the music because of the music itself but because of what we associate to it. We probably do that with most stuff but if you think about it, shouldn’t music taste vary much more from person to person, and yet it seems most generations like the same stuff and continue to like it for the rest of their lives.

My hypothesis is that they are young, they are enjoying life with friends, and whatever music you make them listen to they’ll like by association of having a good time. Now, when I was a teenager I didn’t tend to have a good time with other people, thus I more or less hate the popular music of that era. What I’ve really loved was watching movies.

Soundtracks excite me, soundtracks take me to another world, I love listening to soundtracks, specially as background music. I can close my eyes to Indiana Jones and the last crusade and see the movie. When I was kid that was the movie. I can put it on and feel that every line of code I type gets me closer to the grail, every test that passes is a dead nazi. Great music, great movie!

I’ve just remember this because I’m listening a lot to Inception lately and, this you’ll only understand if you watched the movie, but every time I reach the point where Edith Piaf sings (in Waiting for a Train) I almost say “Oh fuck! Oh fuck oh fuck!”.

Cron jobs for web applications

I always dislike setting up cron jobs for web applications. The whole application lives in /var/www but I manually create this single line somewhere else that I have to remember to setup if I switch servers, and turn off if needed, and maintain accordingly. Well, it so happens that it can be done much better with the tools we already have: the usual cron and capistrano (or any other similar release tool).

In Restraq, a project I’m working on, I created a file named crontab in the config directory with these contents:

MAILTO=pupeno@restraq.com
*/5 * * * * www-data cd /var/www/restraq/current && RAILS_ENV=production rake mail:get

Cron can read from some predefine files but it also reads every file in /etc/cron.d/. That’s a very common Linux pattern, the files in the directory with the .d extension are read as if they were one big configuration file. Other programs do it too.

Then it’s only a matter of installing it there on deployment. At first I’ve tried with a symlink but it seems cron ignores symlinks so I decided to copy it with this line on my deploy.rb file:

run "#{try_sudo} cp #{current_path}/config/crontab /etc/cron.d/restraq"

and that’s it. No more fragmentation. Whenever, wherever I install my web app, there goes the cron jobs.