Ruby Date & DateTime class extension

My recent user story demanded me to extend Date and  DateTime classes to support few more methods.

Following is the code and this is done using Ruby's Dynamic Proxy Technique.

Copy above date.rb file and keep it under initializer directory of your Rails application.

Ruby Threads + ActiveRecord == "Too Many Connections" ?

I recently encountered Mysql "Too Many Connections" issue in my Rails App, which makes my App down periodically. When I checked Mysql processlist, I could see 150+ active DB connections established from Rails App. How could my Rails App can open 150+ connections, though I mentioned AR connection pool size as 30. Baffled me !!!

Following is my development environment,

Ruby(2.0) + Rails(4.0) + Unicorn  + Nginx  + SideKiq + MultiTenant

• No.of Unicorn workers: 5
• No.of SideKiq processes: 1
• AR Connection pool size: 30 

According to experts, it would establish  5 * 1 + 30 = 35 maximum connections. Then why is it making 150+ connections in my App? Started Researching !!!

Found a magical sentence saying that the above calculation is true only for single threaded applications. Oh Yes! In my Rails app, I am spawning application level Threads based on no.of records being processed. So each Thread will try to make a connection to DB. Look at the following example,

class Article < ActiveRecord::Base
  def self.publish_all
    threads = []
    Article.by_unpublished.find_in_batches(batch_size: BATCH_CONST) do |articles|
      threads << Thread.new do
        ActiveRecord::Base.connection_pool.with_connection do
          articles.each do |article|
            article.publish
          end
        end
      end
    end
    threads.map(&:join)
  end
end

According to Rails Doc 'with_connection' method will Check-In the connection back to connection pool once after executing the Block given. But in the above example, it's not  releasing the connection rather it holds as active connection. So further requests to server keep creating new connections, this strange behaviour is the root cause of connection leak issue.

Solution !!!

Yes you are right, Check-In the connection explicitly.  So I created a utility class and wrapped around Thread invocation. For an example,

class ThreadUtility
  def self.with_connection(&block)
    begin
      yield block
    rescue Exception => e
      raise e
    ensure
      # Check the connection back in to the connection pool
      ActiveRecord::Base.connection.close if ActiveRecord::Base.connection
    end
  end
end

 class Article < ActiveRecord::Base
  def self.publish_all
    threads = []
    Article.by_unpublished.find_in_batches(batch_size: BATCH_CONST) do |articles|
      threads << Thread.new do
        ThreadUtility.with_connection do
          articles.each do |article|
            article.publish
          end
        end
      end
    end
    threads.map(&:join)
  end
end

Above example completely solved my connection leak issue, No more "Too Many Connections" notice from server.

Another strange behaviour which I would like to share here from above example is, whenever server spawning new thread, it creates new connection rather than reusing existing connection from pool of connections already available. This is due to the nature of ActiveRecord's Lazy connection pooling technique(connections are opened lazily). Once the no.of established connections in pool reaches the pool size mentioned in config, it grabs and reuses the connections from pool.

Now tell me guys, how many maximum connections that AR can establish from the above code fix for following configuration,

• No.of Unicorn workers: 5
• AR Connection pool size: 10

Is it 10 ? But the truth is 5 * 10 = 50 connections. Bcoz each unicorn process can establish 10 maximum connections in multi threaded application.

Your valuable comments are always Welcome !!!

Rake task to restart Sidekiq process

Rake task to update build version No. with latest tag name

The following code will help us to automate the process of updating build version No. by considering latest tag name available in git repo.

Assuming, you are deploying your Rails app using Capistrano.

1) In your deploy.rb add the following hook,

after "deploy", "installed_version:update"

And add a task(installed_version:update)

namespace :installed_version do
 task :update, :roles => :app, :except => { :no_release => true } do
   run "cd #{current_path} && RAILS_ENV=#{rails_env} rake update:version"
 end
end

2) Create a new rake task file in your rails app(lib/tasks/update_build_version.rake)

Add below lines,

namespace "update" do
 desc "This will update the installed version no. in DB from the latest tag name"
 task :version => :environment do
    tag = `git rev-list --tags --max-count=1`
    tag_name = `git describe --tags #{tag}`.chomp
    puts "Latest Tag No.#{tag_name}"
    attributes = {:version_no => tag_name, :build_dt => Time.now}
   Version.create(attributes)  # assuming you have versions tbl   
    puts "Build updated with Version No.\"#{tag_name}\""
 end
end

That's all, we are done !!!

Note: Before every deployment you need to tag your code with some version No.(ex: 1.2.1) and then run cap deploy. The above code will read the latest tag name(1.2.1) and update into versions table.

This post would be really helpful when you want to automate your Build Management process.

Rake Task with Arguments

The following example shows how to pass an argument to Rake Task. Here version No(:no) gets passed to task and the same gets saved into versions table. This example can be incorporated with your build management script to update the current version No. for each release.

namespace "build" do
 desc "This will update the version No."
 task :version, [:no] => :environment do |t, args|
   raise "Message: Version Number Required to build !!!" if args.no.nil?
   version = Version.first # assume you have a table named versions(:id, :version_no, :updated_at...)
   version.update_attributes(:version_no => args.no)
   puts "Build updated with Version No.#{args.no}"
 end
end

Usage:  rake "build:version[1.3.2]"

Git Tag

Tag  (people use this functionality to mark release points)

git tag  -- Listing Your Tags

git tag -l ’v1.4.2.*’  -- wildcard Tag listing

git tag v1.4    -- Creates Lightweight Tag

git tag -a v1.4 -m ’my version 1.4’  -- Creates Annotated Tag

git tag -s v1.5 -m ’my signed 1.5 tag’ -- sign your tags with GPG, assuming you have a private key.

git show v1.4  -- Shows the last commit of a Tag

git tag -a v1.2 9fceb02  -- Tagging later (if you forgot to Tag a release point)

git push origin v1.2  -- Sharing Tags (pushing to remote origin)

git push origin --tags --  Transfer all of your tags to the remote server that are not already there

git tag -d v1.2 -- Delete a Tag

git checkout -b v1.4-fixes v1.4  -- Creates a branch(v1.4-fixes) from Tag(v1.4)

git diff v1.1 v1.2 --stat  --  Shows no of files changed between two tags

git log --pretty=oneline v1.1..v1.2   --  Shows no of commits between two tags

Desktop to Mobile site switching Apache config

Recently i have come across a situation to write Apache config for Desktop to Mobile site switch and vice-versa.

Below are the Constraints,

1. Lets assume we have two sites, one for Desktop(web version 'www.domain.com') and another for Mobile(mobile version 'mobile.domain.com').
2. If i view my site from Desktop then Desktop version should appear, similarly if i view the same site from any mobile device then it should automatically redirected to Mobile site.
3. Important constraint here is, user should be allowed to change their preference from desktop to Mobile version and vice-versa by the link provided in both the sites('Mobile version','Web version'). Based on the preference set by user, proper version should appear when user Re-visits the site.
   
4. Moreover, if user is in 'web version' under search page, from here if user clicks 'Mobile version' then in Mobile site same search page with result should appear.

In order to implement the above scenarios, i have searched in Google many times but couldn't get help, so i started writing by my own (as below),

In your Web version site Vhost file:

RewriteCond %{REQUEST_URI} ^/switch_to_mobile
RewriteRule ^/switch_to_mobile$ - [co=version:mobile:.domain.com]
RewriteCond %{HTTP_REFERER} http:\/\/www.domain.com\/(.*)
RewriteRule ^/switch_to_mobile$ http://mobile.domain.com/%1 [R=302,L]

RewriteCond %{HTTP_COOKIE} version=mobile [NC]
RewriteRule ^(.+)$ http://mobile.domain.com$1 [L,R=302]

RewriteCond %{HTTP_USER_AGENT} BlackBerry
RewriteCond %{HTTP_COOKIE} !version=web
RewriteRule ^(.+)$ http://mobile.domain.com/$1 [R=302,L]

In your Mobile version site Vhost file:

RewriteCond %{REQUEST_URI} ^/switch_to_web
RewriteRule ^/switch_to_web$ - [co=version:web:.domain.com]
RewriteCond %{HTTP_REFERER} http:\/\/mobile.domain.com\/(.*)
RewriteRule ^/change_to_web$ http://www.domain.com/%1 [L,R=302]

RewriteCond %{HTTP_COOKIE} version=web [NC]
RewriteRule ^(.+)$ http://www.domain.com$1 [L,R=302]

                           (or)  Even simpler you may try the below config


Web version:

 RewriteCond %{REQUEST_URI} ^/change_to_mobile
 RewriteCond %{HTTP_REFERER}  http:\/\/www.domain.com\/(.*)
 RewriteRule ^/change_to_mobile$  http://mobile.domain.com/%1 [R=301,L]

 RewriteCond %{HTTP_USER_AGENT} iphone|ipod|alcatel|android|midp|240x320|blackberry|dopod|htc|huwai|lg|midp|nec|netfront|nokia|panasonic|portalmmm|sharp|sie-|sony|sonyericsson|symbian|benq|mda|mot-|motorola|palm|panasonic|philips|sagem|samsung|sanyo|sharp|sda|sgh-|t-mobile|vodafone|xda|pocket\ pc|opera\ mini|windows\ ce [NC]
 RewriteCond %{HTTP_COOKIE} !preference=web [NC]
 RewriteRule ^(.+)$  http://mobile.domain.com$1 [R=302,L]


Mobile version:

RewriteCond %{REQUEST_URI} ^/change_to_web$
RewriteRule ^/change_to_web$ "$0" [co=preference:web:.domain.com]
RewriteCond %{HTTP_REFERER} http://mobile.domain.com/(.*)
RewriteRule ^/change_to_web$  http://www.domain.com/%1 [R=302,L]

Hope above configuration should help you Guys !!!