About Metaprogramming speed »

Created at: 18.06.2008 10:03, source: Rails on the Run - Home, tagged: benchmarks metaprogramming ruby

In a previous article I took an example of bad metaprogramming and I pushed people to think twice before using metaprogramming.

My main points were that:

  • you might make your code way slower if you don't know what you are doing
  • readability might drop considerably
  • maintainability can become an issue

People left some very good comments about how to write the same module using metaprogramming and keep things fast.

Today Wycats pinged me about this post and told me that the issue was definemethod and that classeval is effectively the same as regular code, it gets evaluated in eval.c, just like regular Ruby code. On the other hand, defined_method has to marshall the proc.

I cleaned up my benchmarks using rbench, added some of the solutions provided to me and obtained the following results:

results

Here is the original/bad metaprogramming example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19


  module MetaTimeDSL

    {:second => 1, 
     :minute => 60, 
     :hour => 3600, 
     :day => [24,:hours], 
     :week => [7,:days], 
     :month => [30,:days], 
     :year => [364.25, :days]}.each do |meth, amount|
      define_method "#{meth}" do
        amount = amount.is_a?(Array) ? amount[0].send(amount[1]) : amount
        self * amount
      end
      alias_method "#{meth}s".intern, "#{meth}"
    end

  end
  Numeric.send :include, MetaTimeDSL

The no metaprogramming module is available there

Refactored:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20


  module RefaMetaTimeDSL

  {:second => 1, 
   :minute => 60, 
   :hour => 3600, 
   :day => [24,:hours], 
   :week => [7,:days], 
   :month => [30,:days], 
   :year => [364.25, :days]}.each do |meth, amount|
    self.class_eval <<-RUBY
      def r_#{meth}
        #{amount.is_a?(Array) ? "#{amount[0]}.#{amount[1]}" : "#{amount}"}
      end
      alias_method :r_#{meth}s, :r_#{meth}
    RUBY
  end

end
Numeric.send :include, RefaMetaTimeDSL

the refactor 2 or eval based solution provided by Matt Jones which uses class_eval like the previous refactor.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19


  module EvalMetaTimeDSL
    def self.included(base)
      base.class_eval do
        [ [:e_second, 1], 
          [:e_minute, 60], 
          [:e_hour, 3600], 
          [:e_day, [24,:e_hours]], 
          [:e_week, [7,:e_days]], 
          [:e_month, [30,:e_days]], 
          [:e_year, [365.25, :e_days]]].each do |meth, amount|
            amount = amount.is_a?(Array) ? amount[0].send(amount[1]) : amount
            eval "def #{meth}; self*#{amount}; end"
            alias_method "#{meth}s", meth
          end
      end
    end
  end
  Numeric.send :include, EvalMetaTimeDSL

and finally, the "better metaprogramming" version:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22


 module GoodMetaTimeDSL

  SECOND  = 1
  MINUTE  = SECOND * 60
  HOUR    = MINUTE * 60
  DAY     = HOUR * 24
  WEEK    = DAY * 7
  MONTH   = DAY * 30
  YEAR    = DAY * 364.25
  
  %w[SECOND MINUTE HOUR DAY WEEK MONTH YEAR].each do |const_name|
      meth = const_name.downcase
      class_eval <<-RUBY 
        def g_#{meth} 
          self * #{const_name} 
        end 
        alias g_#{meth}s g_#{meth} 
      RUBY
  end
end
Numeric.send :include, GoodMetaTimeDSL

Looking at the refactored version by Wycats, you can see he's right and the major issue with the original version was definemethod. Using classeval does make things almost as fast and even faster than the no metaprogramming version.

Interesting enough, the benchmarks show that some methods from the meta modules are faster than the ones from the no meta module. Overall, an optimized metaprogramming can be more or else as fast as a non meta code. Of course, with the new VMs coming up, things might change a little bit depending on the language implementation.

In conclusion, metaprogramming can be as fast as no metaprogramming but that won't help your code readability and maintainability, so make sure to only use this great trick when needed!

p.s: here is the benchmark file if you don't believe me ;)


more »

About Metaprogramming speed »

Created at: 18.06.2008 10:03, source: Rails on the Run - Home, tagged: benchmarks metaprogramming ruby

In a previous article I took an example of bad metaprogramming and I pushed people to think twice before using metaprogramming.

My main points were that:

  • you might make your code way slower if you don't know what you are doing
  • readability might drop considerably
  • maintainability can become an issue

People left some very good comments about how to write the same module using metaprogramming and keep things fast.

Today Wycats pinged me about this post and told me that the issue was definemethod and that classeval is effectively the same as regular code, it gets evaluated in eval.c, just like regular Ruby code. On the other hand, defined_method has to marshall the proc.

I cleaned up my benchmarks using rbench, added some of the solutions provided to me and obtained the following results:

results

Here is the original/bad metaprogramming example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19


  module MetaTimeDSL

    {:second => 1, 
     :minute => 60, 
     :hour => 3600, 
     :day => [24,:hours], 
     :week => [7,:days], 
     :month => [30,:days], 
     :year => [364.25, :days]}.each do |meth, amount|
      define_method "#{meth}" do
        amount = amount.is_a?(Array) ? amount[0].send(amount[1]) : amount
        self * amount
      end
      alias_method "#{meth}s".intern, "#{meth}"
    end

  end
  Numeric.send :include, MetaTimeDSL

The no metaprogramming module is available there

Refactored:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20


  module RefaMetaTimeDSL

  {:second => 1, 
   :minute => 60, 
   :hour => 3600, 
   :day => [24,:hours], 
   :week => [7,:days], 
   :month => [30,:days], 
   :year => [364.25, :days]}.each do |meth, amount|
    self.class_eval <<-RUBY
      def r_#{meth}
        #{amount.is_a?(Array) ? "#{amount[0]}.#{amount[1]}" : "#{amount}"}
      end
      alias_method :r_#{meth}s, :r_#{meth}
    RUBY
  end

end
Numeric.send :include, RefaMetaTimeDSL

the refactor 2 or eval based solution provided by Matt Jones which uses class_eval like the previous refactor.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19


  module EvalMetaTimeDSL
    def self.included(base)
      base.class_eval do
        [ [:e_second, 1], 
          [:e_minute, 60], 
          [:e_hour, 3600], 
          [:e_day, [24,:e_hours]], 
          [:e_week, [7,:e_days]], 
          [:e_month, [30,:e_days]], 
          [:e_year, [365.25, :e_days]]].each do |meth, amount|
            amount = amount.is_a?(Array) ? amount[0].send(amount[1]) : amount
            eval "def #{meth}; self*#{amount}; end"
            alias_method "#{meth}s", meth
          end
      end
    end
  end
  Numeric.send :include, EvalMetaTimeDSL

and finally, the "better metaprogramming" version:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22


 module GoodMetaTimeDSL

  SECOND  = 1
  MINUTE  = SECOND * 60
  HOUR    = MINUTE * 60
  DAY     = HOUR * 24
  WEEK    = DAY * 7
  MONTH   = DAY * 30
  YEAR    = DAY * 364.25
  
  %w[SECOND MINUTE HOUR DAY WEEK MONTH YEAR].each do |const_name|
      meth = const_name.downcase
      class_eval <<-RUBY 
        def g_#{meth} 
          self * #{const_name} 
        end 
        alias g_#{meth}s g_#{meth} 
      RUBY
  end
end
Numeric.send :include, GoodMetaTimeDSL

Looking at the refactored version by Wycats, you can see he's right and the major issue with the original version was definemethod. Using classeval does make things almost as fast and even faster than the no metaprogramming version.

Interesting enough, the benchmarks show that some methods from the meta modules are faster than the ones from the no meta module. Overall, an optimized metaprogramming can be more or else as fast as a non meta code. Of course, with the new VMs coming up, things might change a little bit depending on the language implementation.

In conclusion, metaprogramming can be as fast as no metaprogramming but that won't help your code readability and maintainability, so make sure to only use this great trick when needed!

p.s: here is the benchmark file if you don't believe me ;)


more »

Avoid using metaprogramming (seriously!) »

Created at: 04.05.2008 11:29, source: Rails on the Run - Home, tagged: benchmark merb metaprogramming speed

Ruby is sexy, Ruby is cool and its metaprogramming potential offers some really cook features. However you might not realize that your cleverness is slowing down your code.

Today I was working on cleaning up merb_helper a Merb plugin that brings a lot of the stuff Rails developers are used to. In Merb we aim for speed and try to avoid magic.

merb_plugin didn't receive a lot of love from the main contributors but few features were added by different contributors and the code became hard to maintain.

Looking at the code I quickly found this bad boy:

(Old Merb Time DSL using metaprogramming)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21


   
  module MetaTimeDSL

    {:second => 1, 
     :minute => 60, 
     :hour => 3600, 
     :day => [24,:hours], 
     :week => [7,:days], 
     :month => [30,:days], 
     :year => [364.25, :days]}.each do |meth, amount|
      define_method "m_#{meth}" do
        amount = amount.is_a?(Array) ? amount[0].send(amount[1]) : amount
        self * amount
      end
      alias_method "m_#{meth}s".intern, "m_#{meth}"
    end

  end
  Numeric.send :include, MetaTimeDSL

The above code looks awful to me and I decided to rewrite it a way I thought would be more efficient:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41


  module TimeDSL

    def second
      self * 1
    end
    alias_method :seconds, :second

    def minute
      self * 60
    end
    alias_method :minutes, :minute

    def hour
      self * 3600
    end
    alias_method :hours, :hour

    def day
      self * 86400
    end
    alias_method :days, :day

    def week
      self * 604800
    end
    alias_method :weeks, :week

    def month
      self * 2592000
    end
    alias_method :months, :month

    def year
      self * 31471200
    end
    alias_method :years, :year

  end
  Numeric.send :include, TimeDSL

To make sure I was right, I run the following benchmarks:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127


require 'benchmark'
TIMES = (ARGV[0] || 100_000).to_i

Benchmark.bmbm do |x|
  
  x.report("metaprogramming 360.seconds") do
    TIMES.times do    
      360.m_seconds
    end
  end
  
  x.report("no metaprogramming 360.hours") do
    TIMES.times do
      360.seconds
    end
  end
  
  x.report("metaprogramming 360.minutes") do
    TIMES.times do    
      360.m_minutes
    end
  end
  
  x.report("no metaprogramming 360.minutes") do
    TIMES.times do
      360.minutes
    end
  end
  
  x.report("metaprogramming 360.hours") do
    TIMES.times do    
      360.m_hours
    end
  end
  
  x.report("no metaprogramming 360.hours") do
    TIMES.times do
      360.hours
    end
  end
  
  x.report("metaprogramming 360.days") do
    TIMES.times do    
      360.m_days
    end
  end
  
  x.report("no metaprogramming 360.days") do
    TIMES.times do
      360.days
    end
  end
  
  x.report("metaprogramming 360.weeks") do
    TIMES.times do    
      360.m_weeks
    end
  end
  
  x.report("no metaprogramming 360.weeks") do
    TIMES.times do
      360.weeks
    end
  end

  x.report("metaprogramming 18.months") do
    TIMES.times do    
      18.m_months
    end
  end
  
  x.report("no metaprogramming 18.months") do
    TIMES.times do
      18.months
    end
  end
  
  x.report("metaprogramming 7.years") do
    TIMES.times do    
      7.m_years
    end
  end
  
  x.report("no metaprogramming 7.years") do
    TIMES.times do
      7.years
    end
  end
  
end


 Rehearsal ------------------------------------------------------------------
metaprogramming 360.seconds      0.130000   0.000000   0.130000 (  0.133164)
no metaprogramming 360.hours     0.050000   0.000000   0.050000 (  0.042655)
metaprogramming 360.minutes      0.130000   0.000000   0.130000 (  0.133327)
no metaprogramming 360.minutes   0.040000   0.000000   0.040000 (  0.042401)
metaprogramming 360.hours        0.140000   0.000000   0.140000 (  0.134312)
no metaprogramming 360.hours     0.040000   0.000000   0.040000 (  0.043125)
metaprogramming 360.days         0.130000   0.000000   0.130000 (  0.134949)
no metaprogramming 360.days      0.050000   0.000000   0.050000 (  0.043745)
metaprogramming 360.weeks        0.130000   0.000000   0.130000 (  0.135581)
no metaprogramming 360.weeks     0.050000   0.000000   0.050000 (  0.043544)
metaprogramming 18.months        0.130000   0.000000   0.130000 (  0.135234)
no metaprogramming 18.months     0.050000   0.000000   0.050000 (  0.044354)
metaprogramming 7.years          0.140000   0.000000   0.140000 (  0.144062)
no metaprogramming 7.years       0.050000   0.000000   0.050000 (  0.044392)
--------------------------------------------------------- total: 1.260000sec

                                     user     system      total        real
metaprogramming 360.seconds      0.130000   0.000000   0.130000 (  0.132567)
no metaprogramming 360.hours     0.040000   0.000000   0.040000 (  0.042777)
metaprogramming 360.minutes      0.140000   0.000000   0.140000 (  0.132554)
no metaprogramming 360.minutes   0.040000   0.000000   0.040000 (  0.043193)
metaprogramming 360.hours        0.130000   0.000000   0.130000 (  0.133027)
no metaprogramming 360.hours     0.050000   0.000000   0.050000 (  0.042613)
metaprogramming 360.days         0.130000   0.000000   0.130000 (  0.138637)
no metaprogramming 360.days      0.050000   0.000000   0.050000 (  0.043213)
metaprogramming 360.weeks        0.130000   0.000000   0.130000 (  0.134049)
no metaprogramming 360.weeks     0.040000   0.000000   0.040000 (  0.043713)
metaprogramming 18.months        0.140000   0.000000   0.140000 (  0.134941)
no metaprogramming 18.months     0.040000   0.000000   0.040000 (  0.043980)
metaprogramming 7.years          0.150000   0.000000   0.150000 (  0.143389)
no metaprogramming 7.years       0.040000   0.000000   0.040000 (  0.044585)
 0.136591)

The metaprogramming version of the same implementation is almost 3 times slower!

Moral of the story: only use metaprogramming if you really have to or if you don't care about speed of execution.


more »

Avoid using metaprogramming (seriously!) »

Created at: 04.05.2008 11:29, source: Rails on the Run - Home, tagged: benchmark merb metaprogramming speed

Ruby is sexy, Ruby is cool and its metaprogramming potential offers some really cook features. However you might not realize that your cleverness is slowing down your code.

Today I was working on cleaning up merb_helper a Merb plugin that brings a lot of the stuff Rails developers are used to. In Merb we aim for speed and try to avoid magic.

merb_plugin didn't receive a lot of love from the main contributors but few features were added by different contributors and the code became hard to maintain.

Looking at the code I quickly found this bad boy:

(Old Merb Time DSL using metaprogramming)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21


   
  module MetaTimeDSL

    {:second => 1, 
     :minute => 60, 
     :hour => 3600, 
     :day => [24,:hours], 
     :week => [7,:days], 
     :month => [30,:days], 
     :year => [364.25, :days]}.each do |meth, amount|
      define_method "m_#{meth}" do
        amount = amount.is_a?(Array) ? amount[0].send(amount[1]) : amount
        self * amount
      end
      alias_method "m_#{meth}s".intern, "m_#{meth}"
    end

  end
  Numeric.send :include, MetaTimeDSL

The above code looks awful to me and I decided to rewrite it a way I thought would be more efficient:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41


  module TimeDSL

    def second
      self * 1
    end
    alias_method :seconds, :second

    def minute
      self * 60
    end
    alias_method :minutes, :minute

    def hour
      self * 3600
    end
    alias_method :hours, :hour

    def day
      self * 86400
    end
    alias_method :days, :day

    def week
      self * 604800
    end
    alias_method :weeks, :week

    def month
      self * 2592000
    end
    alias_method :months, :month

    def year
      self * 31471200
    end
    alias_method :years, :year

  end
  Numeric.send :include, TimeDSL

To make sure I was right, I run the following benchmarks:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127


require 'benchmark'
TIMES = (ARGV[0] || 100_000).to_i

Benchmark.bmbm do |x|
  
  x.report("metaprogramming 360.seconds") do
    TIMES.times do    
      360.m_seconds
    end
  end
  
  x.report("no metaprogramming 360.hours") do
    TIMES.times do
      360.seconds
    end
  end
  
  x.report("metaprogramming 360.minutes") do
    TIMES.times do    
      360.m_minutes
    end
  end
  
  x.report("no metaprogramming 360.minutes") do
    TIMES.times do
      360.minutes
    end
  end
  
  x.report("metaprogramming 360.hours") do
    TIMES.times do    
      360.m_hours
    end
  end
  
  x.report("no metaprogramming 360.hours") do
    TIMES.times do
      360.hours
    end
  end
  
  x.report("metaprogramming 360.days") do
    TIMES.times do    
      360.m_days
    end
  end
  
  x.report("no metaprogramming 360.days") do
    TIMES.times do
      360.days
    end
  end
  
  x.report("metaprogramming 360.weeks") do
    TIMES.times do    
      360.m_weeks
    end
  end
  
  x.report("no metaprogramming 360.weeks") do
    TIMES.times do
      360.weeks
    end
  end

  x.report("metaprogramming 18.months") do
    TIMES.times do    
      18.m_months
    end
  end
  
  x.report("no metaprogramming 18.months") do
    TIMES.times do
      18.months
    end
  end
  
  x.report("metaprogramming 7.years") do
    TIMES.times do    
      7.m_years
    end
  end
  
  x.report("no metaprogramming 7.years") do
    TIMES.times do
      7.years
    end
  end
  
end


 Rehearsal ------------------------------------------------------------------
metaprogramming 360.seconds      0.130000   0.000000   0.130000 (  0.133164)
no metaprogramming 360.hours     0.050000   0.000000   0.050000 (  0.042655)
metaprogramming 360.minutes      0.130000   0.000000   0.130000 (  0.133327)
no metaprogramming 360.minutes   0.040000   0.000000   0.040000 (  0.042401)
metaprogramming 360.hours        0.140000   0.000000   0.140000 (  0.134312)
no metaprogramming 360.hours     0.040000   0.000000   0.040000 (  0.043125)
metaprogramming 360.days         0.130000   0.000000   0.130000 (  0.134949)
no metaprogramming 360.days      0.050000   0.000000   0.050000 (  0.043745)
metaprogramming 360.weeks        0.130000   0.000000   0.130000 (  0.135581)
no metaprogramming 360.weeks     0.050000   0.000000   0.050000 (  0.043544)
metaprogramming 18.months        0.130000   0.000000   0.130000 (  0.135234)
no metaprogramming 18.months     0.050000   0.000000   0.050000 (  0.044354)
metaprogramming 7.years          0.140000   0.000000   0.140000 (  0.144062)
no metaprogramming 7.years       0.050000   0.000000   0.050000 (  0.044392)
--------------------------------------------------------- total: 1.260000sec

                                     user     system      total        real
metaprogramming 360.seconds      0.130000   0.000000   0.130000 (  0.132567)
no metaprogramming 360.hours     0.040000   0.000000   0.040000 (  0.042777)
metaprogramming 360.minutes      0.140000   0.000000   0.140000 (  0.132554)
no metaprogramming 360.minutes   0.040000   0.000000   0.040000 (  0.043193)
metaprogramming 360.hours        0.130000   0.000000   0.130000 (  0.133027)
no metaprogramming 360.hours     0.050000   0.000000   0.050000 (  0.042613)
metaprogramming 360.days         0.130000   0.000000   0.130000 (  0.138637)
no metaprogramming 360.days      0.050000   0.000000   0.050000 (  0.043213)
metaprogramming 360.weeks        0.130000   0.000000   0.130000 (  0.134049)
no metaprogramming 360.weeks     0.040000   0.000000   0.040000 (  0.043713)
metaprogramming 18.months        0.140000   0.000000   0.140000 (  0.134941)
no metaprogramming 18.months     0.040000   0.000000   0.040000 (  0.043980)
metaprogramming 7.years          0.150000   0.000000   0.150000 (  0.143389)
no metaprogramming 7.years       0.040000   0.000000   0.040000 (  0.044585)
 0.136591)

The metaprogramming version of the same implementation is almost 3 times slower!

Moral of the story: only use metaprogramming if you really have to or if you don't care about speed of execution.


more »

bgurl
ипотпал
аквичоп


Tag cloud

Categories

Links

Blogs: 43
Posts: 2516