« Ruby 1.9—Right for You? | Main | Pipelines Using Fibers in Ruby 1.9--Part II »

December 31, 2007

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d83451c41c69e200e54fcc97d58834

Listed below are links to weblogs that reference Pipelines Using Fibers in Ruby 1.9:

» Dave Thomas: Pipelines Using Fibers in Ruby 1.9 from correct
Bookmarked your post over at Blog Bookmarker.com! [Read More]

» links for 2008-01-02 from Jorge Mir Dot Com
PragDave: Pipelines Using Fibers in Ruby 1.9 PragDave (tags: ruby programming)... [Read More]

» Pipelines Using Iterators, Lambda Expressions and Extension Methods in C# 3.0 from Community Blogs
Inspired by Pipelines Using Fibers in Ruby 1.9 , I set to explore what could be done with C# 3.0 in the [Read More]

Comments

Mark

Nice one Dave. A truely ruby-tastic article!

Christophe Broult

Thank you for a delightful article. I can hardly wait for the next installment.

Ibrahim Ahmed

A great article. I need to read more about Ruby 1.9 new features especially Fibres and Threads. I recently read about Erlang / Haskell and wonder if Ruby could do the same distributed process / multi core / functional magic, any time in the future!? or that's totally another paradigm (I mean Functional programming) that Ruby programmers could not experience but a few of its real benifits.

Leif Wickland

Thanks for this post and for making me excited about 1.9, Dave. I'm looking forward to the rest of the story.

Daniel Cadenas

Excellent article as always.
Maybe I'm missing something but I think that the block in the first initialize "def initialize(&block)" is not used.

Dave Thomas

Daniel:

That's true: I've fixed it (we will use it tomorrow, though...)


Dave

yaxu

Is this what Haskell programmers call lazy evaluation?

brett

Hmmm, it seems like fibers are (at least superficially) the equivalent of generators in python.

yaxu: Yes, using Fiber.yield is effectively lazy evaluation. It's a lot cleaner in Haskell though since it's built into the language.

Ibrahim: The thing that makes functional languages like Haskell and Erlang good at concurrent programming is the minimization of mutable state. Once you bind a value to a variable that value cannot change so it is safe for other threads to use it without having to worry about synchronization. It would be a pretty radical paradigm shift for ruby to pick up this property.

Dave Thomas

Brett:

Fibers as shown here are basically generators. However, you can also turn them into full-blown coroutines and continuations.

The thing that's interesting to me about Ruby in this context is how much is can bend into multiple paradigms. Haskell does FP way better than Ruby. Smalltalk does OO (marginally) better. But Ruby does them all, and in a way that interoperates nicely.


Dave

Mitch Tishmack

Awesome article Dave, now I have to start using 1.9 more. I tested the difference performance-wise to 1.8.6 just using the fib example you gave (slightly modified). Some first run differences (not indicative to anything given cacheing etc...) Tested on stock ruby in 10.5.1 with security patches applied, 1.8.6 p100 I think is the 1.8 version.

$ time ruby fib.rb 40
102334155
ruby fib.rb 40 190.05s user 0.98s system 98% cpu 3:13.71 total
$ time ruby1.9 fib.rb 40
102334155
ruby1.9 fib.rb 40 49.24s user 0.30s system 98% cpu 50.526 total
$ time ruby1.9 fibfiber.rb 40
102334155
ruby1.9 fibfiber.rb 40 0.01s user 0.00s system 53% cpu 0.020 total

For completeness, those "other" languages from the benchmark folder (sans scheme, all modified just to allow first arg to be passed and to print the number):
$ time perl fib.pl 40
102334155
perl fib.pl 40 157.06s user 0.86s system 98% cpu 2:40.32 total
$ time python fib.py 40
102334155
python fib.py 40 81.56s user 0.38s system 98% cpu 1:23.05 total

And straight old c.
$ time ./fib 40
102334155
./fib 40 2.90s user 0.01s system 99% cpu 2.922 total

Paolo Bonzini

I disagree that Smalltalk does not do FP; Ruby does it "(marginally) better" because of the implied self, but otherwise they do it the same way---with blocks. (Likewise, I disagree that Ruby is worse than Smalltalk at OO, but in this comment I want to show off Smalltalk at OO).

Since you're basically using fibers as generators, you can simply use GNU Smalltalk's own generators (they should be portable to Squeak), which are Stream objects. The "pipe" method is just a stream

Stream extend [
| aBlock [
^Generator on: self do: aBlock
]
]

evens := [ :gen :x | x even ifTrue: [ gen yield: x ] ]
multiplesOfThree := [ :gen :x | x \\ 3 == 0 ifTrue: [ gen yield: x ] ]

data := (1 to: 20) readStream | evens | multiplesOfThree.
data next "returns 6"
data contents "returns (12 18)"

You need GNU Smalltalk 2.95 or later to run this.

My apologies for the poor formatting of this comment.

Ahruman

After the first occurrence of “MultiplesOf”, you’ve got [/PipelineElement] instead of [/code]. This messes up the rest of the page in Safari.

Dave Thomas

Ahruman: thanks. fixed

Paolo: I don't think Ruby or Smalltalk really do functional programming to any deep level. However, both can be used to implement particular FP constructs (such as generators).

Artem Voroztsov

Fiber can be used not only for piping, but also for spliting data streams.

They allow to transperently convert n-pass algorithm to obe-pass algorithm.


def squares
puts "squares before"
each_number do |n|
puts "#{n}**2 = #{n**2}"
end
puts "squares after"
end

def powers
puts "powers before"
each_number do |n|
puts "2**#{n} = #{2**n}"
end
puts "powers after"
end

# A really expensive generator
def each_number
puts "each number orig before"
[1,2,3,4,5].each do |n|
yield(n)
end
puts "each number orig after"
end


# Two pass algorithm
def do_it
powers
squares
end

---------------

# Generator is called twice
# But lets fiber it and make the algorithm one-pass.


alias :orig_each_number :each_number

def each_number
puts "each number before "
while true
begin
yield(@iterator.resume)
Fiber.yield
rescue Exception=>e
break
end
end
puts "each number after"
end

def do_it
@iterator = Fiber.new{ orig_each_number{|n| 2.times { Fiber.yield(n) } } }
@p = Fiber.new { powers }
@s = Fiber.new { squares }
5.times{ @p.resume; @s.resume; }
end

do_it


john robet

alias :orig_each_number :each_number

def each_number
puts "each number before "
while true
begin
yield(@iterator.resume)
Fiber.yield
rescue Exception=>e
break
end
end
puts "each number after"
end1Y0-A13 exam

Nest_d

Good article :) I read it and thought "I want to create it without Fibers". My result:

https://gist.github.com/1016493

It is based on your idea, but without Fiber.

The comments to this entry are closed.

Now in Beta

  • Programming Ruby, 3rd Edition
    Third Edition, Covering Ruby 1.9, now available
My Photo

Pragmatic Stuff

Photos

  • www.flickr.com
    This is a Flickr badge showing public photos from pragdave tagged with pragdave_badge. Make your own badge here.