sobota, 25 lutego 2012
Everything You Always Wanted to Know About Fibers * But Were Afraid to Ask
I recently found out that most Ruby developers that are not familiar with Event Machine usually knows very little about fibers. So I decided to share some very trivial examples that will help you understand them quickly. Each example should push your knowledge and imagination about possible usages a little further. Tell me in comments if that worked for you.
Example 1:
What happened here, you may ask. We created new fiber. At the begining it was stopped. So we resumed it. It executed and ended. Trying to resume it again did not work. Actually that was very similar to Proc except that we can always call Proc multiple times.
Example 2:
While being inside fiber we can yield. That means that fiber stops and control goes back to the line that resumed fiber. We resumed the fiber second time starting from where we escaped (yielded) previously.
Example 3:
As you see, you can return value from fiber when yielding out. That value is accessible to code that called fiber (resumed it). How is it accessible ? Well "resume" simply returns what was yielded from resumed fiber.
Example 4:
Nothing extrordinary here. When fiber is fininshed the value of last statment is returned. Like always in ruby.
Example 5:
Just using iterators inside fibers to yield multiple times and outside to resume fibers multiple times to get you comfortable with doing that :).
Example 6:
This fiber can be resumed infinite number of times. It returns randomly generated value to the caller.
Example 7:
As you can see in this example now you can resume the fibers in whatever order you like. You are in charge of execution order. That's usually the point of using fibers.
Example 8:
Why would you ever want to use fibers ? Well consider this example. You have an algorithm that requires to somehow operate on the result of two different kind of operations. One of them is very tough like computing prime numbers (obviously my solution to that could be way more optimized, but that is not the point). The second kind of operation is very easy (computing square in our example, trivial). You could use threads for doing that but threads are scheduled by Ruby VM or operating system and you have almost no power over them. If we computed prime numbers in one thread and squared numbers in another thread then probably we would end up with lot more squared numbers computed than prime numbers.
In our examples every algorithm is given enough time to compute one, next sequence number and then yields. That gives the scheduler (our code) the opportunity to resume another fiber and compute next number of second sequence . Fibers give you the ability to manually schedule peaces of codes and avoid the need to synchronize threads for accessing shared data structures.
Example 9:
Two fibers can use the same shared variable without using any kind of lock for accessing it because we are guaranteed that only 1 fiber is executing at the same time. So every fiber can be sure that no other fiber changed anything in any variables until Fiber.yield is called next time.
Example 10:
If fibers are your jobs then you need to know if they are unfinished and can be resumed again or finished and calling them again would lead to "dead fiber error". Usually this is not known upfront by the scheduling code but rather recognized based on returned value.
Summary:
I hope that now you know something, something about fibers (at least more than before reading this post) and it is not a black magic anymore. They might not be of any use to you but that is definitely not an excuse for ignoring them completely.
Caveats:
Example 1:
What happened here, you may ask. We created new fiber. At the begining it was stopped. So we resumed it. It executed and ended. Trying to resume it again did not work. Actually that was very similar to Proc except that we can always call Proc multiple times.
Example 2:
While being inside fiber we can yield. That means that fiber stops and control goes back to the line that resumed fiber. We resumed the fiber second time starting from where we escaped (yielded) previously.
Example 3:
As you see, you can return value from fiber when yielding out. That value is accessible to code that called fiber (resumed it). How is it accessible ? Well "resume" simply returns what was yielded from resumed fiber.
Example 4:
Nothing extrordinary here. When fiber is fininshed the value of last statment is returned. Like always in ruby.
Example 5:
Just using iterators inside fibers to yield multiple times and outside to resume fibers multiple times to get you comfortable with doing that :).
Example 6:
This fiber can be resumed infinite number of times. It returns randomly generated value to the caller.
Example 7:
As you can see in this example now you can resume the fibers in whatever order you like. You are in charge of execution order. That's usually the point of using fibers.
Example 8:
Why would you ever want to use fibers ? Well consider this example. You have an algorithm that requires to somehow operate on the result of two different kind of operations. One of them is very tough like computing prime numbers (obviously my solution to that could be way more optimized, but that is not the point). The second kind of operation is very easy (computing square in our example, trivial). You could use threads for doing that but threads are scheduled by Ruby VM or operating system and you have almost no power over them. If we computed prime numbers in one thread and squared numbers in another thread then probably we would end up with lot more squared numbers computed than prime numbers.
In our examples every algorithm is given enough time to compute one, next sequence number and then yields. That gives the scheduler (our code) the opportunity to resume another fiber and compute next number of second sequence . Fibers give you the ability to manually schedule peaces of codes and avoid the need to synchronize threads for accessing shared data structures.
Example 9:
Two fibers can use the same shared variable without using any kind of lock for accessing it because we are guaranteed that only 1 fiber is executing at the same time. So every fiber can be sure that no other fiber changed anything in any variables until Fiber.yield is called next time.
Example 10:
If fibers are your jobs then you need to know if they are unfinished and can be resumed again or finished and calling them again would lead to "dead fiber error". Usually this is not known upfront by the scheduling code but rather recognized based on returned value.
Summary:
I hope that now you know something, something about fibers (at least more than before reading this post) and it is not a black magic anymore. They might not be of any use to you but that is definitely not an excuse for ignoring them completely.
Caveats:
- http://bugs.ruby-lang.org/issues/5437
- Fibers cannot be executed in different threads. This might change in Ruby 2.0.
sobota, 21 stycznia 2012
Model logic
We all know that our models should be fat and controllers / views are supposed to be simple (dumb). Recently I was asked why such code should be put into model :
What's wrong with using Ad.unverified_ads.last in the view. I don't have an answer for the "why?" question often asked by beginner programmers. You can trust your more advanced colleagues that it should, that it is beneficial and will make your life easier or stick with your current way, wait until the application gets bigger, and regret it later. Anyway, I always ask myself one question when I have to decide whether something probably is part of the model. Would I need this method if the application had different way of interacting with the user. If it was a console application or just an API serving JSON to some clients. Given the previous code example. Can I image an usecase that requires me to display last unverified ad to user interacting with the program via command line. Yep, I can. Is it possible that I am gonna need it when serving JSON response? Might be. So I think I would make it part of the model.
How about this one:
Would I use it in CLI ? Nope. When rendering JSON ? Not really ? So... Not a model.
Third example:
I think it can be part of your model but does not have to. Even if this method could be useful in console application or for JSON api it looks like it still fits better in the presenter layer (or helper) instead of model layer.
I know that the examples are extremely simplified but that's not the point. When you are in doubt, just image that you would also have to implement the same app in completely different environment that has nothing to do with HTML. Which parts would you still keep in your model ? Asking myself such questions helps me make right decisions.
What's wrong with using Ad.unverified_ads.last in the view. I don't have an answer for the "why?" question often asked by beginner programmers. You can trust your more advanced colleagues that it should, that it is beneficial and will make your life easier or stick with your current way, wait until the application gets bigger, and regret it later. Anyway, I always ask myself one question when I have to decide whether something probably is part of the model. Would I need this method if the application had different way of interacting with the user. If it was a console application or just an API serving JSON to some clients. Given the previous code example. Can I image an usecase that requires me to display last unverified ad to user interacting with the program via command line. Yep, I can. Is it possible that I am gonna need it when serving JSON response? Might be. So I think I would make it part of the model.
How about this one:
Would I use it in CLI ? Nope. When rendering JSON ? Not really ? So... Not a model.
Third example:
I think it can be part of your model but does not have to. Even if this method could be useful in console application or for JSON api it looks like it still fits better in the presenter layer (or helper) instead of model layer.
I know that the examples are extremely simplified but that's not the point. When you are in doubt, just image that you would also have to implement the same app in completely different environment that has nothing to do with HTML. Which parts would you still keep in your model ? Asking myself such questions helps me make right decisions.
sobota, 14 stycznia 2012
Active Reload - mission accomplished
Once upon a time I was working on a big Rails application and every time I wanted to fix some CSS or JS issues I had to wait for a few seconds to reload the page which was making me really unproductive and unmotivated. I knew I was not the only one who suffered from such delays when working with big Rails apps. I occasionally spotted some comment about it here and there, in blog comments, reddit, rails forums. There were also other solutions that tried to fix the problem rails-dev-boost, rails_dev_mode_performance but had their limitations. So I decided to do something about. My initial trials were overcomplicated, error prone but eventually I created something that was working fine for my project. And it made my productive on that project again and much more happier. The solution was so incredibly simple (in concept and in materialization) that I still find it astonishing that no one before released such thing as a gem.
The fact that it was working for me meant nothing and proved nothing also. The gem needed to be tested by wider audience which would probably be very skeptical about it. So I had to have their attention somehow. I wanted to show to other people how I felt when using this gem, what is the profit of using it but I could not show the application that I was working on at that time. Also that would not convince anyone since well YMMV and nobody would be able to repeat my performance research without access to my application. My friends from DRUG (Dolnośląski Ruby User Group - the organizers of wroc_love.rb conference) advised me to check out if it works with spree . It turned out to work pretty well so I recorded two videos showing the difference. I used bbq which uses capybara under the hood to driver the browser automatically and make the test independent of human activity which could disturb it. I also provided exact steps to repeat so that everyone could challenge me.
Michał Łomnicki invented this catching phrase "Rails development mode 300% faster" and used it to submit a post to reddit. I did not want initially to make this solution part of Rails code because I knew that at that moment it would be probably hard for me to convince Rails core team that it is useful, working and could actually solve some people problems. My plan was to make a gem, hope for some buzz, fix bugs, hope that Rails core won't ignore it and will merge it into Rails itself. It all stared to work...
The fact that it was working for me meant nothing and proved nothing also. The gem needed to be tested by wider audience which would probably be very skeptical about it. So I had to have their attention somehow. I wanted to show to other people how I felt when using this gem, what is the profit of using it but I could not show the application that I was working on at that time. Also that would not convince anyone since well YMMV and nobody would be able to repeat my performance research without access to my application. My friends from DRUG (Dolnośląski Ruby User Group - the organizers of wroc_love.rb conference) advised me to check out if it works with spree . It turned out to work pretty well so I recorded two videos showing the difference. I used bbq which uses capybara under the hood to driver the browser automatically and make the test independent of human activity which could disturb it. I also provided exact steps to repeat so that everyone could challenge me.
Michał Łomnicki invented this catching phrase "Rails development mode 300% faster" and used it to submit a post to reddit. I did not want initially to make this solution part of Rails code because I knew that at that moment it would be probably hard for me to convince Rails core team that it is useful, working and could actually solve some people problems. My plan was to make a gem, hope for some buzz, fix bugs, hope that Rails core won't ignore it and will merge it into Rails itself. It all stared to work...
I must admit I wasn't the best gem maintainer. I could not fix some bugs, merging some pull requests took me too much time, testing the gem with newest Rails releases happened often weeks later. Guilty as charged. But it does not change the fact the gem seemed to be helping other people and most of the time did its job well. I wish I could somehow count the amount of time that it saved.
Than one day I woke up and saw that Jose Valim merged Active Reload into Rails and also fixed some bugs during that process and even added very simple solutions to problems that I could not solve (ex.: how to quickly detect changes on database caused by migrations when model file did not change - his solution: observe also schema.rb file which is regenerated after running migrations). And it was like: OMG my gem is in Rails :). And it happened exactly as I planned.
It convinced me that:
- after so many years you can still have impact over general Rails shape,
- even when you only spend couple of hours trying to make it better;
- it does not matter if people know you or not, you can still make a difference and become somehow known in this process;
- sometimes you only need dozes of lines of code instead of hundreds to make a difference;
- If it is important to the community, Rails core will notice it and incorporate;
- If you have some idea: make gem, advertise it (movie is must have in Rails world, sorry) and watch what will happen, maybe something bigger than you imagined. Maybe nothing but it will be still worth trying.
Since Active Reload is now part of Rails ( Announcement ) and it did its job I will no longer maintain it. If it works for your Rails 3.0 or 3.1 application, good. If it does not, try upgrading to 3.2 and maybe you will have a better luck. If there is any problem with this solution in Rails 3.2, create an issue on github. It is now in the hands of Rails community. We are now all responsible to remember that Rails was supposed to allow us developing applications quickly. That's why we felt in love with it. We wanted to make blogs in 15 minutes, not hours.
Die Active Reload, long live Rails.
środa, 28 grudnia 2011
Validation error objects (and hashes)
TL;DR. You can add Hash or custom class instances as error. It does not have to be a String.
Let's start with a simple Rails code that uses standard validation errors and see what kind of problems we have with such approach.
This is how it looks like when validation error occurs:
Let's just say that for a better ui experience we want to change the formatting of such
validation message (ex. numbers can be bold so that user pays attention to them). However, current solution does not allow us to do it easily. That is because of the fact that error is simply a String. Also the whole message is generated on the model side and just displayed on the view. This is a very quick and simple approach but does not give as a wide spectrum for improvement. Fortunately our errors does not to be of such simple type. We can put there something more complex like a Hash.
Looks perfect but sadly this implementation leads to I18n::InvalidPluralizationData exception so we must get a little dirty to deal with this error.
Now it works :
And now our error messages can be much more customizable and event look totally different. There is no need to stick with plain text. We can do better. There is a nice separation here between view and model. Model tells us as much about the error as possible. View generates the message and presents it nicely.
What is another useful benefit of such approach ? Imagine now, that Rails is just an API for mobile client (iphone, android, blackberry, windows, whatever). It is the responsibility of the view created on the phone to get the errors, translate them and display in a meaningful way for that platform. But you can help as an api creator as much as possible and make your colleges job easier in the future by simply providing lot of information. We don't want to hardcode our minimal and maximal requirements for the login length. That should be easy to change. Neither we want to generate the message in every possible language that can be used. We just want to provide enough information for the api clients so that they can get what went wrong, how to fix and how to display it to the user.
And guess what. This solution already does it. If you send the request to /users.json you will get a response like this:
{"login":[{"code":"INVALID_LENGTH","max":10,"min":2,"current":65,"description":"Invalid length"}]}
Your mobile clients can use the code, min, and max properties to display sensible error message for the user. In case when server is upgraded and a new validation is added, old clients can just show the text from "description" key until they are upgraded. After upgrade they should recognize the new error message code and relay on other provided keys when displaying the message.
All of that was nice but I think that is too much of "hash driven development". Let's make some OBJECTS.
Unfortunately, now data returnd for json clients are not identical.
{"login":[{"min":2,"max":10,"current":65}]}
Let's fix it with a simple approach:
But i don't think that this should be part of that model. After all it is just a json representation of the error. So probably we should go here with roar gem or any other that would simplify this. I see a value in an approach in which we have one representer to our Json client and one to Html clients (being just actionview part of rails).
Another problem is that we basically had to reimplement the validator that already exists in Rails. Let's check if that could be done better:
As you can see I had to fix some parts of Active Model to make it possible. I think it could be a good idea to submit a pull request with those changes. It's in my mind. Also there was no easy way to fill "current" attribute. Filling it would require a rewrite of few more lines in active model as well as making possible to add custom error types as part of validation declaration which is something that did not work well in step 2 (this part that leads to I18n::InvalidPluralizationData exception). I will definitelly tell you went all that happens :)
You can get the whole exemplary application at: https://github.com/paneq/active-model-errors-as-objects-example
Let's start with a simple Rails code that uses standard validation errors and see what kind of problems we have with such approach.
This is how it looks like when validation error occurs:
validation message (ex. numbers can be bold so that user pays attention to them). However, current solution does not allow us to do it easily. That is because of the fact that error is simply a String. Also the whole message is generated on the model side and just displayed on the view. This is a very quick and simple approach but does not give as a wide spectrum for improvement. Fortunately our errors does not to be of such simple type. We can put there something more complex like a Hash.
Looks perfect but sadly this implementation leads to I18n::InvalidPluralizationData exception so we must get a little dirty to deal with this error.
Now it works :
What is another useful benefit of such approach ? Imagine now, that Rails is just an API for mobile client (iphone, android, blackberry, windows, whatever). It is the responsibility of the view created on the phone to get the errors, translate them and display in a meaningful way for that platform. But you can help as an api creator as much as possible and make your colleges job easier in the future by simply providing lot of information. We don't want to hardcode our minimal and maximal requirements for the login length. That should be easy to change. Neither we want to generate the message in every possible language that can be used. We just want to provide enough information for the api clients so that they can get what went wrong, how to fix and how to display it to the user.
And guess what. This solution already does it. If you send the request to /users.json you will get a response like this:
{"login":[{"code":"INVALID_LENGTH","max":10,"min":2,"current":65,"description":"Invalid length"}]}
Your mobile clients can use the code, min, and max properties to display sensible error message for the user. In case when server is upgraded and a new validation is added, old clients can just show the text from "description" key until they are upgraded. After upgrade they should recognize the new error message code and relay on other provided keys when displaying the message.
All of that was nice but I think that is too much of "hash driven development". Let's make some OBJECTS.
Unfortunately, now data returnd for json clients are not identical.
{"login":[{"min":2,"max":10,"current":65}]}
Let's fix it with a simple approach:
But i don't think that this should be part of that model. After all it is just a json representation of the error. So probably we should go here with roar gem or any other that would simplify this. I see a value in an approach in which we have one representer to our Json client and one to Html clients (being just actionview part of rails).
Another problem is that we basically had to reimplement the validator that already exists in Rails. Let's check if that could be done better:
As you can see I had to fix some parts of Active Model to make it possible. I think it could be a good idea to submit a pull request with those changes. It's in my mind. Also there was no easy way to fill "current" attribute. Filling it would require a rewrite of few more lines in active model as well as making possible to add custom error types as part of validation declaration which is something that did not work well in step 2 (this part that leads to I18n::InvalidPluralizationData exception). I will definitelly tell you went all that happens :)
You can get the whole exemplary application at: https://github.com/paneq/active-model-errors-as-objects-example
Subskrybuj:
Posty (Atom)