Archive for the ‘Mechanize’ Category
Invitations galore!
February 1st, 2008
Leveraging the data a user can provide is sometimes very important. This information, however small can be very useful when you want to publicize your product. And when it comes in orders of 10^3, its all the more reason that it should not be neglected 
Yes, I am talking of contact books & friend lists & what not. Just for an example, my Gmail contact list has over 1100 emails and my Orkut profile has over 250 valid contacts.
Keeping all this in mind, I started writing “Tell a Friend” module for ApnaBill.com - which takes care of inviting a single friend to a full bunch of friends from services like Gmail, Yahoo, Hotmail, LinkedIn, Orkut and Rediff (more to come soon!)
Inviting a single friend
Inviting friends from different services
BTW, the table which you see above, actually got me thinking. Its a 1×2 table with another 2×2 table (login stuff) in 1st column and another 1×1 table (service logo) in second column.
The next big problem that I am going to solve is delivery of email invites when they are of the order of 10^3 in number! Its actually a two forked problem - I have to work around the spamming policies of my domain and also provide a UI which is good enough to handle the mail sending status - now thats some brainy work!
Until then - I hope to get support for more services
Posted in ApnaBill, Mechanize, Startups | Comments (2)
How to effectively serialize a WWW::Mechanize object
October 8th, 2007
Last 3 days, I was almost pulling apart my hair, trying to figure out how can I serialize a WWW::Mechanize object to the database.
The problem started surfacing when I was trying to store a Mechanize instance into the session & oddly enough Rails was throwing the “We’re sorry, but something went wrong” error at me! I reported this on the RailsForum on this thread but without any luck. Infact due to my newbie’ness with ruby/rails made me just blindly post the error message on RailsForum without thinking much about the reasons.
I even posted this on Pragmatic Studio Rails mailing list where many helpful people suggested solutions and reasons. I had to dig into the problem or change the problem statement altogether!
After understanding the problem, the best way to solve the issue was to chose the later option - changing the problem statement, ie. - “How can a Mechanize object be serialized?”
Ruby’s YAML support is fantastic. Just requiring “yaml” into your script gives all your objects a functionality to export & import themselves into YAML using
yaml_text = some_object.to_yaml
some_other_object = YAML.load(yaml_text)
But as you could see in the forum post, doing a to_yaml on the Mechanize object (which has been previously used to fetch a page) returns a TypeError
TypeError (can’t dump TCPSocket):
/usr/lib/ruby/1.8/pstore.rb:349:in `dump’
Some googling and RTFM’ing would tell you that IO classes in Ruby cannot be serialized! DRAT!!! Now how do I serialize the object? Behind the scenes when storing anything in the session, serialization is required - & that was exactly why I was not able to store the Mechanize instance into the session!
But creating a new Mechanize object & then authenticating again & again would be too big a hit on my app’s performance. So what now?
Lets try to re-engineer the solution!
My project deals with remotely logging into a website & then scraping content off it as and when the user requests it. So if I can somehow maintain the Mechanize session (cookies & all) across the actions/controllers, I am good.
A quick look at the Mechanize documentation & I found that it uses CookieJar class to manage cookies. So if I can serialize this object, I will have the functionality I was looking for
Voila! I have a solution!
require “mechanize”
require “yaml”
require “pp”agent = WWW::Mechanize.new
agent.get(”http://www.google.com”)
yaml_text = agent.cookie_jar.to_yamlnew_agent = WWW::Mechanize.new
new_agent.cookie_jar = YAML.load(yaml_text)
pp new_agent.cookie_jar
And since the CookieJar can be serialized, I dont even need a database anymore! My session would work just fine by storing the CookieJar directly into it - per user, per session!
Wheeehaa!
Posted in Mechanize, Rails | Comments (0)