spam « Tech, Faltoo Fundae, and NITK ArBits

How / Why did Rakhi trend – A case study

August 3, 2009 at 11:55 pm | In Internet, Technical, Visions, arbit, criticism, fakereviews, humour, spam, twitter | 10 Comments
Tags: average tweets per second, beer summit, bob saget, cash for clunkers, elesh parujanwala, geekiness, gul panag, indian reality show, indiavotes09, mass hysteria, ndtv imagine, pritish nandy, rakhi ka swayamvar, rakhi sawant, review, shark week, snowball, statistics, trending topic, twitscoop, twittebrity, twitter, twitter trends, whatthetrend

I assume that most of you are still recovering from last night’s events. Totally surreal stuff, I must say. Triple Baaraats, 3 grooms , 2 idiot MC’s and of course the multiplee talented Rakhi Sawant. A by-product of this whole ruckus was that India witnessed yet another trending topic [after #indiavotes09 etc ]

This was a pretty interesting experience to watch, as well as to participate in, as one could clearly see the effect of opinion snowballing, and mass-hysteria eventually taking over.

This is all I can say about ‘why’ did Rakhi trend. Well you see, ‘ The human mind is a complex system, and…”.

Oh, ditch that. It’s the ‘how’ that’s explainable.

So, how did Rakhi Madamji muster so much mojo?

Well, the obvious answer has to got to be – “due to a lot of tweets in a short timeframe”.

For the uninitiated, I’ll provide the same background that I wrote on http://whatthetrend.com/trend/Rakhi [ which incidentally, is a wonderful site to know why a topic is trending on twitter. So the next time you see something like "Cash for Clunkers" or " Bob Saget's gone bonkers" trending on twitter, you know where to look for answers ]

Rakhi Sawant is an Indian film actress who participated in a Reality Show on National Television (Channel – NDTV Imagine) called “Rakhi Ka Swayamvar” to select a bridegroom for her. In the Final show aired tonight (2-aug-2009), she selected “Elesh Parujanwala” from Canada as her husband to be.

[ I'd throw in an image but for the fact that I couldn't find a single decent picture, and even among the bhajan variety, there were none with a Creative commons license. Perverts, please search here ]

So, with this much info, lets dive into some statistics.

[ Note: ATPS - Average Tweets Per Second ]

The show started at around 9.00 p.m [ I.S.T], and for the initial 10 minutes the tweet flow was quite slow, at around 5 ATPS ]

Soon, some celebrities [or twittebrities, if that floats your boat ], joined in the action, and all hell broke loose. Some samples.

[ Ah, the marvels of good liquor ]

[ Sums up the nation's opinion with this tweet ]

Soon enough, the twitter universe was flooded with retweets, ” what the eff ” ’s, <brain explodes>, ‘shoot me. shoot me now‘ ’s , on one end.

Not to be perturbed by this, the positive Rakhi Chee forces retaliated with tweets like ” Go Elesh“, ” I’d marry Elesh, He’s cho cute“, ” OMG, Rakhi Sawant looks so hot ya“.

But it was mostly the apathetic crowd that both cheered and jeered [ I know that doesn't make sense on many levels, but work with me on this ], that was the peach of the tweetset. Sample these.

[ Witty that one ]

['Main is Desh ka damaad Hoon', but in a canadian accent. Sounds vaguely familiar]

By this time [ around 9.30 p.m], the term was buzzing on Twittscoop at around 10 ATPS. The next best term was “Shark week” at 30 ATPS, which had already started to trend. So, on an a normal day with not much news happening [ no plane crashes, no celebrity deaths, no white police / afro-american professor drinking chilled beers with the U.S president ], we can make a decent speculation that there has to be atleast 30 ATPS for a topic to trend.

This by itself is no mean feat, so Kudos to Rakhi for keeping us glued to the telly. Assume that the average twitterer, even in the heights of their frenzy tweets about 1 tweet per 10 minutes.

[ The power twitterer does about 1 tweet in 2 minutes say, and the other twitterer who's just an innocent bystander,just pops in to say " Oh, why's this topic trending". say 1 tweet per 30 minutes].

In some universe with a favourable averaging scheme, the number 1 tweet in 10 minutes figures, and I’ll use that.

To achieve a critical state of 30 ATPS, for say about an hour, for the trend to be noticed, we’d need,

1 person can tweet 1 tweet in 10 minutes = 1 / 600 ATPS

x people need to tweet 30 tweets in 1 second. 30 ATPS

So, assuming an equal distribution of tweeting in the timeframe, we needed about 18000 people to start the trend, which is quite reasonable and achievable, since there around 25 million twitter users globally out of which around 2% are Indian, [1] [2], which makes it a potential 500,000 Indian users.

This was the tweet density for yesterday. This is what a one-time-wonder trend looks like.

So, as you can notice, ‘rakhi’ was in the limelight for around 2 hours[ mentally visualize chopping off the curve where the height reaches 30 ]

[ graph courtesy: Twittscoop ]

There were some obvious side-effects of this inane exercise. People not used to seeing their twitter stream getting littered like this, got extremely pissed off, and started issuing death threats in all possible directions. Sample.

[ My my, such strong emotions. Now you know how provocative Rakhi's aura can be ]

So, in conclusion. Bah, easy peasy work. We can make anything trend, given enough determination, teamwork and resolute joblessness.

Hum honge #kaamyaab ek din. [ Loosely translates to " May the #manForce be with you" ]

P.S: Respect to all the twitterers featured in this post. I come in peace. Don’t shoot the messenger.

The Full Monty Problem…

October 6, 2008 at 10:27 pm | In Technical, arbit, humour, literary, sarcasm, spam, travel | 11 Comments
Tags: bobby da dhaba, bournvita quiz contest, chintu candy, dasgupta, derek o'brien, full monty, himesh reshammiya, i respect sardars, iit coaching, jim morrison, karan johar, karzz, koffee with karan, monty hall problem, nigeria, phishing scams, probability, sardar jokes are funny, tata mcgraw hill

Monty Singh was a wise man. The brightest in the land.

On that eventful Thursday, he received an email. This was a mail like.no.other.

It was from a Nigerian King.

A real friggin rich Nigger Raja. [P.C version - Niger. But different country]

Monty Singh was a pyoor Veggie. He didn’t like spam.

But this mail had to be genuine. His IIT alumnian brain could sense it. It was authentic. Right down to the black fonted signature in Wingdings.

He glanced through the contents. He couldn’t believe it. He read it again, this time slowly, and only then did the weight of the matter dawn on him.

He was chosen to participate in a Game show. A quiz of sorts.

Monty smirked. He was an ace quizzer. How he missed those days.

Flashback : Brought to you by Chintu Candy.

It was in seventh standard. He had had his morning’s cuppa’ Horlicks.

Then he went to BQC, thrashed Derek O’Brien mostly left, and occasionally right as well. Pinky Singh was a proud mother that day.

Monty came back from his reverie. He had to think this through.

Monty loved Probability. He simulated a random bit generator. Lady Luck was with him. “Go to Nigeria, you worthless bastard!”, she bellowed.

The queue for the Visa was shorter than he had expected. There was just one local brown model visiting the country for a Fair and Lovely – Limited Nigerian Edition ad-shoot. He grinned as he saw the neighbouring ‘US of A’ Visa line, mostly consisting of bespectacled grad wannabees.

He was received in Nigeria, amidst a royal fanfare. He was led to the only 7 star hotel in the country - Bobby Da Dhaba. Monty felt right at home.

He woke up that morning, and got himself a beer.

Oops. Wrong post!!!

Monty was up and soon spiffily dressed himself. His father’s pink tie would go well with his lemon yellow shirt.

Karan Johar, the host, looked surprisingly hetero that morning. Must be all that Koffee, thought Monty. “Never mind his temporary non-gayness”. “Concentrate”, he said to himself, as he walked to the stage, which was lit by a thousand colour-colour LED’s. A sight to behold.

Monty raced through the questions like Usain Bolt on steroids. They didn’t call him “Monty Mastermind” just like that.

The final question. This was a toughie. Monty kept his cool. He worked it out. Ruddy Brilliant. He was dingchakkingly good.

“And now Mr.Monty. How bout a bonus round”, shrieked Johar.

“ A flirty car, or you lose it all…..”

No, wait. No one had told him about a bloody bonus round.

As if reading his concerns, Johar replied, “ Don’t worry, Its just a tiny game of probability”.

Gosh. Monty almost had a tiny orgasm.

“ Very similar to the Monty Hall scenario, I take it that you know about it”, asked Johar.

“Pfft. Know about it? Why do you think my dad named me Monty?”

“Oh. I thought that was because you like to….. Never mind…”

“Ok. All the doors are hidden behind this wall. Just for kicks”. “And…”

“Oh. Will you start already”. “I choose door no.2”. “Which car is it btw?”

“Premier Padmini’s hot friend, Diablo Lamborghini….” “ Whate joke . Whate joke. Ha . Ha.. I know . I can be a pain in the bottoms sometimes”, quipped Karan Johar.

“Ok. Mr.Monty. I’ll open door no.1 and… WTF…”.

“Damn you, Nigerians, stop touching my goat”.

Monty’s brain started working faster than a computer. All those nuggets from Dasgupta, and T.M.H, heck even some from Krishna’s came back to him in a rush. He evoked Bayes, and his conditional Probabilistic models. And in a jiffy, the answer was gambolling right in front of his eyes.

“So, Mr. Monty, what’s your call? Will you flip your choice, or keep it?”

“ Duh. Flip my choice. Obs”.

“Ok. Have it your way.” ……. “ “Hurrah, You win….”

…..

“ the goat”.

“There were only two doors. Retard”….

All rise for the Nigerian Anthem.

P.S : Monty Singh was a wise man. The brightest in the land.

Update – 7-10-2008

Atul asked me whether this was a Himesh Reshammiya belting post? Actually, I am currently cursing myself for not noticing that Himesh is playing Monty’s role in Karzzzzzzz ( Did I miss a ‘z’? ).

Quoting Himesh – “ Rishi Kapoor is the best-looking Monty, I’m the worst” – We agree.

That, friends, is a different Full-Monty-Problem altogether.

Self-Spam

May 30, 2008 at 5:27 pm | In Technical, Visions, arbit, criticism, humour, image processing, sarcasm, spam | 9 Comments
Tags: akismet, captcha, email, gmail, google, image processing, nigeria, pranketh, recaptcha, scam, self-spam, spam assassin, spamming, spoof, waterloo

A small technical exercise testing the effectiveness of gmail to combat spam, which on their site says “Fast, searchable email with less spam”.

This is true mostly. However, there is a downside to gmail spam filters w.r.t false-positives, i.e (filter mistakenly flags a good mail as spam)
These numbers are particularly high in case of gmail. I’m comparing it to, say using Spam-Assassin , or Akismet as your spam filter. I’ve had a sufficient number of good mails, not so useful, but not spam either-> which have been flagged off as spam.
This is unacceptable. In my case, I’ve the patience, and also a weird interest to go through my spam messages everday. But, busy people would consider this to be a “costly nuisance”.
The Nuisance is obvious. It’s also costly as the flagged message might have critical business information.
Most people however do not worry much about this. Because they trust their spam filters to weed out only the bad guys. As Matt Cutts of Google says about Google Search so elegantly,

“Sure,I could stop all the spam in the world if I didn’t have to return any search results.” [source]

What people don’t like, however, is the appearance of { stimulating/enlarging/Nigerian King bequeathing his 100 million $ /cheap adobe software or Rolex-es / mortgage loans at unbelievable rates } mails in their inbox. This is totally irritating, and this does more damage to the lay-user than the previously mentioned almost-rare case.
Spam filters involve many techniques which are constantly under development. These chiefly include heuristic[ drawing from previous experiences, a learning method], bayesian filters, simple database matching, matching ip’s with regular spammers, and lots more involving complex probability/statistical models beyond my wildest dreams.
So, being the evil chap that I am, I did this nonsensical thing. I sent a spam mail to myself.

Granted, this is stupid. Gmail should place a trust on the sender[ me], and the sending server[gmail.com], and hence classify this as a legitimate mail.
So I repeated this exercise with a well-known email spoofer, www.pranketh.com . What this nifty project [written by two brilliant chaps from University of waterloo] does is pretty simple.
As the site says – “It allows you to send an email, that looks like its sent from someone else.” Or in simple terms.
I can send an email from id’s like mukeshambani[at]reliance[dot]in, or soniagandhirocks[at]congressrocks[dot]gov[dot]in.
[ note the delicate usage of "rocks", Blogger belongs to google. I don't want no risks]. A screenshot of pranketh’s page.

Yes. I know your doubt. If its so easy, why do nefarious miscreants use stupid yahoo/gmail id’s to threaten people, or send smokescreen bomb-footage, to the extremely retarded tv channel Aaj Tak[ self-proclaimed to be "Sarv-Shresht].
The point is the email server, folks. Your id is spoofed. But the smtp server name. No no. I couldn’t dream in my distant dreams to get a gov.in smtp[ Since I don't happen to be a chinese (govt-sponsored) hacker

]. As a test case, try sending a prankethmail to your own id. And check the “original mail” option in gmail.
Ah. The post digresses from its core issue. Lets come back to the main point, shall we.
An important reason to gmail, not blocking my self-spam, was that it trusts my id, and its servers.
I tried sending the same text through pranketh. Guess what. It thrashes the mail left and right, before it even leaves their servers. Why?. They use Aksimet.

My advice to gmail [ In the remotest probability that Matt Cutts is reading this,], and other mail providers is this.
Google’s servers are checking all our mail-contents for generating their automated ads and stuff anyways. So, there is no illusion of privacy. So, the next time, I’m sending a mail, check the contents before hand. Warn me if its spammish. Keep the thresholds appropriately such that I’m not regularly annoyed with these warnings.
If all smtp servers start this routine, we can see at least some major changes.

1. All the worlds emails would take a longer time to reach their destination. [ There's got to be some catch.This is it]

2. Say I send one mail to a big bunch of people, it’d be scanned for spam-behaviour only once. Then some certification can be piggy-backed along, saying its reliable, and not spam. The experts can handle that bit. Not too difficult.

3. Botnets prevention. Say, some dumbo privately runs a smtp server, and has been been subjected to a backdoor/trojan attack. And this is currently acting as a zombie sending out bunch of viagrish mails to innocent people, who’ve left their email id’s lying out in the open.

I’m not saying you give up your earlier approach. That’d be foolish. But if its absolutely obvious that a mail is spammy[ self-spam for eg.]. Block it before it leaves your grounds.

Now, a general warning to all those who think spams are obnoxious. Your bigdaddydog@rediffmail.com might be the prettiest email id around. But don’t leave it on some arbit website, for all the world to see. One syntax-based text crawler and you get thousands of them.
Believe me, some of these spammers are millionaires[ Not the Nigerian kind]. And run their business professionally. And have awesome technical expertise too.
If anything, don’t make their job easier. Let them just fight it out with the big-guys[ yahoo, google, msft et al].

I reiterate. If you desperately want to put your email id on the net, use images like these.

And, a word of caution. Even this is not safe. Within 1-2 years, google image search is going to search the contents in images. And character recognition. Piece of cake.
So, what do you do next. Do not fear, I have the ideas.

1. When you must, put your email id’s with re-captcha. I’d written a post about this some time back. My email id through this schema would be abhi…@gmail.com .Go to their website and register for their free service. The only reason this idea is safe, however, is because Spammers, like all of us, are average-ramesh hard-working people. They do not have time to fill your captchas. Where as, your friends and people who want to see your email id so desperately, do.

2. Use images, but this time, write them with 3-d blocks. Even by extreme image processing hacking standards, this is nearly safe for 5-6 years.

3. Do not put email id’s on the internet.

This is probably the first in a series of spam-related posts to come.

P.S: I kinda remembered the first word of my blog’s title, and how I hadn’t paid any attention to it, for the past few months.

And, let me clarify. I love gmail.