Archive for December, 2007

The top posts of 2007: number 2

Monday, December 31st, 2007

Over the holiday period, the physics arxiv blog is re-running the most popular blogs (by page views) of 2007.

Breaking the Netflix prize dataset
27 November

Netflix data

Hell, this is good work. In October last year, Netflix released over 100 million movie ratings made by 500,000 subscribers to their online DVD rental service. The company then offered a prize of $1million to anyone who could better the company’s system of DVD recommendation by 10 per cent or more.

Of course, Netflix assured everybody that the data had been anonymized by removing any personal details.

That turns out to have been a tad optimistic. Arvind Narayanan and Vitaly Shmatikov at the the University of Texas at Austin have just de-anonymized it.

Here’s how: turns out that an individual’s set of ratings and the dates on which they were made are pretty unique, particularly if the ratings involve films outside the most popular 100 movies. So it’s straightforward to find a match by comparing the anonymized data against publicly available ratings on the Internet Movie Database (IMDb).

That’s exactly what Narayanan and Shmatikov have done. And get this, once the match is made, it immediately links the user to the any private ratings on the Netflix database.

“Given a user’s public IMDb ratings, which the user posted voluntarily to selectively reveal some of his (or her; but we’ll use the male pronoun without loss of generality) movie likes and dislikes, we discover all the ratings that he entered privately into the Netflix system, presumably expecting that they will remain private.”

So what, I hear ya ask.

Here’s what the dynamic duo have to say about one person whose data they outed:

First, we can immediately find his political orientation based on his strong opinions about “Power and Terror: Noam Chomsky in Our Times” and “Fahrenheit 9/11.” Strong guesses about his religious views can be made based on his ratings on “Jesus of Nazareth” and “The Gospel of John”. He did not like “Super Size Me” at all; perhaps this implies something about his physical size? Both items that we found with predominantly gay themes, “Bent” and “Queer as folk” were rated one star out of five. He is a cultish follower of “Mystery Science Theater 3000”. This is far from all we found about this one person, but having made our point, we will spare the reader further lurid details. “

So Netflix may have inadvertently revealed the political affiliation, sexual orientation, BMI and God-knows-what else of 500,00 of their subscribers. Way to go!

Next up the mobile phone datasets we talked about a coupla weeks back

Ref: : Robust De-anonymization of Large Datasets (How to Break Anonymity of the Netflix Prize Dataset)

The top posts of 2007: number 3

Sunday, December 30th, 2007

Over the holiday period, the physics arxiv blog is re-running the most popular blogs (by page views) of 2007.

Invasion of the jivin’ nanoshrooms
24 August

Convertin’ a constant force into an oscillatin’ one is a useful trick. Ya’ll seen em: gravity-powered pendulums and wind-powered turbines for example, them both set machines a-spinin and a-swingin by exploitin’ a constant force.

Them machines might work sweetly at macroscopic scales but ain’t nobody cracked it on the nanoscale even though nanobods are a-chompin at the bit to reproduce this trick. The trouble is that gravity ain’t strong enough at this level and as for wind, who you kiddin?

That leaves only tricky-dicky forces from the dizzy world of electrostatics and magnetics and these are so poorly understood on tiny scales that them nanobods are still a-wondrin and a-ponderin over how to harness them.

But Hyun “Mighty” Kim and his crew at the University of Wisconsin-Madison say they cracked it.

Their device is a kinda nano-mushroom that stands between the plates of a capacitor, in a constant DC field.

Give the mushroom a push and it leans towards the source electrode where electrons tunnel across into the mushroom head. The DC field exerts a force on this extra charge on the ’shroom, pushing it towards the drain electrode where the electrons jump ship. The force disappears and the mushroom’s stiffness sends it swinging back to the source again like metronome, and the process starts again.

Voila! A nanomechanical oscillator that converts a a constant force into an oscillation.

Them nanobods are gonna be cockahoop over this one, betcha!

Ref: Self Excitation of Nano-Mechanical Pillars

The top posts of 2007: number 4

Saturday, December 29th, 2007

Over the holiday period, the physics arxiv blog is re-running the most popular blogs (by page views) of 2007.

Einstein and the greatest scientific fraud of the 20th century
24 September

In 1926, when the scientific world was still a-puzzling and a-wondrin over the wave-particle duality of light, Einstein asked a pal, Emil “Hurry” Rupp, to conduct an experiment that would settle the matter. If anyone could do it, thought Einstein, it was Rupp who was considered the latest and greatest experimental physicists of the day.

The experiment involved so-called canal rays produced in a gas discharge tube. When an electric field passes across a gas at low pressure, the tube shines ‘n’ glows due to the movement of electrons from the cathode to the anode (so-called cathode rays). But if a hole is made in the cathode, so-called canal rays appear start a-streamin and a-strayin’ through the hole in the opposite direction to the cathode rays.

The question that Einstein asked Rupp to resolve was whether the light from canal rays was wave-like or particle-like.

The matter was settled when Rupp said he could see with his own eyes that the light formed interference patterns. That proved it must be wave-like. Einstein presented the result as evidence in his own interpretation of quantum mechanics.

But nobody else could see these interference patterns and physicists soon began to doubt the veracity of Rupp’s work. In 1935 he publicly retracted five of his scientific paper in the previous year claiming to be suffering from “psychasthenia linked to psychogenic semiconsciousness”.

Rupp turned out to be the greatest scientific fraudster of the 20th century, surpassing even Hendrick Schoen from Bell Labs in his boldness and audacity (and mental health). It later emerged that everything Rupp had done in the previous ten years was a fraud.

Einstein swallowed it hook, line and sinker.

Now Jeroen “Kongen” van Dongen at the Institute for History and Foundations of Science at Utrecht University in the Netherlands has re-analysed Einstein’s role in the controversy. He says the evidence “suggests a strong theoretical prejudice on Einstein’s part” which led him to ignore evidence that Rupp’s the experiments were a sham and a-rigged.

Poor old Einstein! But I know ya’ll will forgive him

Ref: Emil Rupp, Albert Einstein and the Canal Ray Experiments on Wave-particle Duality: Scientific Fraud and Theoretical Bias

And: The Interpretation of the Einstein-Rupp Experiments and their Influence on the History of Quantum Mechanics

The top posts of 2007: number 5

Friday, December 28th, 2007

Over the holiday period, the physics arxiv blog is re-running the most popular blogs (by page views) of 2007.

Mathematics: the foundation of reality
2 October

“Our universe is not just described by mathematics — it is mathematics.” That’s the conclusion of Max “Peg Leg” Tegmark, an astrophysicist at the Massachusetts Institute of Technology. But he ain’t nuts, even if he sounds as if he’s a couple of planets short of a solar system.

His argument is actually kinda convincing. In a paper that he says is a director’s cut of an article that he wrote for New Scientist which in turn was based on an earlier paper of his called The Mathematical Universe, he starts with a question: if we accept that the universe has a reality independent of ourselves, then what sort of reality is it?

Peg Leg Tegmark argues that it has to be free of any kinda of physical or cultural bias so that it is the same for all aliens wherever they may be in the universe. The only logical system that fits this description is the one that underlies mathematics, he says. Therefore the universe is mathematics.

Peg Leg Tegmark reckons that this line of thought leads to a number of curious predictions that are actually testable by observation. F0r example, he says that a measurement of the distribution of dark energy within our universe would be a decent test.

Just how we might make that measurement and what exactly we would be looking for is harder to say.

Still we can hardly expect the trifling details behind the actual observation and measurement of the universe to trouble a thinker like Tegmark. All in all, his paper makes a fine addition to the general framework of untestable philosophy cosmology.

Ref: Shut Up and Calculate

The top posts of 2007: number 6

Thursday, December 27th, 2007

Over the holiday period, the physics arxiv blog is re-running the most popular blogs (by page views) of 2007.

Cellphone records reveal new patterns of human activity
29 October

Switch yer mobile phone on and it checks into the local network giving your location and the time you were there. The network also records the calls you make, their frequency, duration and to whom you make them plus wherever they happen to be too.

Multiply that by the entire popualtion (mobile phone penetration approaches 100 per cent in many western countries) and you’ve got a data set that can give an unprecedented insight into the links between people and the way they move and behave.

Albert-Laszlo “Bar” Barabasi at Northeastern University in Boston and a few pals have been a-grindin’ and a-crunchin’ the data from several million cellphones and are now revealing what they’ve found.

Turns out the data can be used to identify friends and family (from the frequency and duration of calls and whether they are reciprocated), they can show how social groups evolve and how they fall apart.

The data can also suggests how to monitor the way people behave in emergencies in realtime. For example, a pile up on the freeway causes lots of rapidly moving phones grind to a halt, a few call the emergency services while others call the office/spouse/lovers. Spot that pattern and its a pretty good indication that an event has occurred. Location information might even help to determine exactly where the accident took place.

Barbara Rasi has also found previously unknown patterns in human behaviour. For example, although the number of people making calls varies hugely during the day and night, the percentage who are on the move (ie who make consecutive calls from different lcoations) is always roughly the same. And the average distance they travel between calls in a half hour period is also stable at about 6 km. He says this is just the beginning of what will be possible with this kind of data.

What ya’ll want to know is how anonymous is it? The message is: Big Brother is watching…but he only has access to anonymized data.

But ah don’t buy it. It wasn’t so long ago that the raw data from search engines was thought to be anonymised if a person’s name was removed from it. But that myth was exploded by journalists from the New York Times who tracked down one individual using only her AOL search records.

How long before we see a similar expose with supposedly anonymized mobile phone records?

Ref: Uncovering Individual and Collective Human Dynamics from Mobile Phone Records

The top posts of 2007: number 7

Wednesday, December 26th, 2007

Over the holiday period, the physics arxiv blog is re-running the most popular blogs (by page views) of 2007.

The frightening prospect of flu
23 November

Flu transmission

Bird flu may get all the headlines but the number of deaths it causes each year is currently measured in hundreds. The real killer, the one that should set yer spine a-shiver, is ordinary fly which kills hundreds of thousands each year.

With winter nearly upon us up here in the northern hemisphere, the spectre that we’ll be a-shakin and a-sweatin our way through a flu pandemic in the coming months is raising its ugly head again.

So what can science tell us about the way pandemics spread? Not enough, according to Gerardo Chowell from Arizona State University in the US and his friend Hiroshi. They’ve posted a comprehensive review of epidemic science dating back to the 1918 Spanish flu epidemic that killed as many as 100 million people by some estimates. We’ve learnt an awful lot since then about the way flu spreads but the frightening thing about Gerardo’s review is how much more we need to understand.

It’s not just small things that are up in the air but sizable pieces of the jigsaw. We still don’t know some basic probabilities associated with infection. For instance, given that an individual is infected with flu, what are the chances that the disease will manifest itself clinically? And given that the disease has manifested itself clinically in an individual, what are the chances of that person dying. And if a virus can be caught from a number of diffferent host species (as it might eventually be with bird flu) what is the probability of transmission?

Without a good understanding of these kindsa factors, it’s gonna be difficult for authorities to plan an effective response to a flu pandemic.

Mah advice? Stock up on food and water.

Ref: Quantifying the Transmission Potential of Pandemic Influenza

The top posts of 2007: number 8

Tuesday, December 25th, 2007

Over the holiday period, the physics arxiv blog is re-running the most popular blogs (by page views) of 2007.

Game theory and the future of Adwords
8 September

Ain’t Google Adwords a miracle o’ modern science? Here’s a system that searches your web page for keywords, hunts for advertisers who wanna have their message displayed next to these keywords and then auctions the advertising space to the highest bidder. All in the twinklin’ of an eye. Adwords is so good that it’s made Google millions or billions or zillions (who’s countin’?)

What can game theory tell us about the future of this digital auction house? According to Sudhir “Secrets” Singh, an electrical engineer at the University of California, Los Angeles and his buddies, game theory suggests that everybody would benefit by the emergence of a new kind of online business that exploits the inefficiencies of the Adwords system.

Here’s the thinkin’. There is a limit to the number of advertising slots on each page and this leaves advertisers a-scramblin and a-scufflin for spaces next to the most popular keywords. Inevitably, the advertisers who miss out are left a-sobbin and a-wimperin at the end of the day. The sobbers and wimperers are perfect fodder for the new businesses.

According to Secrets Singh, these businesses are gonna buy up popular keywords and then resell them to advertisers who want to ensure they don’t miss out on a slot. Secrets and his crew have built a model of this new market and calculated the revenues and payoffs for all parties. The model assumes that this game is a symmetric Nash equilibrium, meaning that everybody plays in the same way (or at least that nobody has anything to gain by unilaterally adopting a different strategy) .

Under these circumstances everybody wins–the auctioneer, the reseller and the bidders.

Ain’t that a bee-yoo-tiful story? So ya’ll gonna see resellers springin up all over the place soon. Secrets Singh and his buddies even have a start up called and there ain’t no prizes for guessing what that’s gonna be doin’.

They’ll already be a-dreamin and a-wondrin about the zillions they gonna make.

I know ya’ll lurv a happy endin. So ya’ll keep quiet about Adwords not being a symmetric Nash equilibrium. Y’hear?

Ref: Capacity Constraints and the Inevitability of Mediators in Adword Auctions

The top posts of 2007: number 9

Monday, December 24th, 2007

Over the holiday period, the physics arxiv blog is re-running the most popular blogs (by page views) of 2007.

The incredible galactic foxtrot
30 August

Ya’ll know the laws of physics are symmetrical–they have no preferred direction. The speed of light (and more or less everything else) is the same whichever way it is pointin’, right?

So it’d be surprisin’ if the universe turned out to be asymmetric, shockin’ if it had a prefered axis and stunnin’ if it had a direction in which things were different.

But it’s beginning to look that way. Two years ago, you could hear astro-jaws dropping all over the planet when a couple of upstarts suggested that data from a giant space-thermometer called the Wilkinson Microwave Cosmic Anisotropy Probe or WMAP looked as if it were aligned in a certain direction.

A-babblin and a-brayin, critics pointed out that the alignment matched the galactic co-ordinate system and so was probably an imprint of the Milky Way. Subtract this properly and the alignment would go away.

But it didn’t. And that left theorists a-scratchin their heads and a-rubbin their chins in amazement.

Now they gotta do a whole lot more a-broodin and a-worryin. Michael “Bongo” Longo at the University of Michigan has spotted another kind of alignment by a-meterin and a-measurin the orientation of 200,000 elliptical galaxies. That’s a lotta galaxies and it turns out they’re all aligned in the same direction too. We just ain’t noticed before now.

That’s put a ‘gator in the pool.

So what’s been causin this cosmic foxtrot? Ain’t nobody sure but we’re gonna be hearin a lot more about it in the weeks and months to come. Betcha!

Ref: The Axis of Opportunity: The Large-Scale Correlation of Elliptical Galaxies

Apples ‘n’ pears

Sunday, December 23rd, 2007

 The best of the rest form the physics preprint server this week:

A Novel Mechanism for Outbursts of Comet 17P/Holmes and other Short-period Comets

The Production Rate and Employment of Ph.D. Astronomers

Expanding and Improving the Search for Habitable Worlds

The Price of Anarchy in Transportation Networks: Efficiency and Optimality Control

Extinction risk and structure of a food web model

In case ya missed ’em

Saturday, December 22nd, 2007

This week’s posts:

 Why MHD propulsion won’t work

The puzzling presence of DIBs

The sunset on HD 189733b

Tune into the snowflake channel

Black holes may convert dark matter into cosmic rays