Google understands that it has a big transparency problem. That much is clear from a blog posting today by Udi Manber, the engineer in charge of its search quality. The problem is, there is very little the company can do about it.

The problem stems from the “black box” nature of its search ranking algorithms. No one knows why some websites come out higher than others in response to certain queries, and the methodology is always changing. That creates headaches for anyone who relies to any significant degree on traffic from Google (i.e., just about every business on the Web.)

Requests for more clarity about how the company arrives at its rankings have always been brushed off with a curt “no comment”: Googlers simply say that giving any of their tricks away would help people game its system.

So it’s highly revealing that Manber felt compelled to write. He rehearses the old line about the need for secrecy, then says:

But being completely secretive isn’t ideal, and this blog post is part of a renewed effort to open up a bit more than we have in the past. We will try to periodically tell you about new things, explain old things, give advice, spread news, and engage in conversations.

“Conversations”? Google? That would really be a culture change.

The trouble is, Manber has little to say other than platitudes. He promises a peek inside Google’s “black box” but gives nothing much away.

This brings to mind something that Harvard professor Viktor Mayer-Schoenberger had to say at the FT’s digital media conference in London earlier this year. He argued that there are two possible futures for Google. In one, a stronger competitor emerges in search. In the other, an unchallenged Google comes under the sway of regulators who feel compelled to act to protect the many Web citizens who are forced to rely on its opaque rankings.

Manber’s blog post is a clear sign that Google understands the risks of being seen as an internet monopolist that has no one to answer to. Whether he can change that perception is doubtful.

May 22nd, 2008 in Internet | Permalink