Tuesday, December 26, 2006

More Caltech bragging rights: patents and PhDs

Guess which university completely dominates all others in patents issued, when normalized to the size of institution? My graduating class was 186 -- compare to MIT (about 1000) and Stanford (about 1600).

2005 USPTO university patent rankings:

1) 10 campuses of the University of California System, 390 patents
2) Massachusetts Institute of Technology, 136 patents
3) California Institute of Technology, 101 patents
4) Stanford University and the University of Texas, each with 90 patents each.

For Nobel prize domination, see here.

For ranking by percentage of undergraduates going on to complete a PhD, see here and here. (I think I saw this on Dave Bacon's blog a while ago.) Caltech leads with about 50% of all undergrads going on to earn a doctorate. Reed College, MIT, Harvey Mudd, Swarthmore, etc. all do pretty well. Interestingly, Yale does well in the humanities, but Harvard is not to be found on any of the lists. That makes their kids the smartest of all, as they probably go directly to Goldman Sachs or hedge funds with no wasted time :-)

To be honest, graduate school seemed a lot easier than undergrad for me. A typical load at Caltech might be 5 technical courses and 1 humanities-social science course - for example, 3 physics classes, a math class, a CS class, and, for relaxation, something like history or economics ;-) In grad school 3 physics classes at the same time was a typical load.

Sunday, December 24, 2006

On the benevolence of financiers

An interesting comment on a previous post:

[Me:] "I guess the other thing to add is that not all financial games that make money for a hedge fund necessarily lead to more efficient resource allocation in the overall economy."

[Commenter:] Well, "all" is a high benchmark. But at least as high a percentage of the "financial games" that hedge funds use drives efficiency gains as whatever it is that start-ups do. In fact, because the winnowing process is more efficient (better measurement), I'd bet that the net increase in efficiency from hedge fund activities is greater than that generated by tech start-ups.

But, as always, specifics help. I can't think of a single major hedge fund strategy which does not improve efficiency for the economy as a whole.

I suspect the commenter, although anonymous, is a hedge fund manager, so we should take his opinion seriously :^)

Let me give a simple example of a case where financial activities that made hedge funds a lot of money did not in any way increase economic efficiency. At the end of the tech bubble, a lot of traders strongly believed that internet stocks were ridiculously overvalued. You might imagine that hedgies, with their ability to short, might play an important role in popping an obvious speculative bubble. But, the reality turned out to be the opposite: as long as they felt they could make a short term gain on riding the bubble, they were happy to do so, despite their strong belief that it was a bubble, with significant distortionary effects on the economy. I know from personal contacts that hedge funds actually behaved this way, and it is supported by subsequent academic studies. (See related discussion here; indeed, according to the famous Keynes observation about beauty contests, it matters not whether you think it's a bubble, but rather what you think others in the market believe!)

Of course, and this is the main point, a trader's job is not to "allocate resources most efficiently in the economy" but rather to make as much money for their investors as possible. Only an extreme interpretation of "efficient markets" would imply that these two goals are one and the same!

From the viewpoint of the old Adam Smith quip:

It is not from the benevolence of the butcher, the brewer, or the baker that we expect our dinner, but from their regard to their own interest

we might conclude that there is no reason to differentiate between forms of economic activity. However, some people are genuinely altruistic and take part of their compensation in satisfaction that their work helps others. For example, many people in startups are idealistic, and part of the attraction of their work is that they are changing the world for the better. The same goes for academic scientists.

Although financial activities play an important role in the economy, as I have emphasized, I suspect the prime motivation for finance as a career choice is maximization of remuneration, not altruism :-)

Saturday, December 23, 2006

Frank Gehry: the lion in winter

A poignant interview with architect Frank Gehry. You might imagine him as an egocentric genius, but instead he comes across as a thoughtful leader and manager. The interview reinforces how important people and management skills are to the completion of any complex project. Mad geniuses have their place, but it takes another set of skills to get big things done.

Iconic figures in the modern world seem to incorporate two sides: the flamboyant outward persona that gets investors, backers and the press excited, and the internal face that actually runs the organization. Confidence in the individual leader may get the project going, but ultimately that leader has to be effective on the inside.

WSJ: Frank Gehry is 77, white haired, paunchy, and when we talked one afternoon in late autumn the topics of age and death never seemed far off. Mr. Gehry is, of course, one of the world's great architects, creator of the Guggenheim museum in Bilbao and enough of an icon to have been among the personalities featured in Apple's "Think Different" campaign.

Describing what it takes for him to accept a commission, Mr. Gehry says, "The determining factor is: Can I get it done while I am still alive?" Explaining why he doesn't build houses any more, Mr. Gehry says, "They involve a lot of personal hand holding. I guess at my age I don't have the patience."

Probably more than most architects, one sees Mr. Gehry's buildings -- buildings that have been described as resembling ruffling sails or looking like they are melting -- and has a sense that there is a single personality behind them.

...Because Mr. Gehry's buildings are as much feats of engineering as they are of architecture, it is strange to walk into his office and notice that there are no computers. Mr. Gehry's office is surprisingly spartan. There is a desk and there is a conference table and on one wall are photographs of friends. Sitting at his conference table and speaking of technology, Mr. Gehry volunteers, "I don't know how to turn on the DVD. I barely can use the technology in my car. It's a wonder I don't get into an accident."

The actual physics and engineering of Mr. Gehry's buildings are managed by teams of employees. Some 150 people work for him, and when Mr. Gehry talks about what exactly he does that leads to a building, it seems that he is almost more a manager of personalities and processes than he is someone who sits down with pencil and paper. "The building process is complicated. You have an idea, an image, a dream. You start to fantasize. You've got to get that feeling through thousands of hands to build a thing. Meetings, bureaucracy, accounting."

Considering that Mr. Gehry's buildings appear almost completely indifferent to conventions, I expected Mr. Gehry to be something of an egomaniac. Instead he turned out to be surprisingly modest. Describing a hotel in Spain that he just completed, Mr. Gehry said, "the rooms are comfortable," and when talking about the Guggenheim in Bilbao, he said that he was relieved that the people of the city liked it. The only time Mr. Gehry showed strong pride was when he was discussing being a good employer.

Most architects of Mr. Gehry's stature can staff the lower rungs of their office with volunteers and interns. "I am very proud," he says and sits up at the conference table. "Everybody gets paid. Everybody here is paid. There's no freebie interns. I've never done that. A lot of my colleagues do that, but that offends me so I've never done that." Like only one or two other topics in our conversation, this issue of how he cares for the people who work for him is something that seems to get him excited. "I am very proud," he says, again referring to his employees, "that they always get cost of living index raises and bonuses and more."

Another aspect of Mr. Gehry's old-fashioned virtue is his concern for what will happen to his employees once he dies. When I ask him if his age adds greater urgency to picking projects and finishing projects, Mr. Gehry says, "No. I am not that megalomaniac. No, I think the day will come and . . ."

Then apropos of very little in particular, he says, "What I am interested in is, since it's 150 people here and a lot of people's lives and futures depend on it, how do you create a succession?" Again Mr. Gehry sounds passionate. "There's a way to leave it and pull the plug and I am fine and they" -- referring to his employees -- "lose." As part of managing for his own death, Mr. Gehry has been trying to build the public personae of the people who work for him, trying to direct some of the limelight that seems always oriented towards him in their direction. In the catalogs and exhibits devoted to his work, he makes sure to mention the people who worked with him on his various projects.

As the interview wound down and the theme of age began to seem a more and more dominant part of our conversation, Mr. Gehry started to talk about some of the problems of getting older. Because he cannot program and has to work through others in order to engineer a building, he said that he is in some ways obsolete. Referring to these computer skills, he asked, "If I knew all that, could I be a better architect?"

...Asked how he handled these limits he saw for himself, he said, "I keep going. Keeping going is important to me and not to get sidetracked and to get caught up in self-pity."

Friday, December 22, 2006

In defense of hedge funds

The article below gives a well-reasoned defense of hedge funds. The money line is, for me, the following:

As any newspaper reader knows, technology firms are the leading edge of the U.S. knowledge economy; they made possible the productivity revolution of the past decade. But the same could just as well be said of hedge funds, which allocate the world's capital to the companies, industries, and countries that can use it most productively.

The argument is similar to the (correct) observation that you need venture capitalists and entrepreneurs to have a startup ecosystem. My only problem with the current situation is that I suspect a lot of managers are getting paid without generating much alpha. As the industry matures, and replication of vanilla strategies better developed, I imagine we'll see the fees come down.

I guess the other thing to add is that not all financial games that make money for a hedge fund necessarily lead to more efficient resource allocation in the overall economy. It's much more obvious to the average person or journalist that a technology company (even if it fails) is trying to do something which advances society.

Foreign Affairs: Imagine two successful companies. Both are staffed by very smart people; both are innovative; both have an impact far beyond their industry, improving the productivity of the capitalist system as a whole. But the first, based near San Francisco, is the subject of adoring newspaper profiles, whereas the second, based in the New York area, is usually vilified.

Actually, you do not have to imagine any of this, because it describes a double standard that already exists. The first company in the story is a technology firm; the second is a hedge fund. As any newspaper reader knows, technology firms are the leading edge of the U.S. knowledge economy; they made possible the productivity revolution of the past decade. But the same could just as well be said of hedge funds, which allocate the world's capital to the companies, industries, and countries that can use it most productively.

Of course, that is not how hedge funds are viewed most of the time. The recent implosion of Amaranth Advisors -- a hedge fund that lost $6 billion in a matter of days thanks to one Ferrari-driving 32-year-old trader (and his greedy bosses' abandonment of proper risk management) -- has rekindled the fears that attended the collapse of Long-Term Capital Management in 1998, an event that even then Federal Reserve Chair Alan Greenspan believed "could have potentially impaired the economies of many nations, including our own."

...In the end, the critics of hedge funds would do well to remember why this sector has emerged as such a force. Until the late 1960s, the financial world was quaintly stable: exchange rates were inflexible, interest rates were regulated, and the whole system was anchored by a fixed gold price. But that world collapsed when inflation drove the dollar off the gold standard and currencies and interest rates began to float; from then on, it became impossible to amass savings without facing financial uncertainty. Tools for coping with that uncertainty -- deep markets in futures, options, and other derivative instruments -- sprang up in response to the newly volatile environment. And hedge funds emerged as the masters of these tools, providing quasi insurance to investors and firms and introducing a healthy dose of contrariness into financial markets. For this, they are accused of generating risk. But their real systemic function is to manage it -- and it is their very success in doing so that has generated both their profits and their phenomenal growth.

Monday, December 18, 2006

Machine Dreams

We're on break right now, so I have time to do some deeper reading. I'd like to recommend the book Machine Dreams by the economist and intellectual historian Philip Mirowski. Mirowski ruffled quite a few feathers in economics with his earlier book More Heat than Light: Economics as Social Physics, Physics as Nature's Economics, in which he argued that much of the mathematical and conceptual framework of neoclassical economics was lifted from 19th century physics. In particular, he argued that maximization of utility was inspired by ideas about energy, and that market equilibrium was inspired by thermodynamics. Origins aside, whether these ideas are useful for the description of complex, nonlinear systems comprised of thinking participants is another question, addressed directly in this book.

Machine Dreams is, if anything, more ambitious than the earlier work. In it, Mirowski traces the influence of ideas concerning information and computation, largely developed by figures like von Neumann, Turing, Shannon and Szilard, on the field of economics since the 1930's. Mirowski refers to these individuals as cyborgs, and their area of interest as the cyborg sciences. He adopts an amusing tone throughout the 600 pages of his book, even as he delivers devastating blows to sacred cows of the economics orthodoxy.

This is a controversial book because it demolishes not just the conventional history of the discipline, but its foundational assumptions. For example, once you start thinking about the information processing requirements that each agent (or even the entire system) must satisfy to find the optimal neoclassical equilibrium points, you realize the task is impossible. In fact, in some cases it has been rigorously shown to be beyond the capability of any universal Turing machine. Certainly, it seems beyond the plausible capabilities of a primitive species like homo sapiens. Once this bounded rationality (see also here) is taken into account, the whole notion of optimality of market equilibrium becomes far-fetched and speculative. It cannot be justified in any formal sense, and therefore cries out for experimental justification, which is not to be found.

But there is more. Mirowski reveals that tenets of rationality and utility maximization were already at odds with results of game theory experiments conducted at RAND as early as the 1950s. He traces von Neumann's (vN's) influence on the discipline, which he claims has been deliberatedly ignored and obfuscated in the official histories. In his summary, he writes

This scientific titan [vN], who could only spare a vanishing fraction of his intellectual efforts upon a science he regarded as pitifully weak and underdeveloped has somehow ended up as the single most important figure in the history of 20th century economics. This mathematician who held neoclassical theory in utter contempt throughout his own lifetime has nonetheless so bewitched the neoclassical economists that they find themselves dreaming many of his formal models, and imperiously claiming them for their own. This polymath who prognosticated that "science and technology would shift from a past emphasis on subjects of motion, force and energy to a future emphasis on subjects of communications, organization, programming and control," was spot on the money.

Mirowski's book has drawn significant attention within the economics community. I suggest the following review by E. Roy Weintraub, a mathematical economist at Duke (Journal of Economic Behavior & Organization Vol. 53 (2004) 419–434):

Philip Mirowski is a singular historian of economics. Every one of his works has made a difference in our understanding of the development of economics. He is brash, uncompromising, and dedicated to producing magnificent historical studies. Nevertheless he raises his colleagues’ blood pressures because his work has both transcended and transformed the subdiscipline, and few are intellectually flexible enough to enjoy rethinking accepted ideas.

...In his new work, Machine Dreams: Economics Becomes a Cyborg Science, Mirowski (2002) reconstructs the history of neoclassical economics to the modern period, writing that history against and intertwined with the emergence of the cyborg sciences concerned with information—computer science, cybernetics, statistics, operations research, game theory, gaming and simulation, etc.—in the World War II and subsequent Cold War period. His major historiographic point, continuous with his past writing, is that one cannot construct a coherent narrative of the emergence of modern neoclassical economics in the postwar period without looking outside economics proper for both the dramatis personae, and the punch lines. His use of methods from history, sociology, and anthropology frames the best that Science Studies has to offer to help one construct a compelling narrative.

...Overall, this is the single most important totalizing narrative of the history of economics that we have had in the last twenty years. Important does not however mean that it will be loved. Mirowski’s writing is vivid and forceful. He enjoys the sound of words and the rhythm of the sentences he constructs. He writes with a musical fluidity. Yet to say that Mirowski is verbally facile, sharp-tongued, and acerbic hardly does justice to the blood he draws from rapier slashes, and cleaver smashes, to the august and famous. It is, given the author’s critical program, an angry book.

Withal, Mirowski is a scholarly treasure. There are few of us who make an immense difference, who see things differently and can maintain that vision through a sustained scholarly life, and who can show others how to make that vision their own. We need to take this book seriously.

For a meta-review of reviews, see Boland, and for a response by Mirowski to several critics, see here (contribution to a symposium on Machine Dreams published by the Journal of Economic Methodology).

Indexing for Googlers

How the Googlers prepared their new millionaires for the investment world: mass education on the merits of indexing.
SF Magazine: As Google’s historic August 2004 IPO approached, the company’s senior vice president, Jonathan Rosenberg, realized he was about to spawn hundreds of impetuous young multimillionaires. They would, he feared, become the prey of Wall Street brokers, financial advisers, and wealth managers, all offering their own get-even-richer investment schemes. Scores of them from firms like J.P. Morgan Chase, UBS, Morgan Stanley, and Presidio Financial Partners were already circling company headquarters in Mountain View with hopes of presenting their wares to some soon-to-be-very-wealthy new clients.

Rosenberg didn’t turn the suitors away; he simply placed them in a holding pattern. Then, to protect Google’s staff, he proposed a series of in-house investment teach-ins, to be held before the investment counselors were given a green light to land. Company founders Sergey Brin and Larry Page and CEO Eric Schmidt were excited by the idea and gave it the go-ahead.

One by one, some of the most revered names in investment theory were brought in to school a class of brilliant engineers, programmers, and cybergeeks on the fine art of personal investing, something few of them had thought much about. First to arrive was Stanford University’s William (Bill) Sharpe, 1990 Nobel Laureate economist and professor emeritus of finance at the Graduate School of Business. Sharpe drew a large and enthusiastic audience, which he could have wowed with a PowerPoint presentation on his “gradient method for asset allocation optimization” or his “returns-based style analysis for evaluating the performance of investment funds.” But he spared the young geniuses all that complexity and offered a simple formula instead. “Don’t try to beat the market,” he said. Put your savings into some indexed mutual funds, which will make you just as much money (if not more) at much less cost by following the market’s natural ebb and flow, and get on with building Google.


The following week it was Burton Malkiel, formerly dean of the Yale School of Management and now a professor of economics at Princeton and author of the classic A Random Walk Down Wall Street. The book, which you’d be unlikely to find on any broker’s bookshelf, suggests that a “blindfolded monkey” will, in the long run, have as much luck picking a winning investment portfolio as a professional money manager. Malkiel’s advice to the Google folks was in lockstep with Sharpe’s. Don’t try to beat the market, he said, and don’t believe anyone who tells you they can—not a stock broker, a friend with a hot stock tip, or a financial magazine article touting the latest mutual fund. Seasoned investment professionals have been hearing this anti-industry advice, and the praises of indexing, for years. But to a class of 20-something quants who’d grown up listening to stories of tech stocks going through the roof and were eager to test their own ability to outpace the averages, the discouraging message came as a surprise. Still, they listened and pondered as they waited for the following week’s lesson from John Bogle.

“Saint Jack” is the living scourge of Wall Street. Though a self-described archcapitalist and lifelong Republican, on the subject of brokers and financial advisers he sounds more like a seasoned Marxist. “The modern American financial system,” Bogle says in his book The Battle for the Soul of Capitalism, “is undermining our highest social ideals, damaging investors’ trust in the markets, and robbing them of trillions.” But most of his animus in Mountain View was reserved for mutual funds, his own field of business, which he described as an industry organized around “salesmanship rather than stewardship,” which “places the interests of managers ahead of the interests of shareholders,” and is “the consummate example of capitalism gone awry.”

Bogle’s closing advice was as simple and direct as that of his predecessors: those brokers and financial advisers hovering at the door are there for one reason and one reason only—to take your money through exorbitant fees and transaction costs, many of which will be hidden from your view. They are, as New York attorney general Eliot Spitzer described them, nothing more than “a giant fleecing machine.” Ignore them all and invest in an index fund. And it doesn’t have to be the Vanguard 500 Index, the indexed mutual fund that Bogle himself built into the largest in the world. Any passively managed index fund will do, because they’re all basically the same.

When the industry sharks were finally allowed to enter the inner sanctum of Google, they were barraged with questions about their commissions, fees, and hidden costs, and about indexing, the almost cost-free investment strategy the Google employees had been told delivers higher net returns than all other mutual fund strategies. The assembled Wall Streeters were surprised by their reception—and a bit discouraged. Brokers and financial planners don’t like indexed mutual funds for two basic reasons. For one thing, the funds are an affront to their ego because they discount their ability to assemble a winning portfolio, the very talent they’re trained and paid to offer. Also, index funds don’t make brokers and planners much money. If you have your money in an account that’s following the natural movements of the market—also called passive investing—you don’t need fancy managers to watch it for you and charge big bucks to do so.

Brin and Page were proud of the decision to prepare their staff for the Wall Street predation. And they were glad to have launched their company where and when they did. What took place in Mountain View that spring might have never happened had Google been born in Boston, Chicago, or New York, where much of the financial community remains at war with insurgency forces that first started gathering in San Francisco 35 years ago.

The article goes on to trace the origin of indexing to a group at Wells Fargo in San Francisco in the 1970s.

“San Francisco was the only place in the country where this could have happened,” says Bill Fouse, a jazz clarinetist in Marin County who was present when the first shots were fired in the investment rebellion. It was 1970, and revolution was in the air.

While hippies, dopesters, and antiwar radicals were filling the streets of America’s most tolerant city with rage, sweet smoke, and resistance, a quieter protest was brewing in the lofty, paneled offices of Wells Fargo. There, a young engineer named John Andrew “Mac” McQuown, Fouse (who like many musicians also happens to be a brilliant mathematician), and their self-described “skeptical, suspicious, careful, cautious, and slow-to-change” boss, James Vertin, were taking a hard look at the conventional wisdom that for a century had driven American portfolio management.

Bank trust departments across the country were staffed by portfolio managers who, as I did at the time, believed that they alone possessed the investment formula that would enrich and protect the security of their customers. “No one argued with that premise,” Fouse recalls.

But McQuown suspected they were pretty much all wrong. He had met Wells Fargo chairman Ransom Cook at an investment forum in San Jose, and at a later meeting at company headquarters, persuaded him that traditional portfolio management was merely an investment variation of the Great Man theory. “A great man picks stocks that go up. You keep him until his picks don’t work anymore and you search for another great man,” he told Cook. “The whole thing is a chance-driven process. It’s not systematic, and there’s lots we still don’t know about it and that needs study.” Cook offered McQuown a job at Wells and a generous budget to conduct research into the Great Man Theory and other schemes to beat the averages. McQuown accepted, and a few years later Fouse came on as well.

They couldn’t have been more different: Fouse, a diminutive, mild-mannered musician, and McQuown, a burly, boisterous Scot. The two were like oil and water—McQuown even tried to have Fouse fired at one point—but their boss, Vertin, was the one who really was in the hot seat.

“You have to understand, Vertin’s career was on the line,” Fouse recalls. “He was, after all, running a department full of portfolio managers and securities analysts whose mission was to outperform the market. Our thesis was that it couldn’t be done.” Proof of McQuown’s theory could lead to the end of an empire, in fact many empires. “The poor guy was under siege,” says Fouse. “It was a nerve-racking time.”

Vertin’s memory of those times is no less vivid. “Mac the knife was going to own this thing,” he once told a reporter. “I could just see the fin of the shark cutting through the water.” Eventually, the research McQuown and Fouse produced became so strong that Vertin could not ignore it. “In effect it said that almost everything that every trust department in America was doing was wrong,” says Fouse. “But Jim eventually accepted it, even knowing the consequences.”

In July 1971, the first index fund was created by McQuown and Fouse with a $6 million contribution from the Samsonite Luggage pension fund, which had been referred to Fouse by Bill Sharpe, who was already teaching at Stanford. It was Sharpe’s academic work in the 1960s that formed the theoretical underpinning of indexing and would later earn him the Nobel Prize. The small initial fund performed well, and institutional managers and their trustees took note.

By the end of the decade, Wells had completely renounced active management, had relieved most of its portfolio managers, and was offering only passive products to its trust department clients. And it had signed up the College Retirement Equities Fund (CREF), the largest pool of equity money in the world, and Harvard University, the largest educational endowment. By 1980 $10 billion had been invested nationwide in index funds; by 1990 that figure had risen to $270 billion, a third of which was held at Wells Fargo bank.

Eventually the department at Wells that handled index ing merged with Nikko Securities and was later bought by Barclays Bank, which created the San Francisco subsidiary Barclays Global Investors. Its CEO, Patricia Dunn, the scandal-tinged former chairman of Hewlett-Packard who had worked for 20 years at Wells Fargo, had been heavily influenced by indexing. Running Barclays, she became the world’s largest manager of index funds.

Fouse, now retired in San Rafael, explains why all this could have happened only in San Francisco. “When we started our research, almost all the trust clients out here were individuals with small accounts. Anywhere else, particularly on the East Coast, trust departments handled very large institutions—pension funds, university endowments, that sort of thing. If Mellon, Chase, or Citibank had done this research and come to the same conclusion, they would have in effect been saying to their large, sophisticated, and very lucrative clientele: ‘We’ve been doing things wrong for a century or more.’ And thousands of very comfortable investment managers would have been out of work.”

Saturday, December 16, 2006

Climate change: the straight dope

From an email exchange with a colleague who has spent significant time studying climate change:

> What is your confidence level in the following statements?

> 1) we know the sign of anthropogenic effects on global temperature

--> high

> 2) current models get the size of the temperature change in the next
> 50 years to within a factor of 2

--> medium

> 3) there are nearby nonlinear tipping points in the climate system
> (i.e., that anthropogenic effects can push us to within 50 years)

--> low (except for the methane channel which I am seriously concerned with)

> 4) climate modelers are clear-headed scientists with a mature grasp of
> the uncertainties in data analysis and modeling complex nonlinear
> systems, and NOT ideologically motivated people whose main skill is
> writing simple computer programs and press releases

--> highly variable - the real problem is "how do you know you have produced the best model "- there recently has been a lot of Markov chain analysis associated with trying to determine this.

And, an account of the "200 year" storm that hit Seattle December 14-15, at one point dropping 1 inch of rain in 45 minutes, and knocking out the entire power grid!

The Storm that Killed Starbucks:

Part I: The Deluge

For me, in Eugene, Oregon, the morning of December 14, 2006 started off perfectly normal. I woke up to my usual idiot snoring dog alarm clock and realized that,

oh yeah, today I drive up to Seattle for my annual celebration of consumer fetishism

by contributing capital to the Bellevue Square shopping center. Every since it got upgraded, I enjoy my one, over-the-top capitalistic moment of the year in that environment - plus the Parlor has nice pool tables and good beer. Moreover, by driving up alone, I could maximize my carbon footprint on the planet and also send foreign governments more money – seems consistent with the Christmas spirit to me.

I knew the weather would, well, suck, since this has been a fairly wet winter so far (even though its not even winter - this global climate change stuff sure is confusing). I was prepared. I have a decent vehicle (it has windshield wipers) and my cell phone is able to receive Doppler radar imaging maps. Going through Portland in moderate rain during the morning commute, I thought it might be a good idea to avert my eyes away from the crowded road and to my tiny, tiny cell phone maps. I noticed a big red area off the Olympic peninsula. Hmm, I said to myself, “that must be Santa because you never see red on the TV radar”. I would later discover, the red area was not Santa.

...I was headed toward the UW to, ironically enough, visit some colleagues in the Atmospheric Sciences department. I parked down near University Village (later to become Lake University Village) and, at about 3 pm, walked towards campus. The rain steadily was increasing. When I got to my colleagues office, there was a big sign on the door that read “Sorry, the end of the world is near, I have to find a Starbucks before it is too late”. I figured this was just a Christmas joke and trudged back to my vehicle (the one with the windshield wipers), which was now rapidly becoming a lone metallic island in a newly formed pool. I snorkeled my way into the drivers seat and extricated the metal from the pool and, in my infinite wisdom, headed towards Capital hill to change out of my wet clothes (well isn’t that what you do on Capital hill ?). Perfect timing – the deluge was upon me – one inch of rain in 45 minutes (that’s equivalent to 32 inches in a 24 hour period for you math geeks). In my true Seattle native spirit I remarked “Hey, what’s a little rain, it ain’t gonna kill me…”. But then (and where the hell is a Hummer when you really need one), I found myself going uphill (do you know how fast water runs downhill? – real fast) on 23rd ave through cascading muddy torrents. However, as I was not observing any cars in front of me actually disappearing, I figured it was okay – plus my windshield wipers were still working. And then came the Red Light at 23rd and Interlaken. At that moment, the red radar spot announced its arrival and it wasn’t saying “Ho Ho Ho”.

I have always wondered why that street was named Interlaken. And for those of you that don’t know – Interlaken is this really cool windy street through lots of dense trees and you don’t even realize your in the middle of the city – it’s a particularly good road for convertibles, but not on this particular day. As I was stopped, brakes tenaciously clinging to more water than road – I suddenly notice the reason for the name of the street as I was now between two lakes! In addition, there was no longer any traffic ahead of me on 23rd to interrupt the ever building cascade rushing downhill at relativistic speeds towards me. Quickly doing a physics calculation in my head (without even consulting my cell phone) I realize that I am better off taking the impending impact broadside, so I quickly took a right on Interlaken, applied some horsepower, made a really big rooster tail, and managed to get through this debris flow (I would later pay for this with a damaged electrical system and 2 new tires but my windshield wipers were still working).

The next few minutes were a blur of water water everywhere. It was surreal, roads were rivers, intersections were lakes, yards were temporary reservoirs, all routes looked equally bad. Fortunately, the rush hour deluge ended (do you suppose Nature did that timing on purpose?) and eventually I did manage to change my clothes on Capital hill,

...

But after donning dry clothes, I heard the ominous words “those #$%## Seahawks lost again” only to be followed by the more ominous words (although I am not sure that it is technically possible for something to be more ominous) “ha, ha, ha, the real Storm isn’t even here yet”.


Part II: The Aftermath

Ah yes – December 15th, the planned Bellevue Square expedition day – and the day all of Seattle will remember. I woke up, even without the aid of the snoring idiot dog, and tried to use something electrical. Then I tried to use something else, electrical. Then I went to turn on the TV to find out why my electricity didn’t work (I am real slow in the morning). Well then, if there is no power at home, the inconvenienced will just go shopping. I knew that the 520 bridge was closed and then everyone would be on I-90 enjoying a prolonged view of Lake Washington. I also surmised that driving North around the lake through Lake Forest Park and Bothell would not be wise as that area is prone to blow down. So I headed north on I-5 to go south on I-405 (yeah, I know it only makes sense in my head). As it turned out, it would have been faster to just drive around the hemisphere to get to Bellevue Square (well that destination was my mission!).

After I hit my fifth stop-and-go traffic incident on northbound I-5 I decided to turn on KIRO news. It took me some time to figure out how to get AM in my microprocessor controlled vehicle, but eventually I got some news and traffic. Basically the news said:

“No one has any power –Starbucks is down, I repeat Starbucks is down:” Basically the traffic said: “traffic is a mess” (presumably from all the confused individuals looking for a working Starbucks). But, what of Bellevue I wondered, did it survive? I had to find out – it was my mission. As I inched along, I continually wondered what the hell everyone was doing out here? Did we all have a collective brain fart and think we could just cruise around when all the infrastructure was down? Of course, being in your vehicle did provide a means of staying warm and getting the news. After all, who has a battery powered AM radio in their house anymore?

Eventually I did get to Bellevue Square, one dark intersection at a time, only to find out that the square had no power. One would have hoped that KIRO would have informed me that Bellevue had no power but no, instead I just heard stories of the guy that turned his hybrid Toyota into a coffee pot and all his neighbors lined up for hot coffee. In the true holiday spirit, I hope that guy made a hefty profit. In the past, when it was possible for people to do arithmetic by hand, one still might have been able to sell goods to customers with flashlights (I never go anywhere without my flashlight). But no, although the Square was open, none of the shops were. There were just hordes of ghostly figures wandering aimlessly in the semi-dark for no apparent reason as if they were all suddenly waiting for the moment The Power Came Back, and they could get on with their regular lives. “Damn”, I said, “my capitalistic Christmas spirit moment has been crushed by a failure of technology. Now where am I going to buy frivolous material goods for friends and family so they can resell them on e-bay a month later?” My spirit was crushed with my personal lamentations being interrupted by frantic souls screaming “do you know where there is an open Starbucks?”.

With my mission failed, I decided to head back to Eugene, via Capital hill, of course. As I got back in my vehicle I noticed that I was running low on gas. Noticing also that there was a high correlation between darkened traffic lights and vehicles parked in gas stations I realized that if the traffic lights ain’t working, the gas ain’t pumping. Fortunately, during this dire time I instinctively remembered my late 50’s early 60’s cold war childhood in Seattle where I was taught that “In the event of a Nuclear Strike, Aurora avenue will still be open for business”. Even though it was, at the moment, on the other side of the World from me, this seemed like the best strategy. Head to Aurora – that living testimony to the 1950s suburban culture - must be functional. Not even Mother Nature could bring Aurora down. Re-tracing my sensible route (at ½ the speed of the original journey), I eventually got to 175th and Aurora. Wow. Heaven. An oasis of technological life -working traffic lights, working gas pumps, and even a working donut store (but alas no Starbucks in that area – which is odd as I though it was a rule that each traffic light in Seattle had to be located next to a Starbucks). With a full tank of gas, my body rejuvenated with donuts, and nothing but time ahead of me, I figured I would now be able to escape.

As I began to breathe easier I lapsed into deep reflection and regarded this as yet another incident of Nature’s rapid fury that can fully paralyze a large scale urban area. It also became clear to me that, as we become increasingly reliant on technology, we are becoming increasingly unprepared to deal with its loss. Our numbers are big, our consumption rates are great and the whole scale of the system has become unmanageable. Events like this should educate and inform us – they should make us more prepared – not less. But, as the major of Seattle said, this storm was a once in 200 years event. And so, I suppose that one could tolerate Starbucks being down once every 200 years, but, strangely, it seems to me that these once every 200 year events are now coming to Seattle at the rate of one per month.- must be that confusing global climate change thing.

Friday, December 15, 2006

The speed of finance

The speed of light is not fast enough if you're in the business of vacuuming up micropennies (keeping bid ask spreads tight) using an automated trading system. You've got to have physical proximity to the exchange in order to keep lag times short! I guess the endpoint to this arms race is to have your servers co-located in the same facility as those of the exchange! Or, to be fair, the exchange could impose a random delay on all trades which is of order of the light travel time around the earth.

Is this kind of activity "making markets more efficient" -- the usual defense of financial shenanigans -- or just gaming the system as much as possible (the other standard description :-) ? If we're talking about milliseconds, I imagine that is about as efficient as we need. Also, I'm a bit confused as to why an outside system would be better at matching bids and asks than the internal algorithm of the exchange itself (unless it is taking on a little risk; see below). It's not surprising that these two types of entities -- trading automatons and exchanges themselves -- would eventually start stepping on each others' toes.

The next step is a little more AI in the algorithms. They probably already peek at futures and options prices to predict the direction of movement in the underlying, taking a bet rather than just locking in an arbitrage.

WSJ: NORTH KANSAS CITY, Mo. -- About four years ago, Dave Cummings moved his trading firm's computers from a storefront in this Kansas City suburb to buildings in New York and New Jersey that house central computers for two big electronic stock exchanges.

The move shaved a precious fraction of a second from the time it takes Mr. Cummings's firm, Tradebot Systems Inc., to buy or sell a stock on computer-based exchanges like Archipelago. It now takes Tradebot about 1/1000 of a second to trade a stock, compared with 20/1000 before the move -- a difference of about the time it takes a computer signal to zip at nearly the speed of light from Kansas City to New York and back.

That may not seem like a big difference, but in Mr. Cummings's obscure corner of the stock-trading universe, speed is critical and fractions of seconds loom large. Tradebot's computers are programmed to detect, among other things, tiny, fleeting differences between bid and offer prices of stocks, then to pounce, buying stocks at one price and almost immediately reselling them for a fraction more. If his firm hadn't moved its computers, says Mr. Cummings, "we'd be out of business."


Dozens of other firms, ranging from Citadel Derivatives Group to a brokerage unit of J.P. Morgan Chase & Co., also employ split-second trading strategies. That has set off an arms race to shave the time it takes for orders to reach the computers of electronic exchanges. In their quest for the choicest locations, at least 40 of Tradebot's competitors have carted their computers to the same buildings, a practice known as co-location.

The kind of trading practiced by Mr. Cummings is a particularly fast form of "algorithmic" or "black-box" trading, in which computer programs decide when to buy and sell securities. Hedge funds such as SAC Capital Advisors, D.E. Shaw & Co. and Renaissance Technologies have been using computers in their investment strategies for years. These days, a variety of new computer strategies rely on lightning-quick trading. For example, computers are being programmed to take news headlines into account when executing trades, and media companies including Dow Jones & Co., publisher of The Wall Street Journal, and Reuters Group have begun releasing news in computer-readable formats that cater to them.

Mr. Cummings's strategy -- which is shorter term than most and is highly reliant on speed -- was made possible by the growth of electronic-trading networks. These trading platforms -- Globex at the Chicago Mercantile Exchange, Archipelago at NYSE Group, INET at Nasdaq Stock Market and others -- account for more than half of all trading in household-name stocks and financial futures contracts.

Trading mostly Mr. Cummings's own money, privately held Tradebot, which has about 20 employees, makes between $30,000 and $150,000 a day and up to $20 million a year, people familiar with its finances say.

Electronic exchanges use computer systems to match buyers and sellers. They execute orders without involving floor traders known as specialists, who arrange transactions through auctions on the NYSE. For years, these specialists and the Wall Street dealers who traded Nasdaq stocks profited on gaps between bid and offer prices.

Opportunity for Profit

The electronic marketplaces bill themselves as a democratizing force -- a way to cut out specialists and dealers. Mr. Cummings realized that these electronic-trading pipelines also provided a new opportunity for profit: If he used fast enough computers and programmed them just right, he could harvest the same kind of trading profits that specialists and dealers long have gathered, albeit on a smaller scale for Mr. Cummings.


In Mr. Cummings's world, the fundamentals of a stock are of little consequence. His firm favors large stocks, such as Microsoft, because it can trade in and out quickly. On one recent afternoon, a computer screen in Tradebot headquarters indicated that the firm accounted for more than 10% of that day's trading in Microsoft. On many days, Tradebot's trading totals account for as much as 5% of all Nasdaq trading, on par with trading levels at such giant firms as Fidelity Investments.

"He's had a huge impact," says Jamie Selway, former chief economist of Archipelago and now head of White Cap Trading, a small New York brokerage firm. By constantly trolling to buy and sell, he says, Mr. Cummings enables others to trade more quickly. "You're probably trading with Dave Cummings, but don't know it," Mr. Selway says.

Mr. Cummings, 37 years old, learned to program as a teenager when he also worked at his father's computer-software store. After earning an engineering degree at Purdue University, he worked for three years at a health-care software company. In the mid-1990s, he moved to the Kansas City Board of Trade, donning a forest-green jacket and taking to the pits with other traders, buying and selling futures contracts tied to stocks and wheat prices.

In 1999, he quit his trading job, settled at a computer in a spare bedroom of his house and set out to develop software to replicate the actions of a floor trader in an arena where floor traders don't exist -- the electronic marketplace. Months later, he rented a small office in a bank building, furnished it with old furniture and a few computers, and launched Tradebot with $25,000 of personal savings.

Harold Bradley of American Century Investments, a Kansas City-based money-management firm, saw Mr. Cummings demonstrate the system while Tradebot was hunting for money to trade. Mr. Cummings told potential investors that he had created a robot to do what floor traders did -- hence, the name Tradebot. His firm would buy and sell based on signals from its computer program, collecting lots of nickels and dimes by being faster than other traders. Mr. Bradley says he declined to invest, figuring it would be difficult to make money on a large scale.

Mr. Cummings raised several hundred thousand dollars from a Chicago trading firm that wanted to use his programs. (Mr. Cummings bought this partner out in 2002.) Tradebot focused on stocks listed on the Nasdaq, which were easier to trade electronically than those at the New York Stock Exchange, where rules, at that time, limited electronic trading.

Tradebot took no long-term views on where stock prices were heading. Instead, it aimed to profit off tiny differences between what investors were willing to pay for heavily traded stocks and what others were willing to sell them for. In 2000, when stock pricing began shifting from increments of 1/16 of a dollar to one cent, many Wall Street trading desks gave up that business, figuring it was no longer profitable enough. That left Tradebot and competitors such as Citadel, Automated Trading Desk and Getco LLC to go after the profits.

Mr. Cummings's programs took into account fluctuations in stocks as well as in financial instruments such as stock-index futures. His computers, for example, would simultaneously try to buy Microsoft stock from investors at, say, $27.69 a share, and to sell it at $27.70, for a penny-per-share profit. Theoretically, human traders could spot the same opportunities, but Mr. Cummings's computers could usually get there first. His firm sometimes traded more than 1,000 times a minute. It ended most days owning no stock, cashing out all its positions.

Tradebot's employees constantly monitored the program trading. Mr. Cummings says he figured that once in a while the strategy wasn't going to work. It was often best, he concluded, to move out of the way when the market was moving sharply in one direction or another. He did not want to be stuck holding big blocks of stocks when their values unexpectedly plunged. If trading in a particular stock began losing money, an employee could switch off the program for that stock.

In the early 2000s, electronic marketplaces such as Island ECN and Archipelago courted professional traders. They offered rebates on certain kinds of complex trades such as "limit orders," in which traders offer to buy or sell shares within certain price parameters. The rebates allowed Tradebot to profit on some trades that merely broke even, according to several former employees.

By 2001, other firms were employing computer strategies similar to his, and some of them had an advantage: Their computers sat in Island's headquarters building in lower Manhattan. As a result, Tradebot's computers were a fraction of a second behind its rivals in trying to grab the best prices. "We were excluded because of the speed of light," Mr. Cummings says. "We had to move our computers."

Mr. Cummings discussed the problem with Island executive Matthew Andresen. "Our customers outside of New York are at a big disadvantage," Mr. Andresen recalls telling him. "We agree it should be a level playing field."

In addition to transaction fees Mr. Cummings's trades generate for Island, he agreed to pay a few thousand dollars per month to put Tradebot's computers in Island's facilities. The difference was significant: In the 1/50 of a second it used to take to execute an order from Kansas City, Tradebot could buy and sell at least 20 times.

Archipelago, an Island competitor now owned by the NYSE, was also after Tradebot's business. Mr. Cummings told Archipelago he was concerned that its system could take as long as a second or two to process his orders. "I can't manage that risk," he told Mr. Selway, then Archipelago's chief economist, Mr. Selway recalls. At times, Tradebot had to "throttle back" its Archipelago trading because that exchange's system couldn't handle the volume fast enough, says Mr. Cummings. Mr. Selway figures he spent a hundred hours on the phone with Mr. Cummings discussing how to make Archipelago better.

A few months after sealing the Island deal, Mr. Cummings visited Archipelago Chief Executive Gerald Putnam in Chicago and sized up the firm's computer room. Archipelago executives worried that letting Tradebot move its computers in would prompt complaints from other brokerage firms. They told Mr. Cummings to wait for Archipelago to complete a new data center in Weehawken, N.J., that could accommodate traders' computers.

Mr. Cummings didn't wait. He moved some Tradebot computers about a mile away from Archipelago's Chicago office. Then, when Archipelago finished its New Jersey data center, he moved the computers there. If a rival's computers "are in the exchange, and I'm across the street, then I lose," Mr. Cummings explains.

On Oct. 9, 2002, Tradebot for the first time traded 100 million shares in a single day, most of them stocks listed on Nasdaq. At that time, the NYSE was still a private cooperative controlled by floor brokers and specialists who traded stocks the old-fashioned way. It maintained limits on electronic trading, including restrictions on certain big orders and on rapid in-and-out trading. With an eye toward trading more NYSE-listed stocks, Tradebot bought Philadelphia-based Bloom Staloff Corp., which traded NYSE stocks from its specialist business at the Philadelphia Stock Exchange. Tradebot installed faster computers, but the NYSE restrictions proved troublesome. Mr. Cummings shut down the operation at a loss within six months and continued focusing on Nasdaq stocks. (In October of this year, the NYSE started opening its 2,700 listed stocks to electronic trading.)

Analysts estimate that Mr. Cummings's net worth surpassed $10 million several years ago. His trading business is probably worth several times that, although as a private firm, its finances aren't disclosed.

Over the past three years, about 40 other firms have moved their computers closer to the action, with bigger brokerage firms following smaller trading firms. Merrill Lynch & Co., Goldman Sachs Group, Deutsche Bank AG and J.P. Morgan Chase all have computers sitting near marketplace mainframes, or at least in the same neighborhoods.

Some exchanges, such as the Chicago Board of Trade, have resisted co-location, mostly because they don't want to put at a disadvantage investors who trade through firms that haven't moved their computers.

'Question of Fairness'

"It begs the question of fairness," says Andrew Brooks, head of stock trading at mutual-fund firm T. Rowe Price in Baltimore. "You shouldn't win just because you have better access to speed." Some investors also complain that rapid traders often cancel and resubmit orders, clogging the trading systems of electronic exchanges when markets are busy.

The Securities and Exchange Commission allows co-location, as long as electronic marketplaces give firms equal access to prized computer locations. The SEC has looked into the fairness issue and intends to continue to monitor it, according to two people familiar with the agency's approach.

Last year, Mr. Cummings says, he grew worried about the future of his trading strategy. More firms were co-locating. Consolidation in the exchange business, he feared, would lead to higher fees. The NYSE had acquired Archipelago, and Nasdaq had bought Island's successor company, INET.

In 2005, Mr. Cummings turned over day-to-day management of Tradebot and shifted most of his focus to a new venture: creating an exchange-like competitor to the NYSE and Nasdaq that would match buyers and sellers for a small fee. He called it Better Alternative Trading System, or BATS.

That June, he emailed about 100 traders and other industry contacts about BATS. Within days, senior executives from both NYSE and Nasdaq called to ask why one of their biggest customers was setting up a rival marketplace.

"The market needs competition," Mr. Cummings recalls telling Robert Greifeld, Nasdaq's chief executive officer.

"I wish you luck, but it's harder than you realize," Mr. Greifeld recalls replying.

Mr. Cummings has already poured about $4 million into the venture, which now occupies most of his time. He now travels on monthly sales calls and sends potential BATS clients a wooden baseball bat with the company's logo.

A few months after starting in January, BATS drew minority investments from Lehman Brothers Holdings Inc., Credit Suisse Group and Morgan Stanley. BATS now regularly handles about 5% of Nasdaq trading, the firm says.

Tuesday, December 12, 2006

Big, complicated data sets

This Times article profiles Nick Patterson, a mathematician whose career wandered from cryptography, to finance (7 years at Renaissance) and finally to bioinformatics. “I’m a data guy,” Dr. Patterson said. “What I know about is how to analyze big, complicated data sets.”

If you're a smart guy looking for something to do, there are 3 huge computational problems staring you in the face, for which the data is readily accessible.

1) human genome: 3 GB of data in a single genome; most data freely available on the Web (e.g., Hapmap stores patterns of sequence variation). Got a hypothesis about deep human history (evolution)? Test it yourself...

2) market prediction: every market tick available at zero or minimal subscription-service cost. Can you model short term movements? It's never been cheaper to build and test your model!

3) internet search: about 10^3 Terabytes of data (admittedly, a barrier to entry for an individual, but not for a startup). Can you come up with a better way to index or search it? What about peripheral problems like language translation or picture or video search?

The biggest barrier to entry is, of course, brainpower and a few years (a decade?) of concentrated learning. But the necessary books are all in the library :-)

Patterson has worked in 2 of the 3 areas listed above! Substituting crypto for internet search is understandable given his age, our cold war history, etc.

NYTimes: Thirty years ago, Nick Patterson worked in the secret halls of the Government Communications Headquarters, the code-breaking British agency that unscrambles intercepted messages and encrypts clandestine communications. He applied his brain to “the hardest problems the British had,” said Dr. Patterson, a mathematician.

Genetic evidence for complex speciation of humans and chimpanzees (Nature) Abstract Only
Today, at 59, he is tackling perhaps the toughest code of all — the human genome. Five years ago, Dr. Patterson joined the Broad Institute, a joint research center of Harvard and the Massachusetts Institute of Technology. His dexterity with numbers has already helped uncover startling information about ancient human origins.

In a study released in May, scientists at the Broad Institute scanned 20 million “letters” of genetic sequence from each of the human, chimpanzee, gorilla and macaque monkey genomes. Based on DNA differences, the researchers speculated that millions of years after an initial evolutionary split between human ancestors and chimp ancestors, the two lineages might have interbred again before diverging for good.

The controversial theory was built on the strength of rigorous statistical and mathematical modeling calculations on computers running complex algorithms. That is where Dr. Patterson contributed, working with the study’s leader, David Reich, who is a population geneticist, and others. Their findings were published in Nature.

Genomics is a third career for Dr. Patterson, who confesses he used to find biology articles in Nature “largely impenetrable.” After 20 years in cryptography, he was lured to Wall Street to help build mathematical models for predicting the markets. His professional zigzags have a unifying thread, however: “I’m a data guy,” Dr. Patterson said. “What I know about is how to analyze big, complicated data sets.”

In 2000, he pondered who had the most interesting, most complex data sets and decided “it had to be the biology people.”

Biologists are awash in DNA code. Last year alone, the Broad Institute sequenced nearly 70 billion bases of DNA, or 23 human genomes’ worth. Researchers are mining that trove to learn how humans evolved, which mutations cause cancer, and which genes respond to a given drug. Since biology has become an information science, said Eric S. Lander, a mathematician-turned-geneticist who directs the Broad Institute, “the premium now is on being able to interpret the data.” That is why quantitative-minded geeks from mathematics, physics and computer science have flocked to biology.

Scientists who write powerful DNA-sifting algorithms are the engine driving the genomics field, said Edward M. Rubin, a geneticist and director of the federal Joint Genome Institute in Walnut Creek, Calif. Like the Broad, the genome institute is packed with computational people, including “a bunch of astrophysicists who somehow wandered in and never left,” said Dr. Rubin, originally a physics major himself. Most have never touched a Petri dish.

Dr. Patterson belongs to this new breed of biologist. The shelves of his office in Cambridge, Mass., carry arcane math titles, yet he can converse just as deeply about Buddhism or Thucydides, whose writings he has studied in ancient Greek. He is prone to outbursts of boisterous laughter.

He was born in London in 1947. When he was 2 his Irish parents learned that he had a congenital bone disease that distorted the left side of his skull; his left eye is blind. He became a child chess prodigy who earned top scores on math exams, and later attended Cambridge, completing a math doctorate in finite group theory. In 1969, he won the Irish chess championship.

In 1972, Dr. Patterson began working at the Government Communications Headquarters, where his research remains classified. He absorbed through his mentors the mathematical philosophy of Alan Turing, the genius whose crew at Bletchley Park — the headquarters’ predecessor — broke Germany’s encryption codes during World War II. The biggest lesson he learned from Dr. Turing’s work, he said, was “an attitude of how you look at data and do statistics.”

In particular, Dr. Turing was an innovator in Bayesian statistics, which regard probability as dependent upon one’s opinion about the odds of something occurring, and which allows for updating that opinion with new data. In the 1970s, cryptographers at the communications headquarters were harnessing this approach, Dr. Patterson said, even while academics considered flexible Bayesian rules heretical.

In 1980, Dr. Patterson moved with his wife and children to Princeton, N.J., to join the Center for Communications Research, the cryptography branch of the Institute for Defense Analyses, a nonprofit research center financed by the Department of Defense. His work earned him a name in the cryptography circle. “You can probably pick out two or three people who’ve really stood out, and he’s one of them,” said Alan Richter, a longtime scientist at the defense institute.

In 1993 Dr. Patterson moved to Renaissance Technologies, a $200 million hedge fund, at the invitation of its founder, James H. Simons, a mathematician and former cryptographer at the institute. The fund made trades based on a mathematical model. Dr. Patterson knew little about money, but the statistical methods matched those used in code breaking, Dr. Simons said: analyzing a series of data — in this case daily stock price changes — and predicting the next number. Their methods apparently worked. In Dr. Patterson’s time with the hedge fund, its assets reached $4 billion.

By 2000, Dr. Patterson was restless. One day, he ran into Jill P. Mesirov, another former defense institute cryptographer, and mentioned his interest in biology. Dr. Mesirov, then director of computational biology at the Whitehead/M.I.T. Center for Genome Research, which later became the Broad Institute, hired him.

“Really, what we do for a living is to decrypt genomes,” Dr. Mesirov said. Cryptographers look at messages encoded as binary strings of zeros and ones, then extract underlying signals they can interpret, Dr. Mesirov said. The job calls for pattern recognition and mathematical modeling to explain the data. The same applies for analyzing DNA sequences, she said.

One common genomic analysis tool — the Hidden Markov Model — was invented for pattern recognition by defense institute code breakers in the 1960s, and Dr. Patterson is an expert in that technique. It can be used to predict the next letter in a sequence of English text garbled over a communications line, or to predict DNA regions that code for genes, and those that do not.

Dr. Patterson said he also has a well-honed instinct about which data is important, after seeing “a lot of surprising stuff that turned out to be complete nonsense.” Dr. Lander of the Broad Institute describes him as a great skeptic, with the statistical insight to tell whether a signal is “simply random fluctuation or whether it’s a smoking gun.”

Making that distinction is one of the great difficulties of interpreting DNA. In studying the human-chimp species split, the genomics researchers strove to rule out possible errors and biases in the data.

Dr. Reich, with Dr. Patterson and Dr. Lander, and two other colleagues, used computer algorithms to compare the primate genomes and count DNA bases that did not match, like the C base in gorillas that had become an A in humans. Because such mutations naturally arise at a set rate, the researchers could estimate how long ago the human and chimp lineages separated from an ancient common ancestor.

A DNA base can mutate more than once, however. To correct for that, Dr. Patterson worked out equations estimating how often it occurred; Dr. Reich revised their computer algorithms accordingly. Two strange patterns emerged. Some human DNA regions trace back to a much older common ancestor of humans and chimps than other regions do, with the ages varying by up to four million years. But on the X chromosome, people and chimps share a far younger common ancestor than on other chromosomes.

After the researchers tested various evolutionary models, the data appeared best explained if the human and chimp lineages split but later began mating again, producing a hybrid that could be a forebear of humans. The final breakup came as late as 5.4 million years ago, the team calculated.

The project was “our hobby” Dr. Reich said of himself and Dr. Patterson said. Their main work, in medical genetics, includes devising a shortcut to scan the genome for prostate cancer genes.

Whether studying disease or evolution, Dr. Patterson noted, genomics differs from code breaking in one key respect: no adversary is deliberately masking DNA’s meaning. Still, given its complexity, the code of life is the most open-ended of cryptographic challenges, Dr. Patterson said. “It’s a very big message.”

Monday, December 11, 2006

Digging through the rubble at Amaranth

Via Brad DeLong, this detailed Bloomberg article on the demise of Amaranth. One intriguing part of the story, discussed before at DealBreaker, is that their prime broker JP Morgan put the squeeze on them, took over their positions, and sold at a big profit:

It must be nice to be a JPMorgan prime brokerage client. First they squeeze you on the margin call, then scoop up your assets and make $750 million by selling them. Wonder how Amaranth founder Nick Maounis feels about JPMorgan these days.

The Bloomberg piece reports that, when it came time to meet margin calls

Amaranth's paper profits in natural gas were significant, yet it couldn't realize all of the gains selling. Traders and hedge funds knew Amaranth was desperate to get out of its trades, so they wouldn't pay current prices....

Brad seems to think the journalists are in error here. Perhaps it is shocking to an academic that markets aren't perfectly liquid ;-) The comment below clears things up.

[DeLong] ''I have to confess to some puzzlement over the statement that Amaranth "was desperate to get out of its trades" but couldn't because its counterparties "wouldn't pay current prices." The current price--the price at which you should mark to market--is the price at which you can sell, not a price at which you cannot sell.''

Untrue. And one of Amaranth's biggest problems was that this is not so... The "last trade" price is not necessarily the "market-clearing price for my whole stake" price. Indeed there are often discounts for block trades -- that is, for liquidity -- even for things with a heck of a lot more underlying value than some out-years natural gas futures contracts.

Thursday, December 07, 2006

How green is my valley

The 85% hydro and wind figure is news to me, and I live here! Eugene beat out Austin, TX and Portland, OR for top honors.

As ranked by the GreenGuide:

Top 10 U.S. Green Cities

1. Eugene, OR (score 9.0375, pop. 137,893)

First on our list is the university town, Eugene, well known as a powerhouse of green industry, clustering sustainable businesses like an environmentally minded Silicon Valley. Nestled in the Willamette River Valley with views of the Cascade Mountains, residents enjoy numerous bike trails, clean air and water, parkland and outlying wilderness areas. Hydroelectric and wind power contribute over 85 percent of Eugene's power, reducing greenhouse gas emissions considerably. A little over 16 percent of Eugene is green space, including athletic fields, city parks, public gardens, trails and waterfront.

Tuesday, December 05, 2006

Independent evidence for dark energy

New paper! We describe observational evidence for dark energy which is independent of the supernova surveys, and therefore not subject to systematic effects from dust, evolution or exotic particle physics (such as photon to axion decay). The figure below shows the likelihood, given best measurements of the age of the universe, the Hubble parameter and the matter fraction, that there was NOT a cosmological epoch with equation of state paramater w < w* (w = pressure / energy density). As you can see, an epoch with a repulsive energy component (w < -1/3) is highly likely. The three curves represent different assumptions about observational errors.

astro-ph/0612106

Precision cosmological measurements: independent evidence for dark energy

Authors: Greg Bothun, Stephen D.H. Hsu, Brian Murray
Comments: 4 pages, revtex, 3 figures

We examine whether observations independent of type Ia supernova surveys are sufficient to imply the existence of dark energy. We find that best measurements of the age of the universe $t_0$, the Hubble parameter $H_0$ and the matter fraction $\Omega_m$ strongly favor the existence of a repulsive, acceleration-causing ($w < -1/3$) component of energy if the universe is nearly flat.

Rolling the dice

Bloomberg has a nice profile of former PayPal CEO and now hedge fund manager Peter Thiel. This guy is a born risk taker. Be prepared for a rough ride if you're an investor in Clarium. I like his intellectual pretensions and sense of style, though. He used to live in a suite at the Four Seasons and his fund has offices in the Presidio.

On a sunny September morning, Thiel and his 10 traders and analysts are at work in Clarium's offices in the Presidio, the former U.S. Army post, now a national park, on the edge of San Francisco Bay.

It's an odd place to find a hedge fund. Like most San Francisco money managers, Thiel used to work downtown, in the city's financial district. Then, last June, he moved Clarium to the Presidio, where Star Wars director George Lucas has built a gleaming new headquarters. A statue of Jedi master Yoda gazes over one of the courtyards.

Death Star

Thiel has built his hedge fund on the premise that people follow the herd. Swept up in the crowd, they lose sight of reality. Thiel moved from the Bank of America Center downtown to keep his team away from other money managers and investment bankers who might cloud their thinking.

Yoda would feel right at home here. Press a button, and the doors hum open -- BRRRMMMM! -- as if you were boarding the Death Star. The 22,000-square-foot (2,044-square-meter) digs include a library stocked with leather-bound works of Charles Darwin, William Makepeace Thackeray, Guy de Maupassant and Leo Strauss. Every few months, Thiel brings in eminent scholars from the worlds of math, psychology and economics to address his troops.

Ralph Ho, Thiel's chief operating officer, says Clarium is part hedge fund and part think tank.

``We are trying to repeat the George Soros of the late '90s and learn from his mistakes,'' says Ho, who was PayPal's treasurer.

$10 Billion

Clarium doesn't want to get too big and risk becoming unwieldy. If all goes well, Clarium might one day manage as much as $10 billion, Ho, 36, says. For now, the fund is closed to new investors.

A few Clarium traders are hunched over their screens on the trading floor, a pit surrounded by the glass-walled offices. Thiel's team members typically arrive at 5 or 6 a.m., hit the gym or go running in the late afternoon and then return at 5 p.m. or so to work a few more hours. They spend 85 percent of their time doing research rather than trading with clients' money. Each has a small account to test trades.

Thiel, who favors polo shirts, jeans and sneakers, tends to keep odd hours. He sometimes answers e-mails in the middle of the night, Clarium General Counsel Bruce Gibney says. Thiel starts his workweek on Sunday nights, following markets in Asia.

Rather than dart in and out of markets, like many hedge fund managers, Thiel has made three broad bets and plans to let them ride. All reflect his big-picture economic view.

Three Wagers

One is that the price of 30-year U.S. Treasury bonds will rise as the U.S. economy slows and deflation sets in. Another is that the dollar will strengthen against the euro as investors scale back investments in emerging markets funded by borrowing dollars. And the third is that energy stocks will keep climbing along with the price of crude as world oil production reaches its zenith.

Thiel uses leverage to juice returns, typically borrowing $3-$8 for every dollar under management.

``We only do one big trade a week,'' says Matthew Kratter, Thiel's head trader. Kratter, 36, has a Ph.D. in English literature from the University of California, Berkeley. Kratter started his finance career in 1999, day trading with his scholarship money.

Clarium investors need a stomach for risk, because Thiel has put all their eggs in a few baskets. In 2003, for example, Clarium made a fortune partly by betting -- correctly, it turned out -- that the dollar would weaken. Clarium turned heads by posting a 65.6 percent return that year.

Ups and Downs

``Investors often called to ask whether he was lucky or good,'' says Steven Drobny, a partner at Manhattan Beach, California-based Drobny Global Advisors, which advises global macro funds on world markets. ``Whenever you have someone who puts up sensational returns out of the gate, people wonder if he's rolling the dice or if there is real thought behind it.''

Then, in 2004, Clarium posted a return of just 5.6 percent. It roared back in 2005 with a 57.1 percent gain, thanks in part to the payoff from a 2003 bet that the price of oil would soar.

Thiel is a proponent of a geologic theory known as peak oil, which holds that global oil production is now at or near its apex. Among his picks was Calgary-based EnCana Corp., which wrings oil from the tar sands of Canada. EnCana stock rose 54 percent in 2005.

Clarium has taken some hits along the way. Three times, Thiel has lost as much as 11 percent in a month. This past September, as the Dow Jones Industrial Average marched toward an all- time high, Thiel read a decline in U.S. new home sales as a sign that his forecast for the U.S. economy was coming true. He bet that the Russell 2000 Index would decline. Instead of slumping, the index kept rising. By the second week of October, Thiel exited the trade after hitting a stop-loss limit. Thiel was down 5.2 percent for the year through Nov. 3.

`Scary Guy'

``Thiel can be a scary guy if he's the only one in your portfolio,'' says David Philipp, a Clarium investor and managing partner at San Francisco-based Gyre Capital Management LLC. ``He's not afraid to put his money where his convictions are, and he's not the guy who's happy with 5 percent returns and 3 percent volatility.''

Thiel has even more riding on Clarium than most of his clients. He says he's invested his entire liquid net worth in his fund. Unlike most hedge fund managers, Thiel doesn't charge his customers an annual management fee. Instead, he pockets 25 percent of Clarium's trading gains. Hedge fund managers typically charge a 2 percent annual fee and take a 20 percent cut of profits.

...Thiel plans to keep hiring. To find candidates, he asks his employees to name the three smartest people they know and then contact those people to see if they'd like to work for him. Employees more than pay for themselves if they come up with moneymaking investments.

``They only need to have one good idea a year,'' Thiel says.

Thiel's alter ego at Clarium is a physicist named Kevin Harrington, who used to do mathematical research for the U.S. Department of Defense. The two sometimes talk strategy for five hours at a stretch.

``Peter is my foil, and I'm his foil,'' Harrington, 37, says.

Saturday, December 02, 2006

Higher Ed in India

I'm very familiar with the excellent scientists and engineers produced by the IIT's (Indian Institutes of Technology). I know much less about the rest of the system. In countries like China, Taiwan and Korea, there has been an explosion in recent years in numbers of private colleges and universities. One hopes that these institutions are driven by market forces, and teach the skills that will lead to good employment prospects.

NYTimes: The job market for Indian college graduates is split sharply in two. With a robust handshake, a placeless accent and a confident walk, you can get a $300-a-month job with Citibank or Microsoft. With a limp handshake and a thick accent, you might peddle credit cards door to door for $2 a day.

India was once divided chiefly by caste. Today, new criteria are creating a different divide: skills. Those with marketable skills are sought by a new economy of call centers and software houses; those without are ensnared in old, drudgelike jobs.

Unlike birthright, which determines caste, the skills in question are teachable: the ability to communicate crisply in clear English, to work with teams and deliver presentations, to use search engines like Google, to tear apart theories rather than memorize them.

But the chance to learn such skills is still a prerogative reserved, for the most part, for the modern equivalent of India’s upper castes — the few thousand students who graduate each year from academies like the Indian Institutes of Management and the Indian Institutes of Technology. Their alumni, mostly engineers, walk the hallways of Wall Street and Silicon Valley and are stewards for some of the largest companies.

In the shadow of those marquee institutions, most of the 11 million students in India’s 18,000 colleges and universities receive starkly inferior training, heavy on obedience and light on useful job skills.

Students, executives and educators say this two-tier education system is locking millions of people into the bottom berths of the economy, depriving the country of talent and students of the chance to improve their lot. For those who succeed, what counts is the right skills.

“It’s almost literally a matter of life and death for them,” said Kiran Karnik, president of the National Association of Software and Services Companies, a trade body that represents many leading employers. “The same person from the same institution with the same grades, but not having these skills, will either not get employed at all or will probably get a job in a shop or something.”

India is that rare country where it seems to get harder to find a job the more educated you are. In the 2001 census, college graduates had higher unemployment — 17 percent — than middle or high school graduates.

But as graduates complain about a lack of jobs, companies across India see a lack of skilled applicants. The contradiction is explained, experts say, by the poor quality of undergraduate education. India’s thousands of colleges are swallowing millions of new students every year, only to turn out degree holders whom no one wants to hire.

A study published by the software trade group last year concluded that only 10 percent of graduates with nonspecialized degrees were considered employable by leading companies, compared with 25 percent of engineers.

Keynes and Planck

Found in the comments on Economist's View:
Professor Planck, of Berlin, the famous originator of the Quantum Theory, once remarked to me that in early life he had thought of studying economics, but had found it too difficult! Professor Planck could easily master the whole corpus of mathematical economics in a few days. He did not mean that! But the amalgam of logic and intuition and the wide knowledge of facts, most of which are not precise, which is required for economic interpretation in its highest form is, quite truly, overwhelmingly difficult for those whose gift mainly consists in the power to imagine and pursue to their furthest points the implications and prior conditions of comparatively simple facts which are known with a high degree of precision.(Keynes, Essays in Biography 1951 158n)
It's true: powerful mathematical minds are not necessarily comfortable with the messiness of the real world. This observation might be applied as well to string (or more formal or mathematical) theorists vs theorists who are more data- or intuition-driven (often called phenomenologists, in a terrible use of terminology). Of course, some people (like Feynman, although he wasn't very mathematical by today's standards) are good at everything...

"We know a lot more than we can prove" -- Feynman

Blog Archive

Labels