November 30, 2017
By Michael Segal
There’s a weird way in which a hedge fund is a confluence of everything. There’s the money of course—Two Sigma, located in lower Manhattan, manages over $50 billion, an amount that has grown 600 percent in 6 years and is roughly the size of the economy of Bulgaria. Then there are the people—financiers, philosophers, engineers—all applying themselves to unearthing inscrutable patterns that separate fortune from failure.
And there is the science and engineering, much of it resting on a towering stack of data. In principle, almost any information about the real world can be relevant to a hedge fund. Employees, so the stories go, have camped out next to harbors noting down tanker waterlines, and in retail parking lots counting cars. This data then has to be standardized, synthesized, and made accessible to the people who place bets on the market.
Building the tools to do this is part of Alfred Spector’s job. As the chief technology officer of Two Sigma, he is responsible for the engineering platforms used by the firm’s modelers. A former vice president at IBM and former professor of computer science at Carnegie Mellon, Spector has seen software transform one industry after another, and made more than a few contributions of his own along the way.
He sat down with us for a conversation at Two Sigma’s headquarters earlier this month.
Why would a company like Two Sigma run a public game competition?
We started the Halite AI Programming Competition because we want to be known in the tech community for doing things that the tech community likes. Game competitions are one of those things. Programmers like interesting programing challenges—particularly ones that are contained enough that they can do off-hours. Plus we open-source everything so that the programmers can actually see the game environment and learn everything about the game. That gives programmers a lot of opportunities for creativity as to how the game can be played, and it becomes more fun.
What effect did you notice the competition having on the Two Sigma brand?
What we noticed is that when we go to campuses, people have heard of us more. We also hired someone that was at the top of the ranking.
What was the goal of the game?
Both this year and last year’s games are turn-based strategy games. In Halite 1 last year there were between two and six bots on the board to start. In the game, each bot starts with a single piece, which it can move up, down, left, or right. It can also stay still, in which case the piece gains strength. If it moves, it leaves behind another piece where it was, so the number of pieces on your side grows. All the players are doing this on a grid that’s maybe 40 or 50 squared pieces, so a bot can make a huge number of different moves. This year’s game, Halite 2, is in many ways similar, but it uses a space war theme, where a ship can move to a planet and take over the planet. When you put all this together, people come up with incredibly interesting winning strategies.
What’s an example of an interesting strategy that came out of the games?
There was an interesting, unanticipated strategy that emerged in the final week of last year’s game. It was a non-aggression strategy. Some players determined that if they hung loose for a while, and didn’t try to defeat other players, and just tried to gain space and stay out of trouble, it could help them. Remaining aggressive players would actually sort of hurt themselves, while the players that were being non-aggressive would actually then be in a position that, with moderately high probability, could win.
How was machine learning relevant to game strategy?
This year, we really focused a lot on advancing the ability of our players to use machine learning. In fact, we provided a limited number of Google credits for players to use GPUs (graphic processing units) in the Google Cloud, which allow for very rapid machine-learning algorithm training to be done. We make replays of all the games that have been played available, so that machine-learning systems could look at them and try to learn how to play the game better. It is still to be determined how well machine learning can do in a game as complex as Halite.
What’s the difference between an algorithmic approach and a machine-learning approach?
Computers always execute code, and code embodies an algorithm of some form. When we think of machine learning, those algorithms learn, and in effect modify themselves, from data. We think of classical algorithms as having everything specified in advance. Machine-learning algorithms have more degrees of freedom. They learn from the environment in which they’re operating.
When we go to campuses, people have heard of us more.
How does Two Sigma get its data?
I kind of liken it to the electromagnetic spectrum of economic data shining in on us. We see certain frequencies and we don’t see others. Certainly, we get tick data. We understand the prices and volumes of tradable entities in public markets. We get fundamental data. We get earnings data and things of that form from many sources. Sometimes we are able to get data on the interest that the sell side of Wall Street has in stocks. We have a product called PICS, which enables sell-side contributors to tell us that they’re recommending something. Both we and they benefit from this.
How can machines interpret this great variety of data?
One way is that a human has a hypothesis. For example, we can hypothesize that something will have an effect on the valuation of some security. What we can do then is create a mathematical representation of that predictive model and then see whether that hypothesis has proven true. We do an enormous amount of testing at Two Sigma. That’s hypothesis-driven work and, in one form or another, it has been the mainstay of investing for a very long time. Another approach is to send some form of a machine-learning algorithm at a lot of data and at some economic outcome, say a stock price or something like that, and see whether the machine can figure out the pattern. That’s a more challenging thing to go do because, even if you get a result that’s positive, you don’t necessarily feel that you know why.
How can humans learn to interpret machine learning?
I think one approach is to experiment and look at the outputs that you get as a function of the different kinds of inputs that you could give. Maybe a deep learning algorithm is going to predict the weather. You know it knows the barometric pressure, the temperature, the wind conditions, et cetera. Now let’s say you iterate over many of those elements and you look at how the predictions would change. You may then be able to say: This algorithm seems to be very sensitive to rapid temperature change. You’re beginning to get an idea of what the algorithm does, and you’re beginning to get explanative power.
Will there always be a human in the loop in this process?
Not necessarily. You could imagine that there’s some way of deciding on orthogonal planes of data that you then send to the system. You could take a lot of data elements and then feed them through the system and see what predictions occur in an automatic fashion, and try to learn from that. You could imagine another learning algorithm layered on the first learning algorithm.
Should we be wary of the unintended consequences of machine learning?
There was a very nice paper published on the reinforcement of gender bias as an unintended consequence. Imagine you build a machine learning system that is designed to show ads to people who will click on those ads. If you’re showing ads for CEO positions, with the state of the world as it is today, perhaps it is more likely that males will click on them. If that’s true, a system that’s doing machine learning may learn to advertise CEO ads in publications that are frequented by men, thereby inadvertently promulgating an undesirable bias. I note that the teams building that algorithm had no idea of this unintended consequence. I give a talk, called the Opportunities and Perils of Data Science, where I argue that we have a much greater need to educate engineers and data scientists in all manner of ethical issues now as data science becomes more pervasive.
How will those education needs be met?
We will need to train people in every discipline and probably every walk of life to have more technological understanding. We already see it happening. We see many more high schools with computer science and programming training. We see a lot more of it in college. In fact, computer science has become the largest major at some universities: Stanford, for example. You see at other great universities introductory computer science courses really growing in enrollment. All this I think is good.