I studied engineering in college, and like most of my contemporaries, I found that I sometimes needed to write computer programs to perform certain types of calculations. These utility software were written in languages now considered the programming equivalent of Latin – Fortran, Algol and Pascal – and what I learned from this experience was that I was not a born hacker. The software I wrote was clunky and inefficient and more talented programmers would stare at it and roll their eyes, just like Rory McIlroy might if he were to play a round with an 18 handicap golfer. But he got the job done and, in that sense, was, in the terse phrase sometimes used by the great computer scientist Roger Needham, “good enough for government work”. And what I took away from that experience is a lifelong respect for programmers who can write elegant and efficient code. Anyone who thinks programming is easy has never done it.
It was impressive, quirky, and perhaps useful in some contexts, but really, it was just picking fruit at hand. Applications are small programs, and the types of tasks Codex can perform are those that can be succinctly described in everyday language. All the software has to do is search through the huge repository of computer code that exists in its database and find a match that will do the job. No actual inference or reasoning is required.
At this point, DeepMind, the London-based artificial intelligence company, became interested in the problem. DeepMind is famous for developing the world champion AlphaGo and AlphaFold, the machine learning system that seems to predict protein structures better than any human. Recently, he announced that he has developed AlphaCode, a new programming engine potentially capable of surpassing many human developers.
In classic DeepMind style, the company decided to see how its system would perform on 10 challenges on Codeforces, a platform that hosts competitive programming contests around the world. Although these challenges are not typical of the average daily workload of programmers, the ability to solve the problems it poses creatively is a good indicator of programming ability. AlphaCode is the very first AI system capable of competing with humans in this context.
Here’s what’s involved: Competitors are given five to ten problems expressed in natural language and given three hours to write programs to creatively solve as many problems as possible. This is a much more demanding task than simply specifying an application. For each problem, participants should read and understand: a natural language description (spanning many paragraphs) that contains a narrative context of the problem; a description of the desired solution that competitors should understand and analyze carefully; a specification of the required input and output format; and one or more sample input/output pairs. Then they have to write an effective program that solves the problem. And finally, they have to run the program.
The key step – moving from stating the problem to finding a solution – is what makes competition such a difficult test for a machine, as it requires understanding and reasoning about the problem, as well as an in-depth understanding of a wide range of algorithms and data. structures. What is impressive about the design of Codeforces competitions is that it is not possible to solve problems by shortcuts, such as duplicating solutions seen before or trying all potentially related algorithms. To do well, you have to be creative.
So how did AlphaCode do it? Alright, is the answer. “Overall,” DeepMind reports, it came out “at the median competitor level. of competitive programming.
Translation: “We will be back.”
They go. It’s starting to look like the story of Go-play and protein folding; either way, the DeepMind machine starts mid-level and then quickly outpaces the human competition. It will be a quick learner. Does this mean that programmers will become obsolete? No, because software engineering is about building systems, not just solving discrete puzzles. But if I were to write software now, it would be reassuring to have such a machine as my assistant.
what i read
eat your words
Cooking with Virginia Woolf is a fine essay by Valerie Stivers in the Paris review on how the author of Towards the Lighthouse didn’t know much In beef stew.
John Seabrook reflects on Ford’s decision to electrify its much-loved F-150 truck in a long New Yorker room, America’s favorite pickup truck is going electric.
Spotify’s True Colors
A neat blog post by Damon Krukowski, The Big Short of Streaming, dissects Spotify’s attempt to defuse the Joe Rogan controversy. TL;DR summary: Spotify is a technology company, not a music company.