In the Stanford HELM (Holistic Evaluation of Language Models) earlier this month, Cohere’s Command Beta model took first place. In a comparison of 36 LLM models, including the Galactica model from Meta, the Davinci model from OpenAI, the Flan model from Google, and the Bloom model from the startup, the one that is trained to respond well to single-statement commands stood out.
Ed Grefenstette, the Head of Machine Learning at Cohere, maintained an admirably low profile despite the honor, describing the accomplishment as a “nice marketing moment.”
“You should always treat leadership boards with a grain of salt. We don’t want to become comfortable and think that simply because we are at the top of the leaderboard, we are automatically superior to models that are right behind us, the NLP expert warned. More than the top spot, he is thrilled with how their models are developing week by week.
He said that the most recent model from OpenAI, GPT-4, which is unquestionably a very powerful model, was not being benchmarked at the time. Therefore, he is not deluded and is confident that the outcome will change in the subsequent run if Stanford benchmarks the present models against GPT-4.
The Tools of the Philosopher
“Reading science fiction and cyberpunk books as a teenager and studying philosophy of mind during my college studies sparked my interest in artificial intelligence. The dearth of employment options in philosophy led me to choose computer science as my field of study, the AI stalwart claimed.
Grefenstette was an aspiring philosopher who was intrigued by the issue of what distinguishes sentient species from humans. He was interested in learning how intelligent people reason about things other than the physical world, such as ideas and metaphysics. He soon realized, though, that this was a challenging undertaking, and he started to consider how artificial intelligence may be used to support thinking.
Grefenstette realised that the method he and his colleagues were developing was comparable to simple neural networks while finishing his dissertation work in natural language processing at Oxford. To commercialise their ideas, they established “Dark Blue Labs” in 2014, however after a few months, Google acquired the company.
Grefenstette merged his team into DeepMind and helped set up the NLP group as well as a programme synthesis and comprehension group since six months before to the acquisition, the tech giant had also acquired the British AI startup, “DeepMind.” As a result, Grefenstette’s team joined DeepMind.
Boosting Entrepreneurial Spirit
The NLP specialist was present throughout the company’s rapid expansion at DeepMind, which went from 80 personnel to over 1,000 in just four years. He points out that this development may make it challenging for individual voices to be heard and may alter the dynamics and culture of a workplace.
