New kind You can also find out more about the following: large language modelResearchers from the Allen Institute for AI developed to allow users to manipulate training data even after they have built a model.
FlexOlmo is a new product that could disrupt the industry’s current paradigm. artificial intelligence companies slurping up data from the web, books, and other sources—often with little regard for ownership—and then owning the resulting models entirely. It’s like trying to get the eggs out of a cake once data has been baked into an AI.
“Conventionally, your data is either in or out,” Ali Farhadi says, Ai2 CEO, based in Seattle Washington. “Once I train on that data, you lose control. And you have no way out, unless you force me to go through another multi-million-dollar round of training.”
Ai2 divides training into different parts so data owners have control. Anyone who wants to add data to the FlexOlmo models can copy a public model called “The “anchor.” Then they train a model with their data and combine it with the anchor model. They give the results back to the person who is creating the final model.
By contributing in this manner, the data is never handed over. It is also possible to retrieve the data in the future, because of the way the owner’s data model has been merged with that final version. The publisher of a magazine might contribute the text in its archives to a particular model. However, they would later delete the submodel trained with that data. if there is a legal dispute If the model being used is not acceptable to the company, they can object.
“The training is completely asynchronous,” SewonMin, the Ai2 research scientist in charge of technical work says. “Data owners do not have to coordinate, and the training can be done completely independently.”
FlexOlmo’s model architecture can be described as what is known a “mixture of experts,” A popular design used for combining several submodels simultaneously into one larger, more powerful model. Ai2 has a unique way to combine sub-models which were previously trained individually. The new way of representing values within a model allows for the ability to merge with other models when the combined final model is ran.
FlexOlmo’s researchers tested the method by creating a Flexmix dataset from proprietary sources, including books and website. The FlexOlmo model was built with 37 billion variables, which is about one tenth the size of Meta’s largest open-source model. Then they compared the model with several other models. The researchers found it performed better than any other model for all tasks, and scored 10 percent higher at benchmarks common to two approaches.
The result is a way to have your cake—and get your eggs back, too. “You could just opt out of the system without any major damage and inference time,” Farhadi says. “It’s a whole new way of thinking about how to train these models.”
