A leap forward in AI technology spearheaded by MIT researchers offers a transformative approach to training artificial intelligence models on everyday devices, including smartwatches and sensors. By increasing efficiency in privacy-preserving AI training by roughly 81 percent, the researchers have paved the way for a wider range of devices with limited resources to use more precise AI models while keeping user data safe. This brings us a step closer to genuinely democratizing AI access.
Federated learning—an approach that permits a network of interconnected devices to train a universally shared AI model—now works with greater efficiency thanks to the team. In this setup, the AI model is first distributed from a central server to various devices. Each device, using its data, trains the model, and then sends the modifications back to the server. This strategy ensures that the data never leaves the device, providing an additional layer of data security. Yet, not all devices in this network have the required capacity, power, and connectivity to work with the model effectively, leading sometimes to delays that affect the overall training performance.
Enter MIT’s ingenious techniques to manage these memory constraints and connection bottlenecks. Their method is specifically designed to operate within a network made up of diverse devices with different resources and restrictions. This makes it increasingly likely to see the use of AI models in high-stake areas with strict security mandates, such as healthcare and finance.
“This work is about bringing AI to small devices where it is not currently possible to run these kinds of powerful models. We carry these devices around with us in our daily lives. We need AI to be able to run on these devices, not just on giant servers and GPUs, and this work is an important step toward enabling that,” explains Irene Tenison, an electrical engineering and computer science (EECS) graduate student and lead author of the research paper.
Addressing existing shortcomings in federated learning approaches, Tenison notes the challenge of devices with diverse capabilities and intermittent connectivity. They often have limited memory and computational power and can’t quickly transmit their updates back to the central server. Plus, waiting for updates from all devices can cause significant lag time: “This lag time can slow down the training procedure or even cause it to fail.”
This is why the researchers developed a new approach called FTTE (Federated Tiny Training Engine). The FTTE packs three main innovations that significantly reduce memory and communication overhead required by each mobile device. These consist of sending only subset model parameters to devices reducing memory, updating the model using an asynchronous approach, and calculating each device’s update impact based on when it was received, which ensures older data doesn’t hinder progress.
The team has tested their method via simulations on hundreds of diverse devices, various models and datasets. The result was impressive—the training has finished up to 81 percent faster than with standard federated learning approaches. The on-device memory overhead was reduced by 80 percent and the communication payload by 69 percent, while, importantly, maintaining the accuracy of other methods.
Noting that they intentionally streamlined the process to allow for faster training and conserve battery life of smaller, less capable devices, Tenison said: “Because we want the model to train as fast as possible to save the battery life of these resource-constrained devices, we do have a tradeoff in accuracy. But a small drop in accuracy could be acceptable in some applications, especially since our method performs so much faster.”
Tenison also highlighted FTTE’s scalability and superior performance with larger device groups, adding: “Not everyone has the latest Apple iPhone. In many developing countries, for instance, users might have less powerful mobile phones. With our technique, we can bring the benefits of federated learning to these settings.”
For the future, the researchers plan to delve into how their methodology could improve personalized performance of AI models on each device. They are keen on carrying out larger experiments on real hardware. This breakthrough is partly funded by a Takeda PhD Fellowship.
If you are in search of AI automation for your business, consider implementing AI solutions with implementi.ai.
For the original story, please follow this link.
This website uses cookies.