Generative artificial intelligence is no longer a technology exclusive to large data centers. Increasingly, the hardware we have at home or in the office can run AI models locally, without relying on the cloud. This trend, known as edge computing , is gaining ground thanks to open models and hardware optimization.
Running AI at the edge has clear advantages: data doesn't leave the device, there are no token or subscription fees, and privacy is guaranteed. But it also presents challenges, such as the need for powerful hardware and energy consumption. Now, platforms are emerging that aim to leverage users' idle hardware to create distributed inference networks.
Open models: the foundation of AI at the edge
Large language models (LLMs) have traditionally been run in the cloud, but open models are changing the game . They can be downloaded freely and used without subscriptions or paid APIs. However, they do require a computer with sufficient performance, especially in terms of memory and processing power. The more parameters the model has, the more resources it will need.
Among the most popular models for local deployment are Llama from Meta, Qwen from Alibaba, DeepSeek, Mistral, and Gemma from Google . Each has its strengths: Llama is very stable and versatile; Qwen excels in programming and multilingual support; DeepSeek stands out for its logical reasoning and efficiency; Mistral, of European origin, uses expert-mix architectures for greater efficiency; and Gemma is lightweight, ideal for teams with limited resources. This variety allows any user with hardware suitable for edge LLMs to have their own AI assistant without an internet connection.
Running data locally not only saves costs but also ensures that data remains on the device . This is crucial for privacy-conscious businesses and users. Furthermore, open-source models capable of processing images and videos already exist, expanding the possibilities for edge computing.
Renting your GPU for AI: the new business
While some run models on their own hardware, others have found a goldmine in idle hardware. Startups like Far Labs and Evolving Edge are creating marketplaces to rent out GPUs from gaming PCs that spend most of the day unused. The idea is similar to Uber or Airbnb, but for AI inference tasks. Owners install an app, and when their computer is idle, the platform runs inference jobs on their graphics card.
Abu Dhabi-based Far Labs is preparing to launch its Far AI platform, which promises latency of 100 ms or less. Austin-based Evolving Edge is already running an open beta. Both companies focus on small, open-source models, not front-end workloads. This approach allows them to use consumer-grade hardware, such as an RTX 4090, which consumes between 350 and 450 watts under sustained load. Electricity costs are a major question mark : if the hourly rate doesn't exceed the electricity bill, the owner would be paying to keep their equipment running.
Salad, a platform with over 60.000 active GPUs daily, has already proven the model can work, but its historical figures aren't very encouraging for users. In 2021, it generated around $3,6 million, but only distributed $500.000 to owners—about 14 cents for every dollar generated. The actual profitability depends on commissions and the price of electricity , which in Spain is around €0,16 per kWh. With an RTX 4090 running 24/7, the monthly bill can exceed €40, so the revenue needs to be higher to make it worthwhile.
Security and trust in distributed networks
Allowing a third party to run code on your computer carries risks. Malicious software could access local files or exploit vulnerabilities . Startups claim to isolate inference in a sandbox environment, with encrypted communication and strict limits on GPU, CPU, memory, and network usage. Evolving Edge has published its node code so anyone can audit what's running, but open source doesn't guarantee that the installed binary is the same.
The precedent set by Pearl, a blockchain network that promised to transform cryptocurrency mining into AI work, serves as a cautionary tale. One study showed that it used the equivalent of 320.000 GPUs for random array operations, without producing any useful AI. Far Labs and Evolving Edge are trying to differentiate themselves by routing real inference jobs, but their claims about cost and latency are yet to be verified at scale. Trust will be key to encouraging users to rent their hardware.
In Europe, the trend is also taking hold. Models like Mistral, of French origin, demonstrate that the old continent has a role to play in generative AI. And in Spain, a growing number of enthusiasts and small businesses are exploring the local implementation of open models, although the lack of clear information on profitability is hindering widespread adoption.
Edge hardware is evolving rapidly. On the one hand, open models allow anyone to run their own AI without relying on the cloud. On the other, GPU rental platforms promise to transform idle gaming PCs into nodes in a distributed network, but they have yet to prove their cost-effectiveness and security. Meanwhile, privacy and cost savings remain the main attractions of running AI at the edge—a trend that's here to stay.
