Sovereign AI in the Gulf: how to run language models on your own GPUs
When a ministry, a bank or an energy company in the Gulf says it wants sovereign AI, it means the model runs on GPUs inside the country and the data never leaves. No document, prompt or answer goes to a foreign API.
For a software engineer that turns a simple API call into real engineering work: choosing an open model, fitting it into GPU memory, and serving many people at once without making them wait. This guide explains that work, with the open Arabic models you can use today.
What does sovereign AI mean for a software engineer?
It means the model runs on GPUs inside the country, under your control, instead of behind a foreign company's API. Documents, prompts and answers never leave. The engineer's job changes from calling an API to serving a model: choosing an open-weights model, fitting it into GPU memory, and serving many users at once with a server such as vLLM.
نبذة بالعربية
دورة عملية باللغة الإنجليزية لمهندسي البرمجيات في الخليج، تتعلّم فيها تشغيل النماذج اللغوية مفتوحة الأوزان على وحدات معالجة الرسومات الخاصة بك باستخدام vLLM، فتبقى بياناتك ضمن بنيتك التحتية. الدروس نصية وتفاعلية، مع مختبرات عملية على خوادم GPU، وبالوتيرة التي تناسبك.
Open weights
Falcon, Jais and the ALLaM 7B preview can be downloaded and run on your own servers.
The hard part
Serving: fitting the model in GPU memory and batching many users onto one GPU.
Embeddings too
If your documents are embedded by a foreign API, they have already left the country.
What you build
A real vLLM server on a rented NVIDIA GPU, measured one setting at a time.
If this helped, the full course goes much deeper.
In it, you rent an NVIDIA GPU, serve open models with vLLM, and measure how each setting changes speed, memory and the cost of every token.
From the Gulf you pay in US dollars, by card or PayPal.
Why the Gulf is building AI at home
Two forces push in the same direction. Governments want AI built on infrastructure they control, and data protection laws set rules on sending personal data abroad. The result is a wave of GPU capacity being built inside the region.
Abu Dhabi's government signed a multi-year deal with Microsoft and Core42 for a sovereign cloud sized for more than 11 million digital interactions a day, as part of an AED 13 billion plan for 2025 to 2027 that aims for an AI-native government by 2027.
Source: Abu Dhabi Media Office, 18 March 2025The first 200 megawatt phase of the Stargate UAE cluster in Abu Dhabi is expected to be ready in the third quarter of 2026, according to Mubadala's chief executive.
Source: The National, 5 December 2025HUMAIN, owned by Saudi Arabia's Public Investment Fund, announced with NVIDIA plans for up to 500 megawatts of AI capacity over five years, starting with 18,000 GB300 Grace Blackwell systems.
Source: NVIDIA newsroom, 13 May 2025Aramco, which holds 49% of HUMAIN, says HUMAIN plans 3 gigawatts of AI data centre capacity by 2030.
Source: Aramco: Building the AI future, read 19 November 2026The UAE's Personal Data Protection Law, Federal Decree-Law No. 45 of 2021, has been in force since 2 January 2022 and sets rules for transferring personal data across borders.
Source: UAE government portal, read 19 November 2026Saudi Arabia's Personal Data Protection Law became enforceable on 14 September 2024, and its transfer regulation asks for a risk assessment before personal data is sent outside the Kingdom.
Source: CMS law firm (a summary, not SDAIA's own page), 26 September 2024
The open models you can run yourself
An open-weights model is one whose trained weights you can download and run on your own hardware. Several strong ones come from the Gulf itself, which matters for Arabic.
One practical tip before you choose: count tokens. A model trained mostly on English text often splits Arabic into more tokens, so the same question can cost more GPU time and fill the context window sooner. Run the same Arabic paragraph through each model's tokenizer and compare the counts before you decide.
TII released Falcon 3 in 1B, 3B, 7B and 10B sizes under the TII Falcon License, which TII describes as based on Apache 2.0 with an acceptable use policy. The weights are on Hugging Face.
Source: TII newsroom, 17 December 2024TII's Falcon-H1 family mixes Transformer and Mamba layers and comes in sizes from 0.5B to 34B parameters.
Source: Hugging Face: tiiuae/Falcon-H1-34B-Instruct, July 2025The Jais family from Inception and MBZUAI has 20 models in 8 sizes, from 590M to 70B parameters, trained on Arabic, English and code and released under Apache 2.0.
Source: Hugging Face: inceptionai/jais-family-13b, read 19 November 2026SDAIA's ALLaM-7B-Instruct-preview is a 7 billion parameter Arabic and English model with a 4,096-token context, listed under Apache 2.0.
Source: Hugging Face: ALLaM-AI/ALLaM-7B-Instruct-preview, read 19 November 2026Fanar 2.0, the Arabic AI platform from Qatar Computing Research Institute, was launched in Doha in December 2025 and adds multimodal features.
Source: QCRI, 11 December 2025MBZUAI's K2-Think is a 32 billion parameter open-weights reasoning model released under Apache 2.0.
Source: Hugging Face: LLM360/K2-Think, September 2025
A model's licence decides what you may do with it. Read it before you build a product on it, especially the acceptable use part.
What a sovereign AI assistant looks like
Here is one common design, an assistant that answers staff questions from an agency's own documents. Follow the numbers on the drawing.
First, the employee's question reaches a gateway inside the country's GPU cloud. Second, a RAG service turns the question into an embedding, using an embedding model that also runs in the country. Third, it searches a vector database built from the agency's own documents. Fourth, it sends the question and the best passages to vLLM, which runs the open model on NVIDIA GPUs. Fifth, the answer comes back the same way, and the RAG service attaches the passages it used, so the employee can check the source.
Notice what crosses the boundary: only the model's weights, copied in once. Step two is the one teams forget. A foreign embedding API would receive every document and every question, which defeats the whole point.

The real engineering work: serving the model
Once the model runs on your own GPUs, three questions decide whether the project works.
Does it fit? A model's weights take about two bytes per parameter in 16-bit precision, so a 7 billion parameter model needs about 14 GB for the weights alone. On top of that, every conversation in progress keeps a KV cache in GPU memory, and that cache grows with every token. Quantization, storing weights in fewer bits, is how teams fit larger models on smaller GPUs, at some cost in quality that you must measure.
How many people can it serve? A GPU is wasted on one user at a time. Servers such as vLLM batch many requests together and keep adding new ones as old ones finish, which is called continuous batching. Tuning that batch is most of the job.
How fast does it feel? People notice the wait for the first word more than the total time. You measure time to first token and tokens per second separately, and you tune for the one your users feel.
If this helped, the full course goes much deeper.
In it, you rent an NVIDIA GPU, serve open models with vLLM, and measure how each setting changes speed, memory and the cost of every token.
From the Gulf you pay in US dollars, by card or PayPal.
How this shows up in an interview
A common design question in the region sounds like this: design an assistant for a government agency that answers in Arabic and English, where no data may leave the country. A strong answer does four things.
It names what must stay local: documents, embeddings, prompts, answers and logs. It picks an open model and explains why, including its licence and how it handles Arabic. It sizes the GPUs from the model's memory and the expected number of users. And it says how you would test answers in each language separately, because a model that does well in English can do worse in Arabic.
Questions people ask
Is sovereign AI only for governments?
No. Banks, hospitals, telecom and energy companies in the Gulf also keep sensitive data in the country. Any team that cannot send its data to a foreign API needs the same skills.
Can I learn this without owning a GPU?
Yes. The Inference Engineering course has you rent an NVIDIA GPU by the hour from a cloud provider, run the labs, and shut it down, so you only pay for the time you use.
Do I need to know machine learning first?
No. You need to be comfortable with Python and the command line. The course explains the parts of the model you need for serving, such as weights, tokens and the KV cache, as they come up.
Which open model should I start with?
Start small. A model in the 7B range fits on a single common GPU and lets you learn every serving setting quickly. Then compare a few models on your own Arabic and English questions before you choose one for a product.
Every source on this page was read on 19 November 2026. Plans, rules and numbers change, so check the linked page before you rely on one.
If this helped, the full course goes much deeper.
In it, you rent an NVIDIA GPU, serve open models with vLLM, and measure how each setting changes speed, memory and the cost of every token.
From the Gulf you pay in US dollars, by card or PayPal.