large_Observium-Illustration-1280x720.png

medium_Observium-Illustration-1280x720.png

small_Observium-Illustration-1280x720.png

thumbnail_Observium-Illustration-1280x720.png

Observium-Illustration-1280x720.png

ReRAM Revival

Generative APIs

Affordable servers with the best price-performance ratio on the market

Start

Perfect balance of processing power, memory and storage

The high performance backbone of your mission-critical infrastructure

Core

For mission-critical data, fast storage, backup and streaming

Store

Dedicated GPU power with reliable performance and stability

Host your Hardware in our secured French datacenters

Dedirack

60 locations worldwide, starting at €4,99/month

Dedibox VPS

Dedibox - dedicated servers

Fully dedicated bare metal servers with native cloud integration, at the best price

Aluminium

Powerful, balanced and reliable servers for production-grade applications

Beryllium

Powerful dedicated server designed to handle high-workload applications

Iridium

Designed with huge local storage to keep, back up, and protect your data

Lithium

Power and stability of dedicated GPU hardware integrated into the Scaleway ecosystem

Titanium

Elastic Metal - bare metal cloud

Enjoy the Mac mini experience with great simplicity

Mac mini M1

Perform your daily tasks with speed and efficiency

Mac mini M2

Realize your most ambitious projects thanks to a new level of power

Mac mini M2 Pro

Latest Apple silicon chip for intensive use cases.

Mac mini M4

Apple

Dedibox and Bare Metal

Dedicated vCPU for the most demanding workloads (x86)

Production-Optimized

Secure, scalable VMs, equipped for high memory and compute demands (x86)

Workload-Optimized

Highly reliable and priced affordably Instances with shared vCPUs (x86 and ARM)

Cost-Optimized

A tiny instance to test and host your personal projects (x86)

Learning

Virtual Instances

Maximize your AI infrastructures with a versatile Instance

L4 GPU Instance

Universal Instance, faster than L4 and cheaper than H100 PCIe

L40S GPU Instance

Accelerate your model training with the most high-end AI chip

H100 PCIe GPU Instance

Dedicated NVIDIA® RTX 3070 with the best price/performance ratio

GPU 3070 Instances

Dedicated Tesla P100s for all your Machine Learning & Artificial Intelligence needs.

Render GPU Instances

Experience an easy way to run your code on the cloud

Serverless Functions

Easily run containers on the cloud with a single command

Serverless Containers

Serverless Jobs

Serverless

Kubernetes exclusively for Scaleway products and resources

Kubernetes Kapsule

Multi-cloud Kubernetes for Scaleway and external providers resources

Kubernetes Kosmos

Container Registry

Containers

Compute

Build the next Foundation Model with one of the fastest and most energy-efficient supercomputers in the world

Custom-built Clusters

Rent a GPU-cluster from 32 to more than a thousand GPUs to speed up distributed training

On Demand Cluster

Clusters

Deploy AI models in a dedicated inference infrastructure. Get tailored security and predictable throughput

Managed Inference

Consume AI models instantly via a simple API call. All hosted in Europe

Model-as-a-service

Accelerate the next generation of AI-enabled applications with the universal L40S GPU Instance, faster than L4 and cheaper than H100 PCIe

Maximize your AI infrastructure's potential with a versatile and cost-effective GPU Instance

Dedicated Tesla P100s for all your Machine Learning & Artificial Intelligence needs

Render GPU Instance

GPU Instances

Amazon S3-compatible and Multi-AZ resilient object storage service. Ensuring high availability for your data

Object Storage

Cold Storage class to secure long-term object storage. Ideal for deep archived data.

Scaleway Glacier

Flexible and reliable storage for demanding workloads

Block Storage

Storage

Secure your cloud resources with ease on a resilient regional network

Virtual Private Cloud

Centralize and simplify your Scaleway IP address management

IPAM (IP Address Manager)

Network

Improve the performance of your services as you grow.

Public Gateway

Establish a hosted connection from your infrastructure to your VPC via a partner's network

Scaleway InterLink

Buy domain names and manage DNS directly with your favorite extensions 

Domains and DNS

Connectivity

Load Balancer

Expose your HTTP services to the internet with security, reliability, and efficiency by design

Edge Services

Protect your web applications by proactively detecting and blocking cyber threats.

Traffic & Security

Managed Database for PostgreSQL and MySQL

Go full serverless and take the complexity out of PostgreSQL database

Serverless SQL Database

Managed Database for Redis®

Create a scalable, secure, and fully managed NoSQL solution

Managed MongoDB®

Databases

Build distributed and scalable client-server applications

NATS

Create a queue, configure its delivery and message parameters

Queues

Sent to a variety of devices and platforms through a single code interface

Topics and Events

Messaging and Queuing

Data

Monitor infrastructures in minutes with a fully managed observability solution

Cockpit

Hosting for individuals, professionals, and everyone in between.

Web Hosting

Ship your applications only in a few clicks.

Web Platform

Instant delivery of your transactional emails

Transactional Email

Speed up data processing over very large volumes of data with an Apache Spark™ managed solution

Distributed Data Lab

A purpose-built bridge between connected hardware and cloud.

IoT Hub

Managed Services

The easiest way to safely collaborate in the cloud

Identity and Access Management (IAM)

Protect your sensitive data across your cloud infrastructure

Secret Manager

Easily track your consumption in an all-in-one tool

Cost Manager

Accurately track your environmental impact and make informed choices

Environmental Footprint Calculator

Security & Organization

Scaleway API

Deploy and manage your infrastructure directly from the command line

Securely and efficiently provision and manage Infrastructure as Code with Terraform

Terraform

Developer Tools

Tools

Gaming

Public Sector

Media and Entertainment

Retail and E-commerce

Financial Services

Industrial

Technology

Healthcare

Industries

Artificial Intelligence

Cloud Storage Solutions

Kubernetes Solutions

Serverless Applications

Managed Web Hosting

Use Cases

Apply for Startup Program

Founders Program

Early Stage Program

Growth Stage

For Startups

Solutions

All products

Product updates

Betas

Changelog

Blog

Ecosystem

Slack Community

Feature Requests

Scaleway Learning

Community

Events

Marketplace

Careers

About us

Customer Testimonials

Company

Partners Program

Find your partner

Become a Partner

Partnership

Resources

Pricing

Docs

Contact

**AI models are everywhere, but sometimes, even the best LLMs produce bafflingly random outputs. Why? And more importantly—how can you tweak it?**

In 2024, the AI landscape has been evolving at breakneck speed. The race to release new models is relentless, with cutting-edge architectures dropping almost weekly. [aiworld.eu](https://aiworld.eu) and [Hugging face](https://huggingface.co) teamed up [to analyze the trends](https://huggingface.co/spaces/huggingface/open-source-ai-year-in-review-2024?day=3), tracking cumulative likes on models over 48 weeks. The data tells a clear story: the market is moving faster than ever, with new features, optimizations, and surprises constantly reshaping the AI ecosystem. 

![](https://www-uploads.scaleway.com/Capture_d_ecran_2025_03_17_a_14_14_50_82569b4dcf.png)

But with all this innovation comes confusion. AI models and the APIs that run them are still quite new for a lot of people and their usage can sometimes be puzzling.  

## Why do AI models sometimes behave unpredictably? How can you regain control? And finally, how can you ensure model accuracy at scale?
Let’s break it down.

**Why do AI models sometimes behave unpredictably?**

One of the biggest surprises? **LLMs aren’t built to be fully deterministic**. That means, even for simple questions like “1+1=?”, you might get unexpected variations. This isn’t a bug—it’s a feature. Some applications thrive on creative responses, while others demand strict, repeatable outputs. LLMs have not been built to be deterministic as any other algorithms, there are many use cases for which we want the LLM to generate creative answers and many others for which we look for a clear deterministic answer 

**Regain control: how to get deterministic outputs for a model?**

Adjust the determinism of the model: 
- **Temperature:** put the temperature at 0, which is the equivalent to tell the LLM to stop being creative. 
- **Seed:** provide a fixed seed, force the 1st tokens to be something you determine. If you provide this seed and temperature is 0, then the tokens that follow will always be the same

While LLMs can generate random lists of words without meaning or grammatical structure, this output is generally not useful. The ideal “creativity level” is something in between completely deterministic and completely random. This can be adjusted through:
- **Temperature:** put the temperature at 0.7. Overall values between 0.5 and 1 will add different levels of randomness. Values above 1 should usually be avoided, as they will usually output completely random words after a few paragraphs (even mixing languages).
- **Top P:** Set up a high top p (eg. above 0.7). This value influences the diversity of output, which is equivalent for the model to have a wider vocabulary. As an example, if you ask the model “List 10 known but original colors”, the model will provide:
Colors like “Mocha” or “Lavender” with a top_p of 0.01, because these words are widely used overall in the language.
Previous colors, and additional ones like “Amber”, “Taupe”, “Caramel”, “Cobalt” with a top_p of 1, which are less frequent in english language 
- **Prompts:** Besides changing typical LLM settings, you can also engineer your prompts directly, to guide the LLM to provide original outputs. Sometimes adding prompts such as “Provide creative outputs” is enough to change a lot of results.

Want to take control of your LLM’s randomness? Play around with different models and settings on [Generative APIs](https://www.scaleway.com/fr/generative-apis?utm_source=website&utm_medium=organic&utm_campaign=202503-cli-ww-all-gen_api_ga&utm_content=blog_post_genapi_). Your first million tokens are free. Experiment, tweak, and see the difference for yourself.

<iframe width="560" height="315" src="https://www.youtube-nocookie.com/embed/Iv3qpGkdFQk?si=nvoeUC1lFGXWSwJE" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

Be careful—modifications can have unpredictable impacts at scale. Whether you're tweaking your model for more deterministic output, adjusting prompts for accuracy in a specific feature, or relying on a model builder who may be deprecating or modifying the model without your knowledge, it's crucial to validate changes. You need to check the accuracy of modifications at your application level, as they can introduce hidden bugs in your workflow that may only surface in certain use cases.

## Model accuracy at scale: how to ensure a different LLM version or configuration will not break my application?

Setting up an AI application (whether it’s a Chatbot or a RAG- based (Retrieval-augmented generation) Agent) usually requires a lot of effort at all steps of the life cycle: prompt engineering, RAG pipelines retrieval accuracy, function calling trigger sensitivity, manual testing…
Performing all these steps again with a newer LLM version can be tedious but is necessary to avoid regressions in your application.

Indeed, there are usually no guarantees that a newer LLM version (even a minor one) or configuration will provide the same output quality for all prompts on all thematics. 

However, you can still limit the risk by:
- **Using models which are fine-tuned versions of the previous model** you were using. Similar output is not fully guaranteed, but will usually differ less than a completely different model.
- **Setup automated testing pipelines on sample dataset:**
- **Based on LLM classification** (also known as “LLM as a judge”): you can ask an LLM (the same one, or a bigger and thus more accurate one) to classify the quality of the output. This solution is not perfect, but can help detect many issues preventively.
- **Based on strict / rule-based verification:** if the output of the LLM can be verified with simple algorithms, you should definitely implement these checks. This can be done either with:
- **Raw outputs (eg. text)**, such as code. If your LLM generates code, you can try running this code, and if it doesn’t work, you know for sure the output is incorrect.
- **Structured outputs (eg. json output)**, such as structured representation of the content or intent. For instance, if you try to digitize and process a picture of a bill, you should use structured outputs, and have the LLM output each line of the bill and the corresponding total. You can then check if the total matches the sum of all the lines. Again if your total doesn’t match, you know for sure you cannot trust the output.

_Note that in both cases, the checks performed will only ensure the content may be correct (as in “looks plausible”). There is no 100% guarantee the content is correct (eg. in the code example, the code may run, but without doing what you expected). Since by design, LLMs provide “plausible” outputs you should be particularly careful of this. Models can indeed easily create hallucinations which appear consistent (eg. in the billing example, total bill lines may match, but one line may have been modified to compensate for an error in another one)._

- **Perform manual testing at scale:** Many applications developers still choose to perform A/B tests on users samples. Although this can take some time and be costly, it is the most reliable way to avoid regressions on a wide variety of usage.

Although advances in generative AI continue to push the boundaries of what’s possible, they also raise significant concerns about the reliability of models and the accuracy of their outputs. LLM hallucinations, far from being simple technical glitches, serve as a reminder that these systems are not oracles of truth. Additionally, they highlight how these models can reflect and perpetuate biases present in the training data.
As AI becomes increasingly integrated into decision-making processes, projects must take into account the ethical implications in the response error rate of these technologies. To address these concerns, research-driven initiatives such as prompt analysis and model testing are emerging, aimed at making AI more explainable. Learn in our next article how we can identify and mitigate biases in AI outputs, ultimately working toward more reliable and fair responses.

Scaleway Blog

Must read

ReRAM Revival

Latest articles

Why Aren’t Your Emails Reaching the Inbox?

Why is my state-of-the-art LLM spitting out nonsense?

“We have enabled 1,200 experts to save over 15% of their overall time” -Golem.ai

Scaleway’s Mac Mini Meets Virtual Private Cloud

Think small, move fast: My journey building a RAG chatbot at Scaleway

Big, Efficient, Open: The AI Future We Saw Coming

5 Key elements for Europe to achieve Sovereign AI

What Is a Document Database? A Comprehensive Guide

Anticipating Cyberthreats: The Winning Duo of WAF and Caching

Scaleway.com redesign: Next.js, TypeScript, (S)CSS Modules

Smart-proven contracts

The fundamentals of anti-DDoS protection

Most popular articles

ReRAM Revival

Why is my state-of-the-art LLM spitting out nonsense?

“We have enabled 1,200 experts to save over 15% of their overall time” -Golem.ai