Training data formats

Two ways to give alallo training data

An alallo training job reads its data from one of two sources, chosen under "Training data" in the Data section of the New job form:

• Hugging Face dataset: type the name of a public dataset on the Hugging Face Hub, for example tatsu-lab/alpaca.
• Upload a file (JSONL, JSON, CSV, OpenAI or Mistral fine-tuning file): upload a file from your computer, or pick a file you or a colleague uploaded before. Training files made for OpenAI or Mistral fine-tuning work as they are; see Using your OpenAI fine-tuning files and Using your Mistral fine-tuning files.

A job uses exactly one source. Uploading files requires an alallo account. Visitors who are not signed in can get estimates for Hugging Face datasets only.

Using a Hugging Face dataset in alallo

To train on a Hugging Face dataset, choose "Hugging Face dataset" and enter the dataset's ID as shown on the Hugging Face Hub (owner/name). alallo downloads the dataset when it estimates and runs the job. [TBD: alallo currently reads public datasets only; private and gated Hugging Face datasets (which need a Hugging Face access token) are not supported. Confirm and say whether token support is planned.]

Uploading a training file to alallo

To upload a file, choose "Upload a file (JSONL, JSON, CSV, OpenAI or Mistral fine-tuning file)" and click "Upload new file". After the upload, alallo shows the file's number of rows, its column names and the first rows as a preview, so you can check that alallo read the file correctly. For an OpenAI or Mistral fine-tuning file, alallo shows the format it recognised and the first conversations instead. For other files, alallo suggests a column mapping (a text template, or prompt/chosen/rejected columns) which you can edit. Uploaded files appear in the "File" list with their row count, and every user in your alallo account can use them. [TBD: whether and how customers can delete an uploaded file.]

File types alallo accepts

alallo accepts three file types, recognised by the file name's extension:

• .jsonl (JSON Lines): one JSON object per line. Empty lines are skipped. This is the recommended format.
• .json: either one JSON array of objects (the file starts with [), or JSON Lines.
• .csv: comma-separated values with a header row. The header row gives the column names.

Every row must be an object with named columns, for example {"instruction": "...", "output": "..."}. Files must be UTF-8 encoded.

Using your OpenAI fine-tuning files in alallo

If you fine-tuned models with OpenAI, upload the same JSONL training files to alallo. You don't need to convert them. alallo recognises the format from the file's first row and checks every row when you upload. You don't need a text template or column names for these files.

OpenAI chat format

Mistral uses the same chat format; see Using your Mistral fine-tuning files.

Each line is one conversation with a messages list:

{"messages": [{"role": "system", "content": "You are a helpful support agent."}, {"role": "user", "content": "Where is my order?"}, {"role": "assistant", "content": "Let me check that for you."}]}

• Roles: system, user and assistant. developer is treated as system.
• Multi-turn conversations: a conversation can have several user and assistant turns. The model learns every assistant message.
• The weight field: an assistant message with "weight": 0 stays in the conversation as context, but the model does not learn it. "weight": 1 (the default) means it is learned. As with OpenAI, weight is allowed on assistant messages only and must be 0 or 1.
• Content: text, or a list of text parts such as [{"type": "text", "text": "..."}].

alallo writes each conversation in the chat format of the model you fine-tune (its chat template), so the result works with the usual chat tools for that model. The fine-tuned model is delivered with that chat format. If a model has no chat format of its own, alallo uses a simple one and includes it with your model. Some models' chat formats have no place for a system message; alallo then puts the system message at the start of the first user message. Use the chat formats for supervised fine-tuning (LoRA, QLoRA or full weights).

OpenAI prompt/completion format

The older OpenAI format, one prompt and one completion per line, also works in .jsonl and .csv files:

{"prompt": "Classify the sentiment: I love it ->", "completion": " positive"}

alallo trains on the prompt followed directly by the completion, as OpenAI did, and the model learns the completion only. Keep any separators or leading spaces your prompts and completions already use, and send prompts the same way to the fine-tuned model later.

OpenAI preference format (DPO)

OpenAI's preference fine-tuning files work for alallo's DPO method:

{"input": {"messages": [{"role": "user", "content": "Summarise: ..."}]}, "preferred_output": [{"role": "assistant", "content": "A short, accurate summary."}], "non_preferred_output": [{"role": "assistant", "content": "A long summary with a mistake."}]}

preferred_output and non_preferred_output must each hold exactly one assistant message.

What alallo does not support in OpenAI files yet

• Tool and function calling: rows with tools, functions, tool_calls or messages with the role tool are refused. [TBD: whether and when tool calling will be supported.]
• Images, audio and files in messages: alallo trains text models only.

When a conversation is longer than the job's max sequence length, alallo cuts it off there. A conversation with no learned assistant text left within that length is skipped, and the job log says how many were skipped.

Using your Mistral fine-tuning files in alallo

Mistral has deprecated its self-serve fine-tuning. If you fine-tuned models with Mistral (on Mistral's platform, with the Mistral fine-tuning API, or with mistral-finetune), upload the same JSONL training files to alallo. You don't need to convert them, and you can train them on Mistral's own open models in alallo (Ministral 3, Mistral NeMo and Mistral Small 3) or on any other model in the catalogue. See Models and training methods.

Mistral instruct (chat) files

Mistral's instruct format is the same as the OpenAI chat format: one conversation per line in a messages list with the roles system, user and assistant.

{"messages": [{"role": "system", "content": "Réponds en français."}, {"role": "user", "content": "Où est ma commande ?"}, {"role": "assistant", "content": "Je vérifie tout de suite."}]}

• Multi-turn conversations work, and the model learns every assistant message.
• "weight": 0 on an assistant message keeps it as context without learning it, as in Mistral's own fine-tuning. weight is allowed on assistant messages only and must be 0 or 1.
• Content can be text or a list of text parts such as [{"type": "text", "text": "..."}].
• The last message doesn't have to be an assistant message. Messages after the last assistant message are kept as context and are not learned.

alallo writes each conversation in the chat format of the model you fine-tune, so the same file works for a Mistral model, EuroLLM or any other catalogue model. The upload preview names the format "OpenAI/Mistral chat format".

Mistral pretraining files (plain text)

Mistral's pretraining format has one text field per line:

{"text": "Der Vertrag beginnt am 1. Januar und läuft zwölf Monate."}

alallo reads such a file as a file with one text column and suggests the text template {text}, so the model learns the whole text of each line. Use it with supervised fine-tuning (LoRA, QLoRA or full weights).

What alallo does not support in Mistral files yet

• Function calling: rows with tools, tool_calls or messages with the role tool are refused, as for OpenAI files. [TBD: whether and when tool calling will be supported.]
• Images: rows with image_url content parts are refused; alallo trains text models only.
• Classifier training files (Mistral's classifier format, with a labels object on each row) are refused with the message this looks like a Mistral classifier training file .... alallo fine-tunes models that write text. [TBD: whether classifier training will be offered.]

Instruction, input and output files (Alpaca format) in alallo

An uploaded file whose rows have the columns instruction and output, and optionally input, is recognised as the Alpaca format and used as it is; you don't write a text template. For example, JSONL rows:

{"instruction": "Give the capital of France.", "input": "", "output": "Paris."}
{"instruction": "Translate to French.", "input": "Good morning", "output": "Bonjour."}

alallo writes each row as one conversation in the base model's own chat format: the instruction (followed by the input, when there is one) as the question, and the output as the answer. The model learns the answer only, so it doesn't learn to write the questions, and an empty input adds nothing. CSV files with these column names are recognised the same way. The fine-tuned model then answers prompts written in its usual chat format.

File size limit for alallo uploads

An uploaded training file can be at most 2 GB. While the file uploads, the button shows how much has been sent; then alallo checks the file ("Checking the file…"). Checking a large file can take a few minutes; keep the page open until the preview appears.

Data for supervised fine-tuning in alallo (text template)

Supervised fine-tuning (LoRA, QLoRA and full weights) trains on one text per row. For OpenAI fine-tuning files and Alpaca-format files you skip this step; see above. For other files, you tell alallo how to build that text with a text template. The template is any text with {column} placeholders; alallo replaces each placeholder with the row's value in that column. For example, for a dataset with the columns instruction, input and output:

### Instruction:
{instruction}
{input}

### Response:
{output}

If your file already has a single column named text with the complete training text, use the template {text}. alallo adds an end-of-text marker after each example, so the model learns where an answer stops. With a text template, alallo does not add any chat formatting of its own: the model learns exactly the text your template produces. Use the same format when you send prompts to the fine-tuned model later.

Data for DPO in alallo (prompt, chosen, rejected)

Direct preference optimisation (DPO) can use an OpenAI preference file as it is. Other files need three columns per row: the prompt, the chosen (preferred) answer and the rejected answer. In the Data section, enter the names of these columns in "prompt column", "chosen column" and "rejected column". The defaults are prompt, chosen and rejected. For example, a JSONL row for DPO:

{"prompt": "Summarise: ...", "chosen": "A short, accurate summary.", "rejected": "A long summary with a mistake."}

The model is trained only on the answer part, not on the prompt. The prompt and an answer together are cut off at the job's max sequence length.

Choosing part of a dataset in alallo (split)

The "Split" field selects which rows alallo uses. Write train for the whole training set, train[:1000] for the first 1000 rows, or train[100:200] for rows 100 to 199. Uploaded files are treated as one set named train. Hugging Face datasets can have other sets, such as test or validation, which you can name instead.

How much of your data an alallo job uses

How much training an alallo job does is set by "Max steps" and the other training settings, not by the number of rows. If the job processes more tokens than your data contains, alallo goes through your data more than once; if fewer, the job uses only part of your data. The estimate is based on the tokens your data actually contains. See Models and training methods.

Upload errors in alallo

When alallo cannot read an uploaded file, the dashboard shows "Upload failed:" followed by the file name and the reason. The reasons are:

Message	Meaning and fix
unsupported file type .xyz; use .jsonl, .json or .csv	The file name does not end in .jsonl, .json or .csv. Convert the file or rename it with the correct extension.
line N is not valid JSON: ...	Line N of a JSON Lines file is not a complete JSON object. Check for line breaks inside a record, missing quotes or trailing commas.
row N is not an object with named columns	Row N is a list, string or number instead of an object such as {"text": "..."}.
this looks like a file in the OpenAI/Mistral chat format, but N of M rows can't be used: row 3: ...	alallo recognised an OpenAI or Mistral fine-tuning file, but some rows break its rules. The message names up to five rows and what is wrong with each, for example message 2 is a tool or function call, weight must be 0 or 1, weight is only allowed on assistant messages, no assistant message to learn from or every assistant message has weight 0. Fix or remove those rows and upload again.
this looks like a Mistral classifier training file (a "labels" object per row); alallo fine-tunes models that write text, and classifier training isn't supported yet	The file is in Mistral's classifier format. alallo doesn't train classifiers yet.
the file has no rows	The file is empty or contains only a header or blank lines.
the upload didn't arrive; try again	The upload was interrupted before the whole file reached alallo, for example because the connection dropped or the page was closed. Upload the file again.
file larger than 2 GB	The file is over the 2 GB limit. Upload a smaller file or select fewer rows.
'utf-8' codec can't decode ...	The file is not UTF-8 encoded. Save it again as UTF-8.

Data errors when estimating or starting an alallo job

These messages can appear when you click "Estimate time and price" or start an alallo job:

Message	Meaning and fix
<file> has no column(s) X; it has a, b, c	Your text template or your prompt/chosen/rejected columns name a column the uploaded file does not have. The message lists the columns the file has. Fix the template or column names.
<file> is in the OpenAI/Mistral chat format, which has one answer per example. DPO needs a preferred and a rejected answer ...	DPO needs preference pairs. Choose LoRA, QLoRA or full fine-tuning for this file, or upload a file in OpenAI's preference format.
<file> is in the OpenAI preference (DPO) format; choose the DPO method to train on it.	A preference file can only be used with DPO.
Choose your training data: a Hugging Face dataset or an uploaded file.	No data source is set. Enter a Hugging Face dataset or choose an uploaded file.
Enter a Hugging Face dataset first. / Upload a file or choose one first.	The estimate button is disabled until a data source is chosen.
uploaded dataset ... not found	The chosen uploaded file no longer exists or belongs to another account. Upload it again.
sign up to train on your own files	You are not signed in. Uploaded files need an alallo account.

For Hugging Face datasets, alallo does not check the column names of a text template in advance. A template that names a missing column makes the job stop with a "Training failed" message that names the missing column. A failed job costs nothing; see Cancelling jobs and refunds.