Category: AI Models

Categories
AI Models AI Tools Conference

Bielik AI Unveils Two New Models with Polish-Optimized Tokenizer at KUKDM 2026

Zakopane, Poland — April 14, 2026 — At the 18th High-Performance Computing Users’ Conference (KUKDM 2026) in Zakopane, the Bielik AI team presented two new language models: Bielik-PL-11B-v3.0-Instruct and Bielik-PL-Minitron-7B-v3.0-Instruct. These are the first variants in the Bielik family to feature a dedicated tokenizer optimized specifically for the Polish language.

Presentation at KUKDM 2026

The premiere took place on April 14 at 10:35 AM during the presentation “Architecture Optimization in Bielik Models” (original title: Optymalizacja architektury w modelach Bielik), delivered by Krzysztof Ociepa and Krzysztof Wróbel from the Bielik AI team. The KUKDM conference, organized by the Academic Computer Centre Cyfronet AGH, runs from April 13–15, 2026 at Hotel Bachleda Kasprowy in Zakopane, bringing together Poland’s scientific and technology community focused on high-performance computing.

What’s New in the Bielik PL Models?

The key innovation in the new models is the transition from a universal Mistral-based tokenizer to a dedicated Polish-optimized vocabulary. Previous Bielik models relied on a tokenizer designed to cover a broad spectrum of languages, which resulted in higher fertility ratios for Polish text, increased inference costs, and restricted effective context windows.

The new models employ FOCUS-based embedding initialization, a multi-stage pretraining curriculum, and an advanced post-training pipeline comprising:

  • Supervised Fine-Tuning (SFT)
  • Direct Preference Optimization (DPO)
  • Group Relative Policy Optimization (GRPO) with verifiable rewards

Benchmark Results

According to the accompanying research paper, Bielik-11B-v3.0-Instruct achieves a 5-shot average of 65.93 on the Open LLM Leaderboard, ranking among the top models and outperforming several significantly larger solutions, including Meta-Llama-3.1-70B-Instruct.

The Polish tokenizer variants achieve scores of 64.11 (Bielik-PL-11B) and 61.66 (Bielik-PL-Minitron-7B), confirming that tokenizer optimization does not come at the expense of model quality. In domain-specific evaluations for the Polish language, including the Polish Medical Benchmark, the 11B model demonstrated a significant improvement over the base version.

Compression Without Compromise

The Bielik-PL-Minitron-7B model was created by compressing the 11B variant using structured pruning and knowledge distillation techniques developed in collaboration with NVIDIA engineers. This approach achieved a 33% reduction in model size and up to 50% faster inference, while retaining approximately 90% of the full model’s quality. The compression methodology was first introduced at NVIDIA GTC in March 2026.

Research Publication

The technical details of the new models are described in the paper “Advancing Polish Language Modeling through Tokenizer Optimization in the Bielik v3 7B and 11B Series“, published on April 12, 2026 on the arXiv platform (ID: 2604.10799). The paper is authored by Krzysztof Ociepa, Łukasz Flis, Remigiusz Kinas, Krzysztof Wróbel and Adrian Gwoździej.

Links: