The article presents a comparative analysis of four large language models (LLMs) — two local national initiatives (Bielik 11B v2.3, PLLuM 12B), an open universal multilingual model (Gemma3 12B) and a large-scale commercial model (Gemini 2.5 Flash) — generating text in Polish, with a particular focus on feminatives. The evaluation used 45 queries grouped into 12 thematic and problem--based classes, allowing for an analysis of the reproduction of linguistic and social patterns across five dimensions: grammatical correctness; stylistic naturalness; task efficiency; the level of feminative usage; and linguistic progressiveness. The experiments were conducted using a dialogue interface, and the responses were subjected to expert assessment. The Bielik model outperformed its competitors in grammar and stylistics, while Gemini achieved the best results in precise instruction following and the frequency of feminine forms. Gemma3 and PLLuM ranked lower, especially in tasks requiring precise adaptation of feminative forms. Analysis revealed that the models were conservative in terms of linguistic progressiveness, suggesting the perpetuation of traditional binary structures by algorithms trained on legacy corpora. The study underscores the critical importance of developing open language models within local national initiatives, highlighting the need for both the continued evolution of Polish-language systems and further development of reliable auditing methods to assess their sociolinguistic sensitivity.
You may also start an advanced similarity search for this article.
