S., Patil S., Dettmers T., Baruwa A., Singh A., Cheveleva A., Ligozat A., Subramonian A., Névéol A., Lovering C., Garrette D., Tunuguntla D., Reiter E., Taktasheva E., Voloshina E., Bogdanov E., Winata G. I., Schoelkopf H., Kalo J., Novikova J., Forde J. Z., Clive J., Kasai J., Kawamura K., Hazan L., Carpuat M., Clinciu M., Kim N., Cheng N., Serikov O., Antverg O., Wal O. v. d., Zhang R., Zhang R., Gehrmann S., Mirkin S., Pais S., Shavrina T., Scialom T., Yun T., Limisiewicz T., Rieser V., Protasov V., Mikhailov V., Pruksachatkun Y., Belinkov Y., Bamberger Z., Kasner Z., Rueda A., Pestana A., Feizpour A., Khan A., Faranak A., Santos A., Hevia A., Unldreaj A., Aghagol A., Abdollahi A., Tammour A., HajiHosseini A., Behroozi B., Ajibade B., Saxena B., Ferrandis C. M., Contractor D., Lansky D., David D., Kiela D., Nguyen D. A., Tan E., Baylor E., Ozoani E., Mirza F., Ononiwu F., Rezanejad H., Jones H., Bhattacharya I., Solaiman I., Sedenko I., Nejadgholi I., Passmore J., Seltzer J., Sanz J. B., Dutra L., Samagaio M., Elbadri M., Mieskes M., Gerchick M., Akinlolu M., McKenna M., Qiu M., Ghauri M., Burynok M., Abrar N., Rajani N., Elkott N., Fahmy N., Samuel O., An R., Kromann R., Hao R., Alizadeh S., Shubber S., Wang S., Roy S., Viguier S., Le T., Oyebade T., Le T., Yang Y., Nguyen Z., Kashyap A. R., Palasciano A., Callahan A., Shukla A., Miranda-Escalada A., Singh A., Beilharz B., Wang B., Brito C., Zhou C., Jain C., Xu C., Fourrier C., Periñán D. L., Molano D., Yu D., Manjavacas E., Barth F., Fuhrimann F., Altay G., Bayrak G., Burns G., Vrabec H. U., Bello I., Dash I., Kang J., Giorgi J., Golde J., Posada J. D., Sivaraman K. R., Bulchandani L., Liu L., Shinzato L., Bykhovetz M. H. d., Takeuchi M., Pàmies M., Castillo M. A., Nezhurina M., Sänger M., Samwald M., Cullan M., Weinberg M., Wolf M. D., Mihaljcic M., Liu M., Freidank M., Kang M., Seelam N., Dahlberg N., Broad N. M., Muellner N., Fung P., Haller P., Chandrasekhar R., Eisenberg R., Martin R., Canalli R., Su R., Su R., Cahyawijaya S., Garda S., Deshmukh S. S., Mishra S., Kiblawi S., Ott S., Sang-aroonsiri S., Kumar S., Schweter S., Bharati S., Laud T., Gigant T., Kainuma T., Kusa W., Labrak Y., Bajaj Y. S., Venkatraman Y., Xu Y., Xu Y., Xu Y., Tan Z., Xie Z., Ye Z., Bras M., Belkada Y., Wolf T. (2022). BLOOM: A 176B-Parameter Open-Access Multilingual Language Model // https://arxiv.org/abs/2211.05100
2625
Zhang S., Roller S., Goyal N., Artetxe M., Chen M., Chen S., Dewan C., Diab M., Li X., Lin X. V., Mihaylov T., Ott M., Shleifer S., Shuster K., Simig D., Koura P. S., Sridhar A., Wang T., Zettlemoyer L. (2022). OPT: Open Pre-trained Transformer Language Models // https://arxiv.org/abs/2205.01068
2626
Zhang S., Diab M., Zettlemoyer L. (2022). Democratizing access to large-scale language models with OPT-175B / Meta AI, May 3, 2022 // https://ai.facebook.com/blog/democratizing-access-to-large-scale-language-models-with-opt-175b/
2627
Taylor R., Kardas M., Cucurull G., Scialom T., Hartshorn A., Saravia E., Poulton A., Kerkez V., Stojnic R. (2022). Galactica: A Large Language Model for Science // https://arxiv.org/abs/2211.09085
2628
AI21 Labs Makes Language AI Applications Accessible to Broader Audience (2021) / businesswire: a Berkshire Hathaway Company, August 11, 2021 // https://www.businesswire.com/news/home/20210811005033/en/AI21-Labs-Makes-Language-AI-Applications-Accessible-to-Broader-Audience
2629
Rae J., Irving G., Weidinger L. (2021). Language modelling at scale: Gopher, ethical considerations, and retrieval / DeepMind blog, 08 Dec 2021 // https://deepmind.com/blog/article/language-modelling-at-scale
2630
Rae J. W., Borgeaud S., Cai T., Millican K., Hoffmann J., Song F., Aslanides J., Henderson S., Ring R., Young S., Rutherford E., Hennigan T., Menick J., Cassirer A., Powell R., Driessche G. v. d., Hendricks L. A., Rauh M., Huang P., Glaese A., Welbl J., Dathathri S., Huang S., Uesato J., Mellor J., Higgins I., Creswell A., McAleese N., Wu A., Elsen E., Jayakumar S., Buchatskaya E., Budden D., Sutherland E., Simonyan K., Paganini M., Sifre L., Martens L., Li X. L., Kuncoro A., Nematzadeh A., Gribovskaya E., Donato D., Lazaridou A., Mensch A., Lespiau J., Tsimpoukelli M., Grigorev N., Fritz D., Sottiaux T., Pajarskas M., Pohlen T., Gong Z., Toyama D., d'Autume C. d. M., Li Y., Terzi T., Mikulik V., Babuschkin I., Clark A., Casas D. d. L., Guy A., Jones C., Bradbury J., Johnson M., Hechtman B., Weidinger L., Gabriel I., Isaac W., Lockhart E., Osindero S., Rimell L., Dyer C., Vinyals O., Ayoub K., Stanway J., Bennett L., Hassabis D., Kavukcuoglu K., Irving G. (2021). Scaling Language Models: Methods, Analysis & Insights from Training Gopher // https://arxiv.org/abs/2112.11446
2631
Kaplan J., McCandlish S., Henighan T., Brown T. B., Chess B., Child R., Gray S., Radford A., Wu J., Amodei D. (2020). Scaling Laws for Neural Language Models // https://arxiv.org/abs/2001.08361
2632
Hoffmann J., Borgeaud S., Mensch A., Sifre L. (2022). An empirical analysis of compute-optimal large language model training / DeepMind blog, April 12, 2022 // https://www.deepmind.com/publications/an-empirical-analysis-of-compute-optimal-large-language-model-training
2633
Hoffmann J., Borgeaud S., Mensch A., Buchatskaya E., Cai T., Rutherford E., de Las Casas D., Hendricks L. A., Welbl J., Clark A., Hennigan T., Noland E., Millican K., van den Driessche G., Damoc B., Guy A., Osindero S., Simonyan K., Elsen E., Rae J. W., Vinyals O., Sifre L. (2022). Training Compute-Optimal Large Language Models // https://arxiv.org/abs/2203.15556
2634
Pichai S. (2023). Google DeepMind: Bringing together two world-class AI teams. / Google Blog, Apr 20, 2023 // https://blog.google/technology/ai/april-ai-update/
2635
Chowdhery A., Narang S., Devlin J., Bosma M., Mishra G., Roberts A., Barham P., Chung H. W., Sutton C., Gehrmann S., Schuh P., Shi K., Tsvyashchenko S., Maynez J., Rao A., Barnes P., Tay Y., Shazeer N., Prabhakaran V., Reif E., Du
