GPT-4 to Aid in the Preservation of the Icelandic Language

Alþingi parliament of Iceland

As noted in an article published yesterday, Iceland has partnered with OpenAI (the company that developed ChatGPT) to use its next-generation version of GPT in an effort to preserve the Icelandic language – and to turn “a defensive position into an opportunity to innovate.” The Minister of Culture and Business Affairs told RÚV yesterday that the partnership had proved that small nations could, if they had “done their homework,” use AI and language technology to aid in the preservation of their languages.

Turning a defensive position into an opportunity

Yesterday, OpenAI released GPT-4, the fourth in a series of the company’s multimodal large language models that powers ChatGPT, an AI chatbot launched last November. GPT-4 will initially be available to ChatGPT Plus subscribers (who pay $20 per month for premium access to the service), and it is already powering Microsoft’s Bing search engine platform.

As noted in an article published on OpenAI’s website yesterday, among those parties using GPT-4 to its advantage is the Icelandic government, which is employing this next-generation version of GPT to preserve its language. GPT-4 has made significant improvements in its ability to respond in Icelandic, improvements that are partly the result of a collaboration inspired by an Icelandic delegation’s visit to OpenAI ’s headquarters in May of last year. The delegation, consisting of Iceland’s President and government ministers, met with OpenAI’s founder, Sam Altman.

“Iceland […] partnered with OpenAI to use GPT-4 in the preservation effort of the Icelandic language – and to turn a defensive position into an opportunity to innovate. The partnership was envisioned not only as a way to boost GPT-4’s ability to service a new corner of the world, but also as a step towards creating resources that could serve to promote the preservation of other low-resource languages.”

The article also quotes Jóhanna Vigdís Guðmundsdóttir, CEO of Almannarómur (a non-profit language technology center): “We want to make sure that artificial intelligence will be used not only to help preserve language, culture, and history, but also to underpin economic prosperity.”

Better, but still flawed

As noted by the New York Times, GPT-4 has shown impressive improvements in accuracy when compared to its predecessor (GPT-3.5): it’s gained the ability to summarise and comment on images, summarise complicated texts, and is capable of passing a bar exam and several standardised tests; however, it still shows a tendency to hallucinate answers.

Likewise, GPT-4, while much better at Icelandic than GPT-3.5, still produces Icelandic with “grammatical errors, ‘translationese,’ and incorrect cultural knowledge.” To make further improvements, Vilhjálmur Þorsteinsson, CEO of Miðeind (a privately owned software company based in Reykjavík that specialises in language technology), assembled a team of 40 volunteers to train GPT-4 on proper Icelandic grammar and cultural knowledge.

In a process called Reinforcement Learning from Human Feedback (RLHF) human testers give GPT-4 a prompt, and four possible completions are generated. After reviewing the four responses, testers choose the most suitable answer and refine it to achieve an optimal completion. The insights derived from this procedure are subsequently utilised to enhance the performance of GPT-4, enabling it to generate more refined responses in the future.

As noted by OpenAI, RLHF produces results with just 100 examples, which makes it “more feasible for other low-resource languages, with less digital language data available, to replicate the process.” Prior to RLHF, the process of fine-tuning a model was labour and data-intensive. Þorsteinsson’s team had initially attempted to fine-tune a GPT-3 model with 300,000 Icelandic language examples, but the results were “disappointing.”

“Now we can just jump directly to the general capabilities of the large models,” Þorsteinsson is quoted as saying on OpenAI’s website, “and enable things with our language that used to require a lot of manual labour, data preparation, and resource collection for each use case.”

With a single round of RLHF complete, the model still has some room for improvement, which provides ongoing work for the Icelandic team: to continue to train GPT-4 with sufficient examples so that the model can power “the most complex and creative applications in Icelandic, rather than defaulting to English.”

The aim is to allow the entire country to interact with OpenAI’s models in their own language, which would, for example, save Icelandic companies from relying on English-speaking chatbots on their websites.

A “very happy day”

In an interview with RÚV yesterday, Minister of Culture and Business Affairs Lilja Alfreðsdóttir stated that she was very happy with the government’s partnership with OpenAI:

“This is what we’ve been aiming for over the last five years. The government has invested over ISK 2 billion ($14 million / €13 million) in creating this basic language infrastructure so that we can get to this point,” Lilja stated, adding that over sixty experts had been working on the project for the last four to five years.

“This was always the goal: that we could introduce our efforts to companies using artificial intelligence and language technology. We met with OpenAI and this was the result: that we’re the first language besides English that they plan to introduce. So we’re incredibly happy.”

Lilja explained that during their first meeting with OpenAI, it was clear that the company was interested in introducing a language that was not as widely spoken as English: “To show that the world is not just English. We somehow managed to talk about it in cultural, historical, and literary terms.”

Lilja added that this partnership was of great significance in an international context.

“We’re proving that small nations, if they’ve done their homework, can use AI and language technology to preserve their languages. And what our collaborators thought was so amazing was seeing all the work that we had already done – creating this infrastructure so that this technology may be harnessed.”

Proposal to Protect Geysir Area a “Cause for Celebration”

Geyser

Earlier this year, the Environment Agency of Iceland submitted a proposal to declare the Geysir region a protected area. The deadline for comment on the proposal passed on April 23. Among the aims of the proposal is to preserve the area’s unique geological formations. The Icelandic Environmental Association (Landvernd) has announced its support for the proposal.

Geysir

The geothermal area of Haukadalur Valley, in southwest Iceland, is home to the hot spring Geysir (The English word geyser derives from Geysir) along with more than 40 other smaller hot springs, mud pots, and fumaroles.

Despite being an integral part of Iceland’s Golden Circle (the most popular tourist route in Iceland), the Geyser area has not enjoyed protected status. This means, among other things, that the Environment Agency of Iceland is not responsible for ensuring the minimum safety of visiting tourists.

In January, the Environment Agency of Iceland submitted a proposal to grant protected status to the Geysir area.

Preservation, scientific research, tourism

According to the Agency, the aim of the declaration is to preserve the area’s unique geological formations and to ensure that visitors may continue to visit the area without negatively impacting the environment: “… the area is of tremendous educational and scientific value, both locally and globally. The protected status will also help ensure the future use of the area by locals and tourists and that it is capable of receiving the thousands of guests who visit every year.”

The proposal forms a part of the government’s effort to extend protected status to vulnerable sites in Iceland in accordance with the government agreement. The proposal is submitted in accordance with Article 39 of Act No. 60/2013. The proposal was authored by a workgroup comprised of representatives from the Environment Agency of Iceland, the Bláskógabyggð Municipality, and the Ministry for the Environment and Natural Resources.

Should have been done “decades ago”

On April 22, the Icelandic Environmental Association announced its support for the proposal. The Union calls the proposal a cause for celebration: “Granting protected status to the Geysir area is long overdue. It’s actually quite astonishing that protected status wasn’t granted decades ago. It will ensure that future generations will be able to enjoy the magnificent beauty and power of the Geysir area for the foreseeable future.”

Seals Gain Protected Status in Reykjavík

Seals are now protected within the Reykjavík City limits and the surrounding area. The next step is to ensure the protection status of seals in the general law, according to mammal ecologist Ester Rut Unnsteinsdóttir at The Icelandic Institute of Natural History, RÚV reports.

The Environment and Planning Committee of Reykjavík City approved a proposal to make seals protected within coastal areas surrounding Reykjavík, as well as near estuaries. All hunting of both common seals and grey seals will cease within the jurisdiction of the city. Reykjavík City’s website states it is necessary to improve the legal status of seals and to create a framework to control seal hunting. The decision doesn’t have a formal legal effect but is more of a statement of intent.

“We hope that more municipalities follow suit, but what matters most is to ensure that seals become legally protected, so they have adequate protection in Iceland,” said Ester Rut from the Icelandic Institute of Natural History.

The common seal is listed as an endangered species in Iceland while the grey seal is listed as vulnerable. The common seal (phoca vitulina) is a coastal animal, living closer to the shore, while the grey seal (halichoerus grypus) is an ocean-going seal. The common seal is sometimes named the speckled seal. The two species are considered among the most common seal species in the world. However, the species have faced tough times in Iceland in recent years. “We have no knowledge of why it’s happening, but there’s a clear reason to react,” said Ester. Further investigation needs to take place regarding the reduction of the seals. Common seal numbers have decreased by 77% in a 35-year timespan. In years past, they were hunted in considerable numbers but no major hunting has taken place in recent history.

“It might be bycatch that causes the reduction when they accidentally get caught by nets intended for other species. That type of hunting is relatively common but is most often not registered,” commented Ester.

New Conservation Laws Go Into Effect at Hornstrandir

New land management and conservation regulations around the Hornstrandir nature preserve in the Westfjords went into effect on Friday, RÚV reports. The new regulations now ban camping outside of specially designated areas and put significant restrictions on cruise ship landings, among other measures that have been put in place to keep the preserve as “untouched as possible” for future generations.

Hornstrandir was established as a nature preserve in 1975. The updated Hornstrandir regulations are the result of a collaboration between local land owners, as well as planning authorities and the Environment Agency of Iceland. They reiterate the overall conservation plan for Hornstrandir, and also lay out an action plan for the more pressing concerns related to the preserve and the order in which they need to be prioritized between now and 2023. Travel habits have changed a great deal since the last time these regulations were examined, Kristín Ósk Jónasdóttir, a specialist working with the Environment Agency, points out, which is why it was important to update them now.

Per the new regulations, it is no longer legal for visitors to camp in Hornstrandir except in specifically designated areas where sanitary facilities have been provided. Likewise, visitors may not ride bikes or bring dogs within the preserve (exceptions are made for dog owners who live within the boundaries of the preserve, as well as people with rescue or service dogs). Tour group size will be limited: a maximum of 30 people in the western part of the preserve and 15 in the eastern part. Larger tour groups will need to apply to the Environment Agency for an exception. The landing of cruise ships with 50 passengers or more will also no longer be permitted within the preserve. It’s also requested that the Coast Guard update its navigational chart such that all ship traffic must be at least 115 metres [377 ft] away from all sea bird colonies and require that permission to take videos or photographs be specially obtained from the Environment Agency, as both can have a negative effect not only on other visitors’ experience, but also on the wildlife itself.

Kristín Ósk says that maintaining the tranquility of the preserve is important, which is why helicopter landings and drone operation is also not allowed within its boundaries. Similarly are small aircraft landings only allowed within designated areas in the preserve. “In all reality, we’re trying to keep the preserve as untouched as possible and what we’ve been trying to do in the preceding decades should still be possible for coming generations to do as well.”