in content lists or on the content page.
Legislative drafting is a heavy and multi-stage process, which includes broad data collection, data analysis and the formulation of memos. In the future, generative AI can enable significant efficiency gains for law drafting. Sitra-funded gen AI pilot projects developed and tested AI solutions based on Finnish language models to support the work of law drafters in reviewing legislation and summarising statements.
The pilots were carried out at the Ministry of Transport and Communications, the Prime Minister’s Office and the Ministry of Justice. The pilots took the first step in using a generative AI tool tailored to the law drafters’ challenges in Finland.
Similar projects have already been carried out in several European countries. In some of them, the central government has taken the role of a trendsetter in the use of solutions based on European language models. In Germany, the technology company Aleph Alpha, a pioneer in generative AI, has developed AI solutions for central government, including the F13 Assistant introduced this year. F13 is an AI tool tailored to the needs of the authorities, which supports officials in tasks such as summarising documents. In France, AI solutions that work on top of open-source language models are actively developed, such as the Albert AI system that supports the work of officials.
AI discussion is known to involve a lot of hype. At the same time, many public actors and companies are uncertain about how to use generative AI effectively and safely. This article contains key observations from the Sitra-funded pilot projects and the requirements and challenges related to adopting Finnish language models.
Four requirements for the adoption of generative AI
The project pilots started directly from very demanding applications, even though this was the first time to develop generative AI applications for Finnish law drafting. The selected applications were also quite suitable for the use of artificial intelligence, as the material used in them were largely public files, which facilitated the processing of the material during the project. In addition, there is a lot of potential in the initial law drafting and consultation process for improving the efficiency of operations and reducing heavy manual work.
We listed the key requirements identified in the pilots that need to be observed when using generative AI in challenging applications.
1. A reliable application requires access to high-quality, manageable and up-to-date data
The data required to perform the task must be of high quality and manageable so that artificial intelligence can use the data effectively. Data that is as up to date and reliable as possible is also a key requirement for the usefulness of the tool.
Lack of up-to-date legislative data weakened the usefulness of the tool in the pilot
Finnish legislation is a key data source for legislative applications. It was also needed in the solution created in the Ministry of Transport and Communications pilot supporting the initial law drafting process. Finlex publishes Finnish legislation in digital format, but its terms of use prohibit the use of the database in other applications by downloading or through open interfaces. The trial therefore used legislative texts from the Semantic Finlex project that were not up to date. This limitation was found to have a significant impact on the reliability of the solution.
Artificial intelligence performed better in handling structured statement material
Statements can be given as “structured”, i.e. divided into different topics, or as “unstructured”, i.e. in one field. Individual statements may also include storytelling and observations referring to information outside the context of the statement, such as media debates or existing legislation. Statements are also not always given in the statement service only, but some may be given via email, for example. Collecting statements from different data sources makes data management more challenging for AI. For example, an AI system could be connected to a statement service so that it would search for statements directly from the service.
In the Prime Minister’s Office pilot, the use of the language model was tested in both structured and unstructured statement rounds. Unstructured statements were first divided into topics using AI, after which generative AI produced a summary. AI performed more efficiently in the processing of structured data.
The pilots also found that to harmonise statements and avoid meandering narratives, a clear set of questions should be used when creating rounds of statements.
2. Teaching a demanding task to AI often requires comprehensive and high-quality sample material
Teaching a challenging task to AI may require comprehensive and high-quality sample material to guide language models. Openly shared Finnish language training material, which can be used to fine-tune pre-trained language models for practical applications, is currently poorly available.
Volume of training material can also be increased with invented examples
Making statement summaries is a complex task for AI. A language model not only summarises the material, but it must identify the opinion (support or opposition) of the statement from the comments given in the statement. It must also summarise the proposed amendments and justifications provided in statements.
For this purpose, the language model had to be trained with examples on how to draw up a summary correctly. In the pilot, the language model was taught both examples collected from previous statement rounds and invented examples to obtain as comprehensive training material as possible.
3. The AI methods to be used should be selected according to the application
Functional AI applications often require the use of various language models and methods together. It may be useful to break down a complex task into parts and initially clearly define which challenge or challenges are to be solved with which language model.
Sometimes “traditional” AI instead of generative one can also be a more suitable or effective option. Experts can help understand which tasks are best suited for generative AI.
Reviewing legislation is a multi-step task for language models
The Ministry of Transport and Communications pilot focused on supporting legislative drafters in reviewing existing legislation and identifying similarities in legislation. From the point of view of the language model, the reviewing of legislation is a multi-step task, in which text is compared, searched, summarised and produced.
In the pilot, different language models were used in different phases: The Finnish Sentence BERT language model was used to compare and search for legislation, while generative language models were used to produce answers.
4. The limitations of language models should be taken into account in the design
The information in language models is limited to the material used in their training. Its timeliness depends on the time of the training. However, it is possible to enter up-to-date information into language models using different methods such as integrating external data sources or entering the information directly into its context.
On the other hand, the language model context window limits the amount of text to be processed at one time, i.e. the text input given to the model, as well as the amount of text produced by it.
Legislative drafting requires the processing of large volumes of constantly updated material
In both pilots, the applications of legislative drafting require such large amounts of data to be processed that the context windows of the Finnish language models were not sufficient to process all the text at once. In addition, the Finnish language models used in the pilots were not familiar with Finnish legislation and the special features of legal texts, as the language models in question had not been pre-trained with legal texts.
Both pilots identified the so-called retrieval-augmented generation (RAG) method as a useful solution to these challenges. In the RAG method, the language model is combined with a data retrieval system.
In the method, information related to the user’s question is first retrieved from the data sources defined for the system (e.g. websites). After that, the language model creates a response to the user using the information retrieved. The RAG method can also be used to improve the transparency of responses by including source references. This reduces the tendency of language models to hallucinate, i.e. to produce incorrect conclusions, and uses up-to-date information without continuous training of the language model.
You may also be interested in
One final note: Savings can only be gained after the initial challenges
Experiments have shown promising evidence of the use of generative AI in law drafting.
Developing a tool that boosts productivity is a multi-step process. Further development and the collection of comprehensive training material must take place in long-term and close cooperation with private-sector actors.
Various artificial intelligence pilots have been carried out in central government for several years. It is of paramount importance that the lessons learned from pilots are put into practice, and that experiments that have proven to be promising are promoted and scaled up.
Making efficient use of generative AI in combination with other technologies will allow for significant productivity gains, but savings will only start to emerge after the initial difficulties.
You may also be interested in
