Make it make sense: the challenge of data analysis in global deliberation

From climate change to emerging technologies to economic justice to space, global and transnational deliberation is on the rise. Global deliberative processes aim to bring citizen-centred governance to issues that no single nation can resolve alone. Running deliberative processes at this scale poses a unique set of challenges. How to select participants, make the forums accountable, impactful, fairly designed, and aware of power imbalances, are all crucial and open questions.

The deliberative community is just beginning to experiment with how to do this, and only through collaborative spirit will we arrive at processes that can make a difference. In my role leading the data strategy for Iswe Foundation, I am supporting their efforts to convene a coalition and institute a permanent global citizen’s assembly, starting next year.

Massifying participation will be key to invigorating global deliberation. Assemblies will have a better chance of being seen as legitimate, fair, and publicly supported if they involve thousands or even millions of diverse participants. This raises an operational challenge: how to systematise political ideas from many people across the globe.

In a centralised global assembly, anything from 50 to 500 citizens from various countries engage in a single deliberation and produce recommendations or political actions by crossing languages and cultures. In a distributed assembly, multiple gatherings are convened locally that share a common but flexible methodology, allowing participants to discuss a common issue applied both to local and global contexts. Either way, a global deliberation process demands the organisation and synthesis of possibly thousands of ideas from diverse languages and cultures around the world.

How could we ever make sense of all that data to systematise citizens’ ideas and recommendations? Most people turn their heads to computational methods to help reduce complexity and identify patterns. First up, one technique for analysing text amounts to little more than simple counting, through which we can produce something like a frequency table or a wordcloud.

Second, more advanced techniques such as Topic Modeling identify underlying themes in a collection of texts. Broadly speaking, Topic Modelling identifies latent themes (Topic Models) from different sources of text, and words associated with each theme.

Third, semantic or word co-occurrence networks show if two words (or groups of words) occur together in a sentence or paragraph. These networks help us identify which words are more central or connected to others, and from this creates impressive visualisations.

Can we interpret from these methods what citizens demand and reason? Perhaps we can produce a general impression, but in my view, these methods by themselves are not suited for managing the outputs of deliberative and participatory democracy. Big Data tools are great because they allow us to process and share insights from tons of textual data – but the loss is also great. We lose the actual political ideas that motivate participation and are left with an impressionist painting of loose words.

This is why, in most cases, participation analysts combine the Big Data methods with what I call Little Data (inspired by Norvaisas & Karpfen and mentor Coni Miranda)- examples, quotes, stories, photos, individual ideas, and other anecdotal insights that more richly illuminates citizens’ perspectives. This means walking a thin line between overwhelming readers with too much information (if we just focus on the Little Data) and losing the nuance that makes deliberation worthwhile (if we just focus on the Big Data).

It’s a fine line between overwhelming readers with too much information, and losing the nuance that makes deliberation worthwhile.

How do we go about picking the right examples for the Little Data? At least in my experience, in many cases the organiser or researcher just selects a bunch of examples based on their preferences. In other cases, people use statistical procedures that identify sentences that are supposedly representative of discussions, known as extractive summarisation. But the outcome often lacks diversity, as it typically churns out common denominators that end up being generic and bland, rather than exploring the breadth of perspectives.

Other projects have now also started using Large Language Models (LLMs) to actively create “exemplars” of citizen stances (abstractive summarisation). Under this approach, participation analysts can use pre-trained models (including commercial ones such as ChatGPT) by providing all the relevant text and asking it to produce a summary or to describe groups of opinions. However, it doesn’t make sense to fabricate amalgamated perspectives when we have actual citizen ideas that can be traced back to their local context.

Projects like Talk to The City and Fora from Cortico utilise all of these methods: statistical modelling, LLMs, and also doing their best to share unadulterated quotes and videos. This approach is definitely promising but is still in development. For example, we still do not know how trustworthy LLMs are in summarising text.

There are also more creative options. If we think of the themes or most prevalent words we identified through statistical modelling as initial filters, a more participatory, decentralised approach could work. If we believe in the democratic importance of decentralisation and participation, our data analysis strategy should also refrain from leaving all the data interpretation to a single organisation. For example, if we identify in our analysis that the concept of “clean water” is prevalent, we could take all assembly outputs that refer to that concept and send them to NGOs worldwide that specialise in clean water as “guest curators” of their top ideas.

We could even engage with citizen science approaches and have contributors help with flagging and ordering the data. This means we can engage volunteers to help analyse the data by letting them organise, classify and select ideas to show. A very similar approach is being developed by Fora in inviting as many people as possible to become “sensemakers” in their projects.

But there are other creative ways to analyse global deliberation we have not thought of yet. We envision our Global Assembly platform to be a vehicle for experimentation, and there is a lot of space for learning and experimenting. We’ve learned from Socratus in India about using drawings and visual thinking to share and materialise Little Data. Of course, visual translation also raises questions of how fairly the original ideas are conveyed. One way to address this is to make the process more accountable. We are collaborating with the Metagov team to make our tools more interoperable and open, allowing multiple actors to conduct their own analyses and challenge our interpretations. Pluralism of data strategies and interpreters should be at the core of global deliberation.

In essence, while computational methods offer incredible tools for managing and analysing large-scale data, they also have limitations. The challenge lies in balancing these powerful tools with a commitment to preserving the richness and nuance of the political ideas that fuel and emerge from global deliberation. By integrating Big Data and Little Data, along with a normative commitment to democratising how and by whom that data is curated, we can create a more holistic approach to understanding and amplifying the voices from global citizen deliberation.

About the Author
Iñaki Goñi is a data associate at the Iswe Foundation and a doctoral candidate in Science and Technology Studies at the University of Edinburgh, where he studies technology and democracy. He works in large-scale participation and participatory technologies from a critical perspective.