Slow progress towards screen scraping conversations

It has been a long time since I blogged on this site. This is partly due to my involvement in a couple of really interesting research projects (the Digital Data Analysis project of the http://www.communitiesandculture.org network with Dr Helen Kennedy and Dr Giles Moss, and the Leeds Media Ecology project which will result in a forthcoming book edited by Prof. Stephen Coleman). It is partly due to my taking over as the programme leader of the BA Hons New Media degree at the Institute of Communications Studies at the University of Leeds (most of this summer has been spent developing a new level three module entitled “Mobile Media”, looking at the impacts and influences of and on mobile communications and introducing students to the basics of mobile web and app development). So I’ve been pretty busy.

I have continued to develop my PhD work, though, reaching the stage where I am about to start to harvest a lot of conversational data from all across the web where people are talking about UK political issues. When I say a lot, I mean thousands of contributions. Hopefully tens of thousands, maybe lots more. To enable this, I managed to find three weeks this summer to devote myself to the development of a tool that will allow me to grab all this data pretty quickly and get it all stuffed nicely into my database. The Conversation Scraper is the solution I came up with – a Mozilla Firefox plug-in that operates in the same way as many screen scraping applications out there, allowing users to select parts of a web page, mark them up as a category of relevant content, building up a profile for a particular website before clicking a button and watching the data be selected and harvested automatically before their eyes -at least in theory, as long as the user has marked up the page carefully enough. The difference with this screen scraper is tat it is customised for conversation, allowing users to mark up fields like usernames, dates, message content, reply-to names and ratings or likes.

The tool works really well. I was quite surprised at how easily and quickly I could produce it (after the initial delay of trying to come to terms with how to build a Firefox plug-in). Using just a XUL sidebar and some custom JavaScript code (with AJAX and jQuery bundled in to make it all a bit nicer) I now have a fairly user friendly tool that allows me to mark up web pages and build profiles for conversation spaces all over the web. I have profiles for spaces like theguardian.com and dailymail.co.uk, some local forums and some government spaces such as the redtapechallenge and I intend to keep building up the list over the next couple of months until some big story lands and I can start to harvest conversation about it. With a few clicks of the mouse I can harvest several pages of comments from a web page straight into my database. All the usernames are anonymised before insertion, and the data is encrypted, so I don’t know anything about real individuals, I just have a large store of contributions that I can use to calculate metrics about the different conversations.

I will eventually release this tool on GPL. At present it puts data into my own database (which is no good to anyone else and is not robust enough to handle crowd sourcing) but I hope to modify it to produce a JSON or CSV export instead of a database insert, so that anyone can use it. Get in touch if you want to have a look. Maybe we could do a trade – you tell me some good ways to measure metrics like domination in a conversation and I’ll give you a look at the plug-in!

Advertisement

How much do we need to think when making decisions

There has been widespread public and political disquiet in the UK recently at the bishops in the house of lords sending the benefits bill back to parliament for amendment. Why? People suggested that there was widespread public support for the policy, the public wants a benefits cap. But why would the bishops develop a point of view contrary to that of the public? Should we be dismissing the point of view of the bishops without really considering it? Have we considered it?

Bearing in mind that the bishops, like all clergy, have traditionally been given a position of wise lifestyle advisors, guardians of morals and keepers of the truth, why is everyone so quick to remove that role this time around. Separation of church and state, perhaps. A fine principle, and far be it from me to defend the role of organised religion in politics. But are all the objectors doing so because they do not believe unelected individuals, wielding power purely because of religious belief, should not be allowed to exert influence over policy? I thought people were actually outraged because the bishops were at odds with public opinion… It was widely reported, at least, that the latter was the more relevant complaint on this issue.

So I come back to the issue of developing opinion. The bishops have long been trusted as wise sages, worthy of respect and with opinion of great value. Why is this, and why don’t people accept it on this issue? My guess is that bishops (and other clergy) have pondered, considered, deliberated over issues before forming an opinion. I think this is part of their job, part of their raison d’etre. The fact that there is (reported) widespread disapproval of their “veto” of the bill perhaps indicates that the public are missing something. Why have the public come to a different conclusion than the wise old bishops? And why isn’t anyone flagging up concerns about this?

Perhaps if all members of the public engaged with the issue at hand as closely as the bishops have they would have a different opinion. Perhaps they would see different viewpoints, consider different consequences and weigh up all of the outcomes instead of simply the most accessible and obvious.

The public, of course, cannot engage with every issue in great detail. We all have lives to lead, jobs to do, families to care for. That is why we elect people to do the consideration for us. Maybe we should trust them a little more. Of course, we can’t just let them get away with everything. We have to hold them to account. Our democratic system is certainly not good enough for us to be realistically represented by the options available to us on the polling card. So the public has to exert pressure over issues to ensure the democracy works.

How, then, do we address the issue of a public that needs to be involved and a public that is not discussing the issues? Well… we could talk to one another a bit more… including those people that do not have the same views as us. We could have cross cutting debate and we could deliberate.

And so to familiar ground. How can 65 million people deliberate about public policy issues? Of course, we don’t have to do it all together, we just need to be exposed to a wider cross section, to talk to a wider range of people about a wider range of subjects and, as Diana Mutz would say “hear the other side”.

Online deliberation holds so much promise for this. Whether it is Channel 4’s WifeSwap forum, Wikipedia or the Red Tape Challenge, people have used the internet to discuss public issues. Of course challenges remain and we are yet to find the key to tying online conversation to political participation. But if we can just get people discussing things a little bit more widely, perhaps the bishops won’t appear at odds with us all so much after all.

Digital Democracy: More than Public Opinion Pieces

The UK coalition government today published the list of e-petitions it has received so far through its e-petition website. Top of the list of topics that may be debated in parliament is a request to bring back the death penalty in the UK. Apparently, of the first 200 petitions received, 40 called for the return of capital punishment, while 7 called for it to remain out of the UK justice system. So how does this initiative perform as a tool of online democracy? Citizens get to have their voice heard, at least if their opinion makes it past the moderators and enough people agree with them that MPs take notice. As Amnesty International’s Jeremy Croft said on BBC news: “It is democracy… of a sort“. It is democracy of the populist sort, where the debate is done in Westminster and not by the public. Of course many participants may have discussed and deliberated issues and thought long and hard before choosing to support a petition. Others won’t have. The platform for participation does not incorporate any facility for exchange of views or consideration of alternative ideas or opinions. It is simply a space for citizens to express an opinion and hope that MPs will take note.

This is the model of participation that many believed would be empowered by the internet as it allowed citizens to speak freely and equally, wrestling control from the institutions and handing it to anyone with a phone line and a keyboard. But as Hindman (2009) points out, social hierarchies and winner-takes-all power laws have been quick to materialize in the on-line world, just like those in other areas that have been criticised so much. Hindman argues that this demonstrates that the internet is not the democratising tool that many hoped it would be but instead simply mirrors the patterns seen in traditional media. The e-petition initiative tackles this problem by at least providing a direct route to Westminster and the potential to get an issue on the floor of the commons. However, if Hindman is correct and the fractal structure of the internet pervades this tool as all others on the web, while some issues prosper and gain support other, possibly equally valid, petitions will fall by the way side simply due to not having the weight of numbers to be noticed by the public. As the example of the capital punishment petitions shows – 40 different petitions were set up instead of one petition with 40 times more signatures. How many other issues are represented by many, tiny unnoticed petitions?

Perhaps it is too much to ask for any technological innovation to ‘democratise’ a political system, as what is really required is action by the public to break down the barriers that segregate people and ideas. Segregation between social groups – genders, ages, ethnicities, income levels, education. Segregation between political ideologies. One method that could help to break down these barriers is cross-society communication between citizens. People talking to each other, learning about the viewpoints of others and reflecting upon them them when forming their own opinion. In other words, public deliberation. But I’ve talked about that before. Public deliberation and the potential of the internet, through the facilitation of large scale conversation, to enable it. This is not the result of the “trickle-up discourse” which Hindman states has failed to materialise; it is the potential for actual interaction, conversation in forums across the internet. Not publishing content and fighting for readership or signatures but inter-personal communication between individuals from diverse groups. So I am therefore presented with the following paradox: Technological innovation cannot democratise our society, people must do it. But people may be enabled to do so by technological innovation. The internet cannot democratise politics, yet it can enable citizens to do so. Maybe I haven’t thought that through, but the more I do so the more it makes sense. Empowering the public to shout out their views is fine. Encouraging them to discuss, share and consider each others views is better.

Hindman, M., 2009, The Myth of Digital Democracy, Princeton University Press

Beyond the AV debate

A recent poll by the Institute of Public Policy Research showed that support for a switch to the Alternative Vote system (AV) from First-Past-The-Post may be growing in the UK. One of many, often contradictory, polls this one is just a small part of the growing debate and speculation over the potential switch to a new voting system. Various arguments have been made by the ‘No’ and ‘Yes’ campaigns, touching on issues such as public engagement, enabling voices, empowering the people, representativeness, extreme views and minority parties. This list of discussion topics is by no means linked only to the technicalities of a voting system, however. Indeed, one might wonder whether the issues are actually a product of the voting system at all.

Poor public engagement and voter turnout have been cited as examples of a failing democracy in the UK and reasons for the lack of public engagement have been discussed. Some blame a lack of voting choice but others hold to account the perceived lack of accountability of MPs or distrust and cynicism amongst the public as well as a feeling of detachment from decision making, often linked to the limiting of public involvement to a single vote per parliament. Such public disengagement can negatively affect either voting system by encouraging poorly informed voting and agonistic politics. Indeed, some have argued that a weakness of AV is its potential to give greater power to extreme views through increased support of minority parties. This is a debatable theory rather than a fact and perhaps misses the point about our democratic failings. The debate should not be about how can we constrain, silence or deny these extreme views through design of a voting system but how can we encourage the public to engage deliberatively, exploring issues and exchanging opinions so that individuals can participate in a more informed and enlightened way. Improving governance through informing and consulting the public, encouraging deliberative and collaborative interaction and integrating public feedback into policy making are popular topics raised by many scholars. Coleman and Gotze (2001) addressed the issue, showing how engaging the public in more deliberative activity can transform political involvement from preference assertion to preference formation. They described how online spaces have the potential to facilitate mass conversation and deliberation, exchange of views and information and ultimately more considered political involvement.

While electoral reform is currently all over the UK blogosphere, there seems to be a lack of formal public deliberation about the issue. Interestingly, a series of offline debates has been held to allow citizens to deliberate electoral reform. Recorded and made available on the internet, it is notable, however, that this is not accompanied by an online debating facility to allow a wider community of citizens to discuss the issues themselves. The current focus on the voting system itself is perhaps hiding the issue that needs our attention – whatever the voting system, voters should be encouraged to form considered, well-informed opinions.

Ref: Coleman, S. and Gotze, J. (2001) Bowling Together: online public engagement in policy deliberation, London: Hansard Society, 2001

Deliberation in e-Participation

In a PhD supervision meeting with Professor Stephen Coleman, we reflected on the role of deliberation in e-participation. Drawing a distinction between collaboration and deliberation and noting the differences between talk, political talk, deliberation and political deliberation, we talked about the theory behind deliberation in a way that I did not do in my previous blog post about system design. Nearly all of the papers that I discussed in that post devoted some space to non-technical considerations of deliberation, examining the requirements for off-line, face-to-face deliberation, discussing Habermasian theory of deliberation and previous work based upon it. Beth Noveck, for instance, drew on theoretical requirements for deliberation when producing the list of values that she recommended be incorporated into the design of online deliberative systems. Deliberative theory will of course form part of any more formal writings I produce on the subject.

We discussed some of the different platforms for deliberation, from face-to-face conversation to asynchronous online discussion in forums as well as innovative new forms of engagement such as conversing avatars in virtual worlds. We discussed the difference between procedural deliberation (movement towards any possible outcome, but through specified procedures) and substantive deliberation (free discussion that progresses towards an outcome that lies within an allowable range) and also talked about the counter-opinion to much that I have read so far: that deliberation is not the answer to democratic problem solving and policy making. This agonistic opinion prefers the rational choice – doing what the majority indicate their preference to be (“counting heads”)  or encouraging the poorly represented to organise themselves so that their voices are heard. The agonistic pluralistic argument embraces the differences of opinion found in politics and seeks to use them constructively. Such an argument can be influenced by a range of factors such as real-world pragmatism or behavioural theory (for instance, that deliberation may fail to take into account the emotions and strong feelings of human beings, such as our innate reasonableness that may keep us quiet during a debate instead of voicing opinions).

This discussion of deliberation led us on to the topic of consensus – often an important issue in design of deliberative systems as it can be seen in some cases as an optimal end point for deliberation on which policies can be based. In collaborative systems, such as wikis or document-centred discussion, this is an understandable position though its shortcomings are illustrated by the example of a controversial topic on Wikipedia which lacks content on the actual encyclopaedia entry but has an extremely lively and full talk page behind it. The opinion that “if the public are well enough informed and deliberate in a helpful enough manner, they will eventually agree with the ‘correct’ viewpoint” often fails, of course, because the “correct” viewpoint seldom exists. In reality, while some may be embarrassed into backing down through deliberation, a consensus will often not be reached and minority views will remain. However, contrasting arguments may be exposed; specific interests may be identified, preferences may be put forth and immovable values may become apparent. This is clearly a crucial issue in public participation, and therefore e-participation, as to truly integrate participant opinion into policy making – identified by many as a key dimension of successful public participation initiatives – the representative data from a participation system must be captured and acted upon, even though a convenient consensus may not be present.

The above discussion perhaps suggests a need to draw a line between deliberative and problem-based collaborative system design. Engaging the public as a means of finding opinion and exploring social norms is different from engaging the public to solve a particular problem or create a particular policy. The former might be much less constructive and more observational, whereas the latter may only succeed if a rigid structure is used to drive participants towards a constructive outcome. Of course, an e-participation system is likely to be neither of these extremes and there may be room for a deliberative aspect within a problem-based framework. Having read recent work by Mark Klein et al., I was impressed by the careful structure built into their deliberative platform deliberatorium to create a highly social dialogue from participants’ contributions. Stephen mentioned, however, that this rigid structure may exert its own effect on the conversation which made me think about whether the deliberation supported by systems such as deliberatorium and Douglas Shulers E-Liberate is equivalent to deliberation allowed by unstructured conversation platforms.

So I am left with a few points with which to move forward: the question of how designing to encourage deliberation, collaboration or consensus may affect the type and quality of deliberation contributed by participants; the question of whether deliberation is the highest priority in e-participation system design or whether pragmatic demands of usable content and outcomes may require a more agonistic approach; finally, the question of how well the various possible data structures harvested by e-participation systems can be utilised in the challenge of integrating contributions with policy and law-making precedures – though as this is not just a data-centric question it will perhaps require more institutional research, rather than the experimental, technical literature focussed upon so far.

Mediating debate for large scale argumentation

Mark Klein and colleagues recently developed a system for large scale online argumentation which goes some way to answering the question I posed in my last blog entry about how design can enable large scale deliberation within the bounds of structured AV platforms. Defining deliberation as “exploring and converging on problem solutions rather than just… conversing“, the team developed Deliboratorium, a tool to harvest large scale discussion in a tight, argument-structured way, based upon the IBIS argumentation formalism. Allowing free choice of topic the Deliboratorium system is not a problem-based system but it does enforce a problem-solution-argument model that creates an easy to analyse contribution structure as well as encouraging contributors to look for related and contrasting ideas and to encounter opposing arguments before contributing. Like Raymond Pingree’s Decision Structured Deliberation (DSD), the Deliboratorium design enforces typing of messages (issue, idea, pro, con) by the contributor prior to submitting and also prohibits replication and insists upon posts being placed in a logically sound part of the argument map. The system requires moderation in order to ensure posts are structured, categorised and placed correctly and are of good quality. The moderation is not silent, however, as moderators have a “part education and part quality control” role and can communicate with contributors to help them to produce acceptable posts. Klein states that one moderator is required for each twenty contributors an effort level “much lower than those needed to harvest, post-hoc, discussions hosted by such conventional social computing tools as web forums“.

The team acknowledge the challenge of “attention allocation” and the need to help users to find areas of interest within large argument maps and find the appropriate place to add their views and expertise. To this end the developers included several popular web 2.0 solutions such as ratings systems, watchlists and personalisation through the use of personal homepages. Each of these facilities comes with its own risks to democracy and deliberation, as discussed in previous blog entries, but they serve as a method of navigating the large map, enabling contributors to home in on areas of interest, making the large scale of the conversation less of a hindrance to participation. Furthermore version histories, of the type found in popular wiki software, are kept so that editorial control can be maintained as a defence against contributors changing each others arguments rather than sticking to the “live and let live” principle and constructing their own counter-argument.

Deliboratorium is an exceptional concept in large scale deliberation with tremendous potential to allow large scale deliberative participation within argument visualisation systems. It does, however, work towards a very specific definition of deliberation. So far the system has been tested on groups numbering in the hundreds that seem to be somewhat prepared for use of such a system, such as information management students. I look forward to seeing how well the innovative structure translates to use with the general public, if that occurs, and whether the 1:20 moderation level remains at a manageable ratio and the quality of deliberation remains as high as seen in the trials so far.

Designing for deliberation (continued)

To add to my previous post on designing deliberative systems I have been reading Online Deliberation: Design, Research, and Practice by Todd Davies et al. (2009), particularly Part VI: Design of Deliberation Tools, as well as an interesting paper by Deen G Freelon from 2010 about models of democratic communication and their effect on online conversation.

Chapter 25 of Online Deliberation, by Davies et al, featured DEME – an innovative design for best practice in online asynchronous small (to medium) group deliberations. The system included a variety of features to improve deliberation such as easy access to related information, formation of smaller topic-based groups from the larger overall participant community, built in tutorials and features to encourage directed discussion. There were also features to enhance collaboration such as document centred discussion and the sharing of files and links. One important feature was the incorporation of email into the system. The authors stressed the importance of incorporating current practice into systems to aid participation and email was seen to be a dominant communication method among the participants outside of the DEME environment. The authors also highlighted how the early design of the system limited participation due to a confusing interface. This problem was solved by a redesign, utilising newer technologies that had arrived during the project timespan. This example was used as an illustration into the importance of designing the system codebase for incremental improvement – the internet is a fast moving environment and systems must be adaptable to current expectations of the modern user.

In chapter 26 Douglas Schuler introduced E-Liberate: a tool for online civic deliberation. This system was built around the use of Roberts Rules of Order – “a set of directives that designated an orderly process for equitable decision making in face-to-face meetings“. Roberts Rules of Order are used by many organisations and, in the US at least, the directives are legally mandated for use by governments. The directives involve the “typing” of messages into discrete groups – a practice common in conversation mapping and analysis and one that makes discourse particularly suitable for computer processing. The structuring of discourse in this way allows the system to facilitate conversations by enforcing the rules and only allowing legal “moves” such as posing and responding to questions. Schuler admits that this is not always popular with users and built in an “auto pilot” feature that allowed conversation to move more freely if users deemed that the facilitation was impeding conversation. Schuler also addressed the issue of ascertaining who is online at any particular time. Asynchronous conversations are often assumed to be inclusive but at any particular moment it is hard to tell whether a quorum is present or not. Schuler states that solving these issues will take “social as well as technological approaches” such as windows of time where commenting can take place or comment quotas for participants.

Roberts Rules of Order are not the only framework for structuring collaborative or deliberative debate and in chapter 27 Shanks and Dahlstrom describe the Parliament system which can use an external set of rules to facilitate an online discussion. Using a specification language the user can describe new rule sets that meet their particular needs. This functionality helps to ensure that useful discussion is not constrained by limitations in software structure.

Chapter 28, by Raymond J. Pingree introduces Decision Structure: “a new approach to three problems in deliberation“. Pingree describes how off-line discussion is “often assumed to be the gold standard of deliberation” but argues that deliberation is not automatically achieved in offline discussion and designing to mimic it online is not always the best policy. Indeed, doing so can mean neglacting other possibilities of online discussion that could improve deliberation. He suggests that designers of online forums “should instead strive to take advantage of the unique design flexibility of the online discussion environment” to design towards deliberative ideals rather than offline characteristics. To an extent, I think my previous review has highlighted attempts to do just this. Pingree describes how some problems are outside the scope of design, such as diversity of participants views or the willingness of participants to follow deliberative norms but he goes on to describe three problems in deliberation that should be the subject of attempts to solve them through design of online systems. First he addresses the Problem of Scale. Large group deliberation often suffers from problems of coherence and full reception, problems largely mitigated by the written nature of online asynchronous conversations. In these environments “the problem of scale manifests as a difficulty in keeping up with all messages being sent“. Secondly, Pingree describes the Problem of Memory and Mental Organisation, in which the the limitations of human memory impede deliberation, requiring design that assists human memory. I am immediately drawn to my previous notes on argument visualisation as an example of this. Thirdly, Pingree describes the problem of Conflict between Organisation and Democratic Legitimacy, the topic of moderation and facilitation and its effect on democracy that has been discussed by others, some of which is summarised in the previous blog post. Pingree proposes a new model of online deliberation, Decision Structured Deliberation (DSD), that may be used to solve the three problems outlined above. The model combines some of the previously discussed topics such as message typing, profile and trust based systems and ratings systems. The message structure is imposed on users, who must choose a category of message for their contribution when composing it. The structure is “more specific than the mere reply relationships found in existing forums” and is configurable by an administrator to fit a particular environment. Potential types include “problem“, “solution” and “cause” and contributions of these types can build up a conversation structure as discussion occurs. For instance, problems can have causes or solutions proposed and reasons added to back them up. Any number of such problems, causes, solutions or reasons can be added but crucially, the users can vote for or against each, building up a picture of valued contributions which can be presented in ranked order. This user-created argument visualisation seems to present a real opportunity for harnessing mass contributions and building deliberative, democratic consensuses. The problem that immediately jumps to mind is that of the effect of coordinated contribution such as the work of lobby groups. The ratings and profile system could play straight into their hands and such environments could become battle grounds between organisations.

Turning to Deen Freelon’s article, it was interesting to read his account of the widely varying patterns of online political conversation created by different types of community – an effect often overlooked in research about design for deliberation. Freelon describes common strategies of research into online political discussion spaces as either too focussed on purely deliberative quality, ignoring other “equally compelling conversational phenomena”, or studies of diverse systems that fail to categorise each in “ways that are systematic, commensurable and tailored specifically to online discussion”. Freelon suggests a third strategy, based on work by Habermas and, later, Dahlberg, which takes into account the “disparate online discussion cultures” of different communities and allows spaces to be characterised using “distinct scholarly conceptions of democracy“, taking into account types of online political discussion, other than deliberation, that can also be democratically useful. Freelon describes three characteristics that Habermas stated should be present in the public sphere: rational-critical discussion; defined limits of scope of discussion; openness to all members of the public. These criteria can be included in definitions of deliberation and as such deliberative quality has been used as a “yardstick” to assess democratic performance of online spaces. Freelon asserts that “the key disadvantage of this approach is that by focusing solely on characteristics of relevance to deliberation, it ignores many other theoretically interesting features of online political deliberation“. Freelon also notes how increasing deliberation may decrease participation, and vice versa, due to the exposure to opposing views of true deliberation, demonstrating the need to look beyond single democratic ideals and look for neccesary trade-offs between ideals and customs. There have been a number of studies focussing on the different types of online public spheres, notably Dahlgren (2005) identifying five categories of online discussion space (e-government, advocacy/activist, civic, parapolitical and journalistic) and Pickard (2005) identifying three categories of “internet-based grassroots action” (partisan, pluralist democractic and radical participatory) but Freelon describes a lack of consistent analytical criteria and a lack of framework for assessing deliberative quality in these articles, respectively. He describes an alternative model that improves on previous studies in three ways: allows a broadened scope of evaluation, encompassing more than just deliberation; specifies categories of communication rather than sponsorship or institutional linkage; allows the connection of empirical results to political theory. The framework identifies three distinct models of democracy from Dahlberg’s work: liberal individualist; communitarian; deliberative; and allows “more precise conclusions such as ‘communitarian with some deliberative aspect’” rather than simply “more or less deliberative“. Freelon illustrates how these different communities behave differently on the web and shows how his three-model framework can be applied to current areas of research through analysis of political conversation rather than just deliberation. He postulated that the three-model framework could provide insight into how configuration of discussion spaces may affect the type of community dynamic, and therefore discussion structure, that is developed as well as analysing the affect of political influences on design and how design for one of the models of democracy in the framework may be as influential in fostering democracy as design for deliberation.

There are two notable concepts that have remained in my mind after reading the above literature, that of innovative structure of message dynamics illustrated by Raymond J. Pingree in his Decision Structured Deliberation and Deen G Freelon’s account of democratic model and the need for incorporation of online community dynamics in research and evaluation initiatives for e-participation. The former is an illustration of how thoughtful and innovative platform design can potentially structure large masses of contribution into useful conversational materials that lead towards consensus, are suitable for computer analysis and can be integrated into policy formulation. The problem of attracting and encouraging online individuals to sites of such design remains, but the work does address the issue of handling input should they choose to participate. The latter is an extremely interesting account of online community dynamics which shows a clear limitations of evaluation simply by deliberative quality of content. When designing the deliberative and collaborative e-participation systems of the future, models of democracy and the relationship between system design and participant community may be an important area of consideration.

Designing for deliberation

I’ve been pulling together bits of background reading over the last two weeks, particularly in the area of interface design for e-participation and designing systems for deliberation. This is far from an exhaustive list of papers, but I hope I have covered the main points regarding design for deliberation and e-participation.

An important starting point for  electronic enhancement of deliberation may be seen as far back as 1970 with Kunz and Rittel and their IBIS collaboration framework, then Conklin & Begeman in 1987 developing a GUI on top of IBIS (gIBIS) – an early example of the harnessing of technological advances to enhance collaborative processes. This framework was actually “a deliberative platform for design” in the software industry rather than a design for deliberation but serves as a good basis to start the discussion of collaborative and deliberative connected software. Online collaboration proliferated with the growth of the internet and online communities through the nineties, but there was a noticeable scepticism with which this was treated by some (in contrast to the media furore about the dot com rise) – various authors pointing out weaknesses in the internet model and in particular two concerns: first, the “hypersegmentation” created by multiple channels and the “digital loneliness” that this leads to; second, the creation of enormous quantities of public content – or “masses of gibberish” and “mere chatter”.

In fact, in terms of deliberative content, by 2003 when Beth Noveck wrote at length about the subject, deliberative systems were extremely rare and only a handful (she found seven examples worldwide) of e-participation initiatives existed. She states that “the absence of appropriate technology to transform private conversation into public deliberation is at the root of electronic democracy’s stunted growth“.  Deliberation is a “function of a particular type of structured speech” and in cyberspace the architecture, the code itself, “directly shapes and structures conversation“. To tackle this problem Noveck create Unchat an online, real time (synchronous) discussion tool which enabled a global community of invited lawyers to participate in type-written (as opposed to speech) group conversations. When developing Unchat Noveck discussed the factors affecting, and required for, deliberation offline and developed them into a set of values to which online systems must adhere in order to foster real deliberation. They are listed below, along with explanations of how Unchat design implemented them:

• Accessibility – Noveck discussed technical accessibility but not issues such as the digital divide
• No censorship – freedom of speech must be protected
• Autonomous – allow users to configure the system and set their own rules
• Accountability and relevance – minimise anonymity to create accountability, but blended anonymity model suggested
• Transparency – Conversation archives kept and web logs stored for analysis.
• Equal and responsive
• Pluralistic and inclusiveness – role-based permissions and moderation, set up to reflect the community
• Appropriate moderation/facilitation – facilitation is “a clear risk to democracy” but can also improve conversation and teach people to deliberate and participate effectively. Facilitation can occur but flexible models should be available (e.g. moderator can be elected/deposed and can give private or public feedback during facilitation)
• Informed – data summarized and stored, presented to participants, transcripts available to latecomers to “catch up”. Noveck also hints at the possibility of post-debate content analysis
• Speed bumps – the navigation system forces participants to be exposed to relevant information before entering a debate, encouraging them to read it and become informed, rather than just heading straight in and talking. Taking this a step further, a quiz was introduced before debate to expose participants to key concepts and arguments, possibly design to target arguments that they have not previously considered.

Of course, there is more to an e-participation system than simply deliberation. Factors such as integration into policy making also influence their effectiveness. Ann Macintosh has written a number of articles about evaluation of e-participation systems producing a framework to be used for the purpose. In 2004 she described ten key dimensions of e-participation initiatives:

1. Level of participation
2. Stage in decisionmaking
3. Actors
4. Technologies used
5. Rules of engagement (privacy, registration, site rules)
6. Duration and sustainability
7. Accessibility (digital divide and WAI)
8. Resources and promotion
9. Evaluation and outcomes
10. Critical factors for success

A lot has been written about the topics included in Noveck’s values and Macintosh’s key dimensions. James Fishkin wrote about his “Deliberative Polling” platform in 2009, addressing points of accessibility and representativeness, highlighting the difficulties of engaging with representative samples and the effect of pressure groups and lobbyists on e-participation initiatives as well as the deliberative costs of systems that attempt to solve some of these problems without the others. Noveck, and similarly Fishkin, was convinced of the deliberative qualities of synchronous over asynchronous deliberation, due to the time commitment required to participate fully in an asynchronous debate. Cavalier, Kim and Zeiss, in their 2009 paper about the PICOLA project, also preferred the synchronous method of deliberation, claiming that use of new technologies in carefully designed interfaces could replicate the level of deliberation of face-to-face conversations. Noting the scheduling difficulties of synchronous participation, the group combined this tool with asynchronous discussion areas to create a 24-hour platform for participation. Tucey (2009) described weaknesses in synchronous models such as difficulty in expressing an opinion due to speed of conversation and suggested a hybrid strategy in which highly engaged groups might interact synchronously but with limits to their frequency of posting. Tucey, like Noveck, also advocated breaking large groups into smaller ones (up to 24 people) in order to replicate the deliberative quality of face-to-face conversation. Both Tucey and Fishkin highlighted the importance of repeat interactions between participants, perhaps a requirement to discuss issues weekly for several weeks, in order to help them get to know each other and understand their contrasting ideas. In fact, asynchronous debate ( in the form of bulletin boards/messageboard/forum systems) has dominated internet participation. Ann Macintosh wrote in 2004 “Typically e-engagement is based on discussion forum technology” and described online communities based upon discussion forums as examples of empowerment but described how e-engagement initiatives of this form imply that an indication of level of agreement with proposals is sought. Tucey (and others before her such as Wright, Coleman) suggested that moderation can help to increase deliberative conversations in groups but can be impractical at times and should be tailored to the group involved. Scott Wright wrote about “The necessity of moderation” in 2009, citing Kearns et al., Barber and Blumler & Coleman when describing how moderation, and indeed facilitation, can be vital in turning the uncontrolled expression of free speech into more focussed and useful discourse. Wright stated that moderation was justified as the anonymity and physical separation allowed by the internet causes behaviour that requires moderation. He described two models of moderation (as well as the third model of no moderation): content moderation, in which humans (and also possibly automated programs) pre-moderate content against pre-defined criteria, and interactive moderation, in which the moderator acts as a facilitator, giving feedback, supplying resources and directing the conversation in productive ways. Wright also illustrated the problems of poorly designed or implemented moderation strategies and the necessity of distinguishing legitimate from illegitimate moderation by highlighting the problems that the latter can bring, particularly for governmental platforms where the issue of censorship may be raised.

I was particularly struck with the  2007 paper by Schlosberg, Zavestoski and Shulman, which analysed a number of online initiatives including e-rulemaking and showed how online systems can be just as deliberative, though not more than, offline methods. They described with clarity how internet systems can either constrain or promote deliberation and showed how functionality and interface design can be an important factors in deciding how deliberative input will be. Scott Wright and John Street also wrote about the importance of design for deliberation in 2007, illustrating how design determines deliberative quality, but can either facilitate or impede it. They stressed that they were not describing technological determinism, partly as the technology was not a guarantee of the use it would generate, but also because the technology used was often the result of political choices made when the system was designed and commissioned. It was these choices, they argued, that were influencing the quality of deliberation rather than the technology itself.

Scott Wright’s later work stressed the importance of looking for trends across platforms and not focussing just on new innovations and technology as well as the importance of less “institutional” discussion fora. Many non-political sites host somewhat deliberative content and Scott has stressed that these “third spaces” are potentially an untapped resource of political opinion, or at least a model of engagement that could be used as an example in future work. This led me to think about the real strengths of the internet and where its potential lies. There is much debate about the usefulness of online platforms for deliberative purposes. For sure, small scale deliberation can take place on specialist platforms and deliberation occurs to varying degrees in asynchronous fora. But how do we effectively design for representative engagement and deliberation on a very large scale, harnessing the greatest strength of the internet: the ability to connect and promote interaction between large, national and even global communities? As Scott Wright showed, there are communities out there in cyberspace, willing to discuss politics and even deliberate. How do we engage those people in constructive ways? Beth Noveck has shown clearly with her peer-to-patent platform that online collaboration is possible. Increasingly, numerous web platforms offer commenting services that prove to be wildly popular, if not deliberative. So how do we turn this potential and enthusiasm into integrated, deliberative political participation?

Aside from the problem of recruiting participants and keeping their attention and faith, a possible method for maximising the usefulness of their input may lie in structured argument visualization (AV) interfaces which combine many of the technologies, designs and principles of the systems mentioned here. Ann Macintosh has written a number of papers describing how computer supported collaborative argumentation (CSCA) or computer supported argument visualisation (CSAV) (possibly based on IBIS) could be utilised to provide graphical representations of arguments to enable better deliberation. Simon Buckingham-Shum has written a number of texts describing the problem of knowledge representation and management and the potential of technology to provide platforms for successful visualisation of knowledge and argument in public participation decision making and planning systems. He has been involved in a number of innovative public-participation initiatives including the development of tools such as Compendium, a “hypermedia and sense-making” tool used to structure and represent contents of public planning meetings which can be used to inform web consultations (and vice versa).

Argument visualisation has been shown to be of great potential in a number of policy making situations. But there are parts of these models that can be looked at in more detail. Presenting information and challenging, even educating, the participant is important and the visualisation techniques required to help participants make informed contributions are vital. However, providing a platform on which they can contribute deliberatively, to interact and work towards a consensus is also vital. Allowing reciprocal and networked input is vital for deliberation, though structures to allow this on a very large scale are few, if any. Furthermore, large-scale contributions need to be analysed and integrated into decision making. The list of values above is applicable to every part of these AV systems and there is room for research into how interface design, social technologies and content analysis can be best combined to produce effective very large scale deliberative systems.

Dr Simon Smith: Online research – analysing forums

The latest PhD seminar featured Simon Smith presenting to us two pieces of research that he had worked on: a health management project in digital inclusion and an international local e-democracy project. Both projects involved the use and analysis of online forums but differed in the communities of participants involved and the methods employed to facilitate their use. The two projects presented interesting, and contasting, challenges to research which Simon presented under two broad themes: ethical issues and validity of claims about forum content.

The first example was a piece of action research in which a community of older people with a particular illness were encouraged and facilitated to manage their illness using online tools provided by the researchers. The participants were encouraged, in a very hands-on way, to form online discussion groups through the use of free handouts of computers and connections and real-world focus groups to initially ease the participants into interacting virtually. The online environment consisted of forums that were open initially to small groups of users and later the entire group. There was also an instant messenger (IM) service which the researchers assured the participants would not be monitored or analysed. The researchers analysed the content of the forum messages for information such as “self reported health” (in which they found positive trends as the illnesses were more effectively managed, though GP visits initially rose as better informed patients sought information). A full suite of web analytics was also employed so that usage trends could be monitored as well as the discourse analysis.

The second example was an ethnographic study built upon a previous EU e-participation strategy, concentrating on local issues discussion forums in the Czech Republic, Slovakia and the UK. These forums consisted of user created threads (not seeded by councils) and the researchers monitored these over the long term to build up a picture of the people contributing and the topics covered using methods such as “qualitative meta-reading” or keyword searches. Issues such as social inclusion, diversity, local or collective identity were investigated, high volume users were identified as well as the level of intermediation (people representing groups). It was found that, although the users did not constitute a representative sample, socially marginal groups did have a voice through people voicing opinion on their behalf. The researchers also investigated how the topics discussed by contributors compared to topics covered in local media, council minutes etc, to identify differences in topics reported and topics discussed by locals. Interestingly, they noted that many of the topics discussed on the forums went on to form discussion topics in council meetings.

Discussing ethical issues raised by the two studies, Simon highlighted the uncertain boundary around online social data and the ethical duty implicit in reporting of it. The health management study was set up as a medical intervention and operated under informed consent. However, the discussion area was specifically designed for inexperienced users and the final structure was not exactly known at the beginning of the study so for this reason the researchers needed to address concerns by negotiating privacy terms throughout the study, rather than relying on the catch-all agreement made at the outset. The researchers also defined boundaries of privacy, signposting private areas, such as the IM service, that were not monitored or analysed. Further ethical issues arose due to the fact that full web stats (log files containing IP addresses etc.)  were also collected for analysis, a step possibly not understood by the participants. Simon explained that the researchers worked on a principle that they should not “exceed reasonable expectations” in terms of personal details designed their research accordingly. They even took the step of presenting preliminary drafts of reports to the participants for comments before publishing. The ethnographic study presented slightly different challenges. The participants were discussing public issues on a public forum but the local focus of the environment made it difficult to asses what people considered to be public and private. People were also able to post anonymously, giving an air of protection against identification. Both studies highlight the importance of considering what is quotable in your research and what is not. It is important to look at how people are using an online environment and developing norms (just like in “real”, offline environments). For instance, the difference between the in perception of privacy of IM due to its assumed ephemeral nature and that of a publicly archived forum. Finally Simon discussed an interesting view of the ethical use of online data when considering authorship issues and intellectual property rights. Researchers often try to avoid privacy issues by anonymising data but we need to consider the filp side of this: do we need consent to use a persons contribution to a forum? Should we be citing their name? The specific environment used must be considered before deciding where ownership of content lies.

Both of the studies described presented findings about the content of the forums and conclusions drawn from it. Simon addressed the issue of validity of conclusions drawn from data from discourse/content analysis. Technology is sociologically constructed  and its use is socially mediated – we don’t harvest any details about the rest of the persons life and it is is often impossible to contextualise their opinions in terms of their individual situations. The knowledge upon which a contribution is based is not neccesarily readily apparent. However, online environments do lend themselves to the collection of sociologically rich data as people may be less inhibited (perhaps if shielded by the cloak of anonymity)  with regard to contributing personal details. Participation is recorded in its entirety, in situ, and contextual knowledge can be included as the participant takes time to formulate a response, adding details and links to illustrate factors that have helped to build a viewpoint. Falsehoods may be rarer as they stand as a record to be challenged by large numbers of online participants.

The validity of conclusions drawn from online discussion is particularly pertinent to my own research. Any e-participation system designed to harvest public opinion must be designed in such a way to ensure that accurate contributions are solicited and the collective opinion of the virtual online community compares as closely as possible to its real world offline alternative. Indeed online systems can go beyond their offline equivalents to produce a method of soliciting opinion that is free from the factors that can degrade its quality. Structures have been built in to a number of e-participation systems for just that purpose. In Beth Novak’s  peer-to-patent system, communities and tasks are organised in granular fashion with clear goals and expectations to ensure that a task is completed in efficient manner and a reputation based system ensures quality comment from trusted individuals. Debatewise.org creates space for debate that is strictly structured into chunks of contributions, for or against an argument, which can be voted upon in an attempt to create a consensus of opinion. Innovative designs such as these will be crucial in the development of more effective e-participation solutions and investigation into them will form a crucial part of my research.

Supervision meeting three

After my meeting with Ann, Stephen was aware of my difficulties in projecting a structure of research that is worthy of PhD. I talked about the areas I had worked on: case studies; different technologies and ways of looking at e-participation; conversation mapping for analyzing content; argument visualization; interface design and its effect on input. We then started a discussion back at the basics of what is needed for a PhD and what I wanted to investigate: deliberative content? efficacy of an initiative? success of a project (could be in terms of attracting an audience, as well as other factors of success)?

These questions influence the different types of research questions that might be generated: How does interface design effect participation? How does interface design affect deliberative quality? How can we visualize or assess deliberative quality? How can large scale arguments be visualized? These questions should be formulated based upon my interests, as well as a gap in the research.

Stephen highlighted the potential weakness in this approach: is it actually a sociological question? Do opposing groups deliberate less and resort to “flaming” or other polarized techniques, whereas closer communities deliberate more easily? Are more cognitively challenging subjects more or less likely to create deliberation?

I described how I wanted to find ways of summarizing input, helping to create useful data from the mass of large scale social input to a participation platform but described my concerns that the analysis of data using conversation mapping and argument visualisation is “muddied” by the effect of interface design on the quality of data. Stephen remarked that the questions relating to data analysis/visualization are therefore separate from questions about interface design, but postulated that we could link the two. Interface is structurally determinant, has an effect before the conversation is started whereas visualization happens post discussion. Inquiry along the lines of “Integration of design and analysis technologies to create successful e-participation initiatives” could be a 2-3 stage process:

How do you design to allow for deliberation?

• Examples of participation initiatives
  Including GIS pParticipation, online forums, voting systems, etc.
• Literature about interface design – HCI, usability, accessibility as well as social science studies of participation

How do you analyse and visualize conversation?

• Conversation map
• Sentiment analysis
• Semantic web / Web 2.0 methods such as word clouds etc
• More traditional methods
• Argument visualization (could form third stage, below)

How do you create a platform structure for deliberation that allows adequate post-conversation analysis to take place?

• Integrating the previous two stages
• Blueprint for successful design
• Potential prototype

With this in mind we discussed the following action points to be undertaken:

1. Produce a literature review of materials relating to interface design and participatory systems
2. Produce very rough thesis chapter structure which Stephen envisioned as having an introduction, 2-3 chapters about interface design, 2-3 chapters about discourse analysis and further chapters about integrating the two.