Back to the drawing board…

I decided to run some of my thoughts past Professor Ann Macintosh – an expert in argument visualisation and one of the most knowledgeable people in my area of research. Putting to her my thoughts about interface design and its effect on e-participation platform effectiveness, Ann’s immediate reply was to ask “Why is it a PhD and not just something a consultancy could do?“. She was right of course – my ideas had become a little limited in scope and were concentrated a little too much on the practical. Expanding my thoughts, we talked about what I wanted to do: create a method for generating useful information out of the mass of content often contributed online; create a platform to allow users to be fully informed and develop and illustrate useful ideas; evaluate e-participation strategies to discover areas of strengths and weaknesses that could be used to provide improved services in the future. Talking about analysis of content Ann stressed that linguistic analysis is very hard and  it has taken years of research to get to the level I am proposing. Acknowledging that I do not  have expertise in linguistics I described some of the examples where I thought solutions could be created from the “building blocks” of previous research and Ann agreed that building on others work and combining technologies is do-able. Building a solution is good, but needs to be realistic.

Argument visualisation is one area where there is previous work that could be built on and as we talked about ways to investigate the interests that I had outlined, such as evaluation frameworks for e-participation initiatives, Ann highlighted it as an area on which I could create a solution to be used to evaluate and enhance previous work. Looking at a research question such as “The appropriateness of argument visualisation in evaluation of e-participation platforms” I could create a solution to be tested in empirical works with focus groups and against previous evaluations.

Going in to the meeting I had a range of ideas and a strategy for research that was becoming troublesomely unsuitable to a PhD. Coming out of the meeting I had a clearer understanding of where things were going wrong but I needed to have a real think about the steps needed to rectify the situation and get my research plan back on a firm footing.

===================

Some notes on an idea in the resulting thoughts:

“Role of technology in enabling and evaluating e-participation”

• • Designing for deliberation
  • Developing systems for argument mapping
  • Analysing content

Developing tools to analyse and evaluate e-participation technologies:

Tools to ENABLE deliberation/participation

• • Evaluate interface design
  • Create innovative platform (blueprint/prototype)

Tools to ANALYSE deliberation

• • Evaluate input to platforms
  • Summarise input to platforms

Tools to VISUALISE argument/process

• • Enhance tools for argument visualisation
  • Evaluate use in consultation/participation initiative

Examples for thought:

“Take one or more initiatives and analyse in a novel way to show value of new method”
Approach used by Ricky Ohl in his PhD (Knowledge Cartography, 2008, Buckingham-Shum et al, Ch3)

Create an evaluation technique and evaluate different platforms

• • Deliberation assessment
  • Integration assessment
  • Summarising

Advertisement

Developing ideas…

Having looked at a range of case studies and read some of the literature I have tried to formulate some ideas about my research, starting with the major themes that must be present in research about e-participation.

Participation: allowing the public to be involved in decision making and to have their voices heard (and acted upon); helping to educate the public about issues, providing resources and promoting informed opinion.

Deliberation: social and reciprocal conversation between individuals allowing them to illustrate viewpoints and form opinions; can help groups of individuals to reach a consensus.

Collaboration: allowing individuals or groups to directly contribute to a process (in this case policy or decision making); helps to build solutions that represent public views; helps to improve government by harnessing the relevant skills and resources of the public.

Technology: can be used to help create informed opinion (e.g. using argument visualisation); can analyse the level of deliberation in content (e.g. conversation map); can influence deliberation (interface design); can enable collaboration (web 2.0, the semantic web).

The thought that weighs most heavily on me when considering the above themes is that of interface design and its influence on the amount of deliberation, participation and collaboration allowed by a platform. How much of the quality, or lack of quality, found in e-participation efforts is down to technological determinism? Of course, many other factors are also involved: the communities attracted; the subject matter of debate; the resources and information supplied in support of a debate; the institution used (government, “third space” or “citizen generated” platform).

It seems as though, in order to truly analyse the technological interference of platforms on their contents one would have to take into consideration platforms that are comparable and contrasting examples in each of the above categories to get an accurate picture – national/EU government and local government platforms, community groups (geographically localised or vocation/age-specific), citizen spaces and those that Scott Wright calls “third spaces”. Comparable examples in these contrasting categories would need to be analysed for interface and structural characteristics, deliberative and collaborative nature of content and their level of integration in policy making as well as their accessibility and representativeness.

e-Participation case studies

Taking an in-depth look at a few more case studies, my initial list of good examples is dwindling fast. Many have been discontinued and the data is not readily available (TalkSwindon and BBC Action Network, for example) and some are woefully underused (such as HighlandLife). Three more examples are worthy of note here though: Ask Bristol, a local government example of consultation and e-participation; the Communities and Local Government Forum, a UK scoped moderated forum; HeadsUp, a UK-wide youth-oriented debating site.

Bristol City Council has a long record of innovating with regard to public online participation. The website e-participation.net lists their offering as Viewfinder Bristol but the latest incarnation seems to be AskBristol, a wordpress.com site that allows BCC to converse with the public via a forum based platform. Single debates are held at a time with results of previous debates scrutinised and published on the site. Information in the form of text, data, webcasts etc. is presented about different threads within a debate and a space for public contributions can be found beneath. Comments are moderated against a set list of fairly standard rules (e.g. “Don’t be too offensive…”) and a ratings system allows users to show support for comments they like and opposition to those that they don’t. Most popular comments and ideas are easily viewed via a simple rollover. The content is not particularly deliberative, there are no explicit social features of the interface, ratings aside, but there does seem to be a level of integration of content into policy and a good level of feedback. Collated reports are available in pdf format showing responses to each “idea” submitted and summaries of public responses as well as details of how the ideas were integrated into policy. There are also links in each page to other methods of communication such as email, links to surveys or details of offline public consultation. It’s also interesting to see that there is a facility to comment on how the council could engage with the public online better!

The Communities and Local Governments forum is an interesting example of a simple platform as it is heavily moderated – nothing is published without prior approval – and moderation only occurs in office hours. This has the result of limiting active participation, other than individual comments, to those times. The system is not particularly interactive but does allow input to be structure as a reply to previous input and allows discussion threads to be “tagged” by users to help categorise the conversations. The content is surprisingly deliberative for such a strictly controlled environment. It would be interesting to analyse the ethno/demo/sociographic makeup of the community of users to see what effect the barriers to use (moderation and “opening hours”)  have on the participation. It would also be interesting to find out the effect of the moderation on debate. It can only be assumed that moderation silences certain voices but the effect to which this improves or degrades the debate is unknown.

HeadsUp is an interesting site for youngsters that contains a lot of information about how to debate as well as reasons to, and how to, contribute to debate about society. The site employs a simple messageboard platform to debate one theme (with sub categories) at a time. Other than customisable avatars, the system does not provide explicit social features (such as reciprocity) but it does have the novel feature of a board of “heads” – facilitators and moderators that have a presence on the messageboard of each discussion to help the debate move along. The debates seem to be slow to build but I need to investigate further as the current debate is new and closed debates do not seem to be readily available for scrutiny, though in-depth reports do provide a good summary of content.

e-Participation initiatives uncovered

My initial investigations into potential case studies and examples of e-participation initiatives has proved very fruitful indeed. I have been ably assisted in my trawl of the hundreds of examples currently available by the website e-participation.net – a useful database of e-participation initiatives in the UK, Germany and EU. Aiming to “show diverse developments and highlight examples of good practice” as well as trying to “encourage people to join one of these projects or to start their own“, this site is the perfect launching pad for my work. The site allows project teams to upload details of their own e-participation initiatives and allows users to tag and rate the projects. I have been able to look at a number of projects so far, with some posing great potential. There are locally targeted projects, such as TalkSwindon and HighlandLife, as well as UK national initiatives like the BBC Action Network and petitions.pm.gov.uk and EU forums such as legese.org. There are initiatives targeted at specific communities, too, such as LondonTeens, the Northern Ireland Youth Forum and HeadsUp. All of these examples, and many more warrant close inspection, a process that I am currently engaged in. I am hopeful that a range of case studies will prove to be suitable to the kind of research I have in mind. I am looking at the structure of the content, as allowed by the interface of the web site used to present the initiative as well as topic of conversation and community behaviour.

Another interesting example that I stumbled across is the UK Police initiative to harvest ideas about cost cutting and efficiency savings. Aimed at internal personnel but available to all via the internet this site has a similar structure to recent government initiatives such as SpendingChallenge and YourFreedom websites in that it allowed participants to publish ideas and comment on the ideas published by others. Like the other initiatives, the system suffered from submission of spam postings as well as inflammatory, and possibly deliberately overstated,  comments and at first glance a lack of deliberation as comments seemed to be individual statements of opinion rather than interactions of discourse. Unlike the the other initiatives, the police review seeded conversations with initial questions and did not allow voting on comments. The example could prove useful if comparable systems are identified as it was aimed at a particular user group which may behave in contrast to the general public when participating.

Following on from the thoughts I had about Scott Wright’s research about “Third Spaces“, I researched a number of such for comments and interaction facilities. One particularly interesting example was found at Amazon.co.uk where it looked as though the facilities for discussing products had been adapted to become general discussion boards, categorised into a number of different themes, including a very well used politics section. The interface was designed to allow interaction through reply facilities and the social “path” of a conversation was recorded through username, date of posting and previous message responded to. There was also a facility to mark whether a post “adds to the discussion”, a measure of usefulness of contributions and, in aggregate, quality of the debate. These marks are also recorded against a user profile, raising the potential of a profile “respect” model. On the whole, the parts of the forum that I looked at seemed remarkably deliberative and discussions existed about particularly topical themes (for instance “Will a degree bring graduates enough income increase to pay for the degree?). It will be interesting to see if these “third spaces” can provide successful models for deliberative interface design or whether other factors are involved in fostering deliberation, such as the communities drawn to different platforms.

Text analysis and website archiving

I have been keeping an eye out for new and useful technologies that might be important in my studies and have recently come across a couple that are worthy of note:

DiscoverText
Launched just this month,  DiscoverText – a product of Texifter, a spin-out company based on text analysis research by Dr Stuart Shulman of the University of Massachusetts –  is a web based text analysis tool which allows users to upload or identify text data sources to be analysed using a range of automated and “human-in-the-loop” processes. There is a simple free service and more complex analysis in paid-for solutions. The software can be used to capture live data from facebook, twitter or any RSS feed and can also work with uploaded files such as large volumes of PDF or Microsoft Office documents or an email archive. In this way, users can create an online archive of text data which can be analysed at will. The software combines automated text analysis with expert analysis by allowing collaboration between networks of trusted peers. Initially, comments can be grouped into “clusters” of similar contextual meaning and common themes identified. Some of Shulman’s recent research focuses on the detection of threats and novel ideas communicated in public comments, blogs and other media, which raises the prospect of using the system to analyse for the presence of formulated public opinion in text content relevant to my studies. I have yet to identify the exact mechanisms and theories used in this product but hopefully some study of Shulman’s work will shed some light on the issue.

WinHTTrack Website Copier
This open source free (GPL) download allows users to store entire copies of websites, archived in a clear storage structure, for future offline browsing or analysis. Rather than simply requesting pages and storing them in a structure exactly like that of the target site the software can also create a copy of dynamic data served from databases onto a web site. Every hyperlink is investigated and the html returned is stored as a page and becomes accessible by the URL, query string and all. This is a valuable tool for my research as it enables me to capture entire e-participation initiatives and store them locally before they disappear when their operational period comes to an end.

Colin Alexander: Public Diplomacy in Central America

The second part of this week’s PhD seminar was student led, with Colin Alexander presenting his research to us. The following is a summary of my notes from the presentation. Colin’s research analyses the tussle between China and Taiwan in public diplomacy in Central America. He first gave us the context of the research providing an overview of the political and diplomatic status of China and Taiwan, highlighting the critical nature of Central American countries as diplomatic allies of Taiwan and describing the diplomacy initiatives of China and Taiwan in the region. Colin then described his research – a media content analysis which measured media sentiment in Costa Rica, El Salvador and Guatemala towards China and Taiwan over a period of time in which political change took place – which was designed to answer questions such as:

• How successful are Chinese/Taiwanese diplomacy policies in affecting public opinion?
• Is local (Central American) media a more powerful influence on  public opinion?

The research was based partly on a previous study which measured media sentiment in Europe before and after the introduction of the Euro as a single currency. In a specific example, Colin examined a number of newspapers (available online) that are popular in the region for six months prior to and six months after the point in 2007 when Costa Rica switched from recognising Taiwan as a nation to recognising China (a mutually exclusive pair of options in international diplomacy). Selecting articles that contained the word “China” in the title or opening paragraph, Colin manually assessed the sentiment towards China and aspects of Chinese life and policy by reading the article objectively and applying a seemingly very light evaluation framework, assigning a score from 1 (very negative) through to 5 (very positive) for sentiment towards China. The analysis showed that, while the number of articles varied largely throughout the study period, indicating increased interest and debate surrounding China, the overall sentiment of the articles did not vary markedly, an insignificant trend towards a more positive sentiment was recorded. There was, however,  noticeable and contrasting variations among articles grouped by topic. Articles with “political” and economic content became significantly more negative in sentiment towards China in the second half of the year (after the diplomatic change had taken place) while articles about foreign relations, Chinese society and human rights, amongst other topics, became significantly more positive. The research is not yet concluded but the methodology seems to be uncovering some interesting trends in media sentiment which may be of use when evaluating the public diplomacy efforts of China and Taiwan.

Colin acknowledged the need for objectivity in sentiment analysis of articles and the challenges presented by his approach in terms of consistency. No automatic, computer-driven method of sentiment analysis was employed, in contrast to some of the studies I have been looking at in relation to my own research. This did allow Colin to make decisions about the objectivity of the articles themselves, such as tangential comments about Chinese culture that were not related to the article but seemed to have been put in to increase the positivity or negativity of an article, without adding to the article itself. It would be interesting to apply some sort of evaluation framework to the sentiment analysis to see how successful the method was in producing accurate and consistent sentiment scores, particularly in contrast to other, possibly computer-driven methods – not simply to evaluate the manual method applied by Colin, but to see whether the latest automated sentiment analysis algorithms can perform at an accuracy level similar to that of the objective human mind.

Upgrade process

Part 1 of this weeks PhD seminar focussed on the upgrade (or transfer) process that we have to go through, in my my case within 24 months – by September 2012. The discussion started with brief introductions and one-line description of our research (there were two new students to be introduced) which which led into an illustration of the need to eventually be able to define research concisely and clearly. At present, my one line was quite long and in very broad terms – clearly a little more definition is needed but I am confident that will come with a bit more work.

The transfer is a formal requirement, embedded in University structure and somewhat “out of the hands” of ICS staff. It is a serious (and scary) part of the PhD process and decides whether the PhD goes ahead  or not. The process reflects the viva in that a document (in this case 10,000words)  is prepared and examined and then the candidate meets a board of examiners in order to be questioned. The panel is looking for a project that looks like a PhD:

• Original
• Substantial
• Feasible
• with a capable candidate

There are four possible outcomes from the meeting with the board:

• The PhD proceeds (and will therefore almost certainly be a success)
• The document is given back for revisions (not an uncommon outcome)
• An MPhil is recommended instead of PhD as the project lacks something
• The board recommends that the candidate withdraw

The first year (or possibly up to 2 in my case) can be an unsettling time of constant change as the candidate builds up to the transfer, attempting to develop, narrow and finalise research questions and decide upon a title. I have the added complexity of a part time teaching job with its own unsettling characteristics – learning about the university teaching environment, producing materials, supervising students and learning to teach via the ULTA-2. However, this has a positive side as half of my time can be used to escape from the PhD and concentrate on other issues, returning to research questions and titles in a better frame of mind. I am keen to not take six years over this PhD, even with the teaching responsibilities and at this point it seems that the part time approach may actually be something of a help to productivity, even if it does cut down the amount of hours I can spend researching and writing. Whether I keep that opinion when closer to deadlines for the transfer is yet to be seen…

Intellectual Craft, David Hesmondhalgh

The latest ICS PhD seminar focussed on the development of research questions and the theoretical background in which they sit. Dave started by discussing the concepts of theory, method and technique and brought our attention to Robert Alford’s  1998 book The Craft of Inquiry which discusses the principles and values underlying methods as well as approaches to both separate and integrate theory and method. Dave described and contrasted Grand Theory (C Wright Mills, theory without evidence) and Abstracted Empiricism (Paul Lazarsfield, method without theory) as well as Merton’s Middle Range Theory and described how Alford dismissed the latter as a solution to the schism of the two former due to its lacking of fundamental theory of modern society, such as the effects of power and inequality.

Dave went on to show how such theory of modern society can be used in the formulation of research questions as basic ideas created by personal experience, reading and social, moral or political concerns can be translated into the formulation of research questions – a process Alford describes as “The Craft of Inquiry“. Important characteristics of research questions were highlighted: every word must count; questions must point two ways at once (justified by a theoretical framework and requiring empirical evidence to be answered). Dave posted key questions to be addressed when designing research:

• What am I interested in explaining?
• Which are central and which are peripheral aspects?
• How can relevant theoretical chains be justified?
• What are necessary and possible kinds of evidence available?
• Why do the project at all?

Also noted was the importance of a rolling, iterative, recursive structure of research design and the importance of combining theoretical and empirical trends of analysis and “moving between them” while maintaining some stability in the process.

Some of these thoughts about theory have been fascinating to me. Having come from a practical/industrial/scientific background I entered into my studies with a somewhat abstract empiricist and technological deterministic viewpoint.  Some of the intellectual problems that I immediately encountered stemmed from my lack of knowledge of and exposure to the theoretical platforms of social science. Even before discovering some of the overarching themes and theoretical structures I was forced to address some of my problems through theoretic analysis garnered from the work of others in the literature: differentiating between theories of deliberation, such as the value of discourse based upon its structure and the value of democratic input based upon its deliberative structure, with methods of analysing and measuring deliberation or designing platforms to allow and encourage and capture deliberation. The dichotomy of technological determinism and social theory is exceptionally relevant as a background to my work and, as pointed out by Dave, I will also need to draw distinctions between the intensive and extensive nature of both my own work and the literature which informs it. As I develop research questions and plan my next few years of work the words of Alford, and indeed Hesmondhalgh, will be a very firm foundation on which to stand.

The real first supervision meeting

Having learnt a lot about the work done in argument visualisation as well as definition, creation and importance of deliberation in policy creation, I met with Stephen again, this time able to really talk about moving forward with the PhD. We talked about the work of Sack and Donath, my opinions about its usefulness and its specificity to USENET, as well as the many different aspects drawn together by Sack which could be used on different platforms: thesauri, social networking based upon text content rather than headers.

We talked about the level of deliberative quality of forums as discussed by Sack and the ways in which the form of deliberation may be different in different subject areas. We discussed Todd Graham’s study of deliberation on BBC, Guardian Unlimited and Wifeswap forum in which he showed that deliberation was not just confined to the “serious” subjects. We also discussed how the measurement of deliberation was dependent upon the way that we define deliberation and the impact of that on any study of deliberation that I may carry out. I talked about possible studies of different user groups using similar platforms and Stephen agreed that it was important not just to concentrate on technical constraints on deliberation but look for other possible impacting factors.

We talked about the potential to produce a blueprint for a deliberative participation platform, some way down the line, as a way to create content worthy of PhD, even though it is rare to get such practical applications in a thesis. The appropriateness of this will only become apparent after more research into existing and upcoming platforms and their relative strengths and weaknesses.

Moving forward, Stephen suggested that I start to identify systems and groups that are used for potential deliberative input and non-systematically analyse them (close read) for applicable qualities. We discussed systems such as cohere and debatewise.org. We talked about the usefulness of caching / archiving websites for future use – something I am keen to look into as I don’t want to miss any vital examples! I will also be exploring norms and expectations of deliberation in order to start to think about the definition and characteristics of deliberation against which I will assess participation strategies and platforms.

Engaging critically with literature, Giles Moss

In PhD seminar #2, Dr Giles Moss spoke to us about the importance of engaging with literature in a critical way and about the importance of a literature review as well as its purpose and place in a document, particularly the use of the literature review  to “make space” for a PhD, justifying research and displaying its uniqueness and originality. Giles outlined the basics such as how key texts must be addressed and engagement must be critical. He illustrated techniques used for the task – language used, taking sides, agreeing, disagreeing, pointing out holes – as well as pointing out their merits and failings, strengths and weaknesses.  We studied a range of ICS PhD theses which used a variety of approaches to literature review in terms of chapter structure and integration into the document as well as the writing style used.