by Giuseppe Pellegrini
On several occasions in recent years, experts, scientists and communicators have stressed the need to carry out assessment of scientific communication processes. For this reason, many institutions that organize festivals, exhibition, visits and public information campaigns have launched initiatives to test the effectiveness of these processes of public communication of science.
But what are the elements that make an effective evaluation? And how you can develop tools, processes and practices to achieve efficiently a correct assessment?
One of the main goals of the communication assessment is to eliminate or substitute the actions that generate undesirable effects when performing a communication activity. For this reason, the evaluation focus should be on what is working, and what is not, in the ongoing activity.
An example of effective assessment in communication, the so called formative evaluation, through the Web is the integration of technology in university coursework (Bowman 2013). A semester-long wiki project was planned and evaluated using a formative assessment plan. The assessment indicated future structural and communicative modifications, taking into account specific students’ suggestions. An open-ended questionnaire was used to verify content learning, «the effectiveness of producing a better understanding of real-world applications through the wiki project and their perception of the wiki project’s shortcomings and/or challenges».
There is also another important place in which an evaluation is effective, at the end of the communicative process. This is the summative evaluation that should be effective if the needs and the assessment design are aligned establishing the opportunity to monitor differences before and after the communicating intervention.
In many cases is possible to mixed quantitative and qualitative measurements in order to depict a complete frame of results and effects, also for small case initiatives such as one of those organized in informal contexts. This was the case of the Discovery07, a Dutch science party. The Discovery 07 had the main goal to verify a changing in public opinion about the image of science and scientist organizing a “party with a meaning” with live music, live activities of scientific research, cocktails music and other presentations made by young scientists.
Efforts to improve evaluation in public communication of science and technology have extended across multiple facets of the process. Effective evaluation must take into considerations results and effects, that are quite different. So, you can imagine results as products of what a particular communication program envisaged, privileging the point of view of the promoters. Effects, instead, are viewed as changes produces by the communication with a broad focus on all actors involved in the process. This means that it is possible to have different judgments in the two cases, so that good results in terms of results offer no guarantees in terms of effects.
An evaluation plan for a scientific communication activity is based on two main questions: what you want to know and how you will know it. These questions allow us to identify the changes that an activity produces over time. In this way you can select the most sensitive indicators to verify the results and the effects of a communication program. You can then find that the assessment activities is a circular process that allows you to start with some goals, check them with the help of the actors involved collecting their different views and return to plan new activities thanks to the lessons learned with the evaluation.
Effective assessment activities therefore require the ability to control this circular process assigning proper weight to each element.
